[1/1] Cn10K crypto Tests: Added Marvell Cn10K specific crypto Tests to TestCryptoPerfCryptodevPerf

Message ID 20241030164419.3164992-1-gpalanethra@marvell.com (mailing list archive)
State New
Headers
Series [1/1] Cn10K crypto Tests: Added Marvell Cn10K specific crypto Tests to TestCryptoPerfCryptodevPerf |

Commit Message

Gnanesh Kambalu Palanethra Oct. 30, 2024, 4:44 p.m. UTC
Changes included in this patch:
    -- New TestCases are added for Cn10K CPT Hardware crypto accelerator.
    -- when Testing newly added Testcases it is found Necessary to make Below Changes
          1> tests/cryptodev_common.py
               --> bind_qat_device function is updated to generate VFs for the given crypto_dev_id
               --> added New function bind_mrvl_devices to accept list of PCI Ids and bind it to vfio_pci driver.
          2> framework/crb.py
               --> updated pci_devices_information_uncached_linux Method to handle additional Nic Speeds for Cavium,
                   as it was restricting only to 1 GIG NIC speed
          3> framework/settings.py
               --> Changed the default Cavium NIC driver to rvu_nicpf.
          4> nics/net_device.py
               --> added Missing expect object
          5> conf/crypto_perf_cryptodev_perf.cfg
               --> added Marvell Cn10K configs for crypto_perf_cryptodev_perf
          6> framework/ssh_pexpect.py
               --> output.replace command's output is not set before returning from get_output_all method
                   hence Unnecessarily returning Prompt along with the output
          7> tests/TestSuite_crypto_perf_cryptodev_perf.py
               -->  added Marvell CN10K Testcases
             Updated Below Private functions
                6.1> _run_crypto_perf() --> receives additional KW arguments
                6.2> _parse_output()  ---> Updated to handle Marvell Cn10K app command output
                6.3> _run_crypto_perf_throughput() --> receives additional KW arguments


Signed-off-by: Gnanesh <gpalanethra@marvell.com>
---
 conf/crypto_perf_cryptodev_perf.cfg           | 849 ++++++++++++++++++
 framework/crb.py                              |   6 +-
 framework/settings.py                         |   4 +-
 framework/ssh_pexpect.py                      |   2 +-
 nics/net_device.py                            |   3 +
 tests/TestSuite_crypto_perf_cryptodev_perf.py | 322 ++++++-
 tests/cryptodev_common.py                     |  11 +-
 7 files changed, 1172 insertions(+), 25 deletions(-)
  

Comments

David Marchand Oct. 31, 2024, 10:09 a.m. UTC | #1
Hello Gnanesh,

On Wed, Oct 30, 2024 at 5:44 PM Gnanesh <gpalanethra@marvell.com> wrote:
>
>     Changes included in this patch:
>     -- New TestCases are added for Cn10K CPT Hardware crypto accelerator.
>     -- when Testing newly added Testcases it is found Necessary to make Below Changes
>           1> tests/cryptodev_common.py
>                --> bind_qat_device function is updated to generate VFs for the given crypto_dev_id
>                --> added New function bind_mrvl_devices to accept list of PCI Ids and bind it to vfio_pci driver.
>           2> framework/crb.py
>                --> updated pci_devices_information_uncached_linux Method to handle additional Nic Speeds for Cavium,
>                    as it was restricting only to 1 GIG NIC speed
>           3> framework/settings.py
>                --> Changed the default Cavium NIC driver to rvu_nicpf.
>           4> nics/net_device.py
>                --> added Missing expect object
>           5> conf/crypto_perf_cryptodev_perf.cfg
>                --> added Marvell Cn10K configs for crypto_perf_cryptodev_perf
>           6> framework/ssh_pexpect.py
>                --> output.replace command's output is not set before returning from get_output_all method
>                    hence Unnecessarily returning Prompt along with the output
>           7> tests/TestSuite_crypto_perf_cryptodev_perf.py
>                -->  added Marvell CN10K Testcases
>              Updated Below Private functions
>                 6.1> _run_crypto_perf() --> receives additional KW arguments
>                 6.2> _parse_output()  ---> Updated to handle Marvell Cn10K app command output
>                 6.3> _run_crypto_perf_throughput() --> receives additional KW arguments
>
>
> Signed-off-by: Gnanesh <gpalanethra@marvell.com>

This mailing list is not used anymore.
If you want to reach out to current DTS maintainers, you should send
mail to dev@dpdk.org (I Cc'd Paul and Patrick).
  
Patrick Robb Oct. 31, 2024, 10:15 p.m. UTC | #2
Hi Gnanesh, thanks for sending this. My name is Patrick Robb, and I'm a CI
Testing Lab manager at the DPDK Community Lab hosted at UNH, and a
maintainer for the "new" DTS framework which exists within the DPDK repo
and should be replacing the legacy DTS project in the long term(what this
patch is for). We do have a Marvell CN10K board at the lab, and I would
love to begin running some crypto testing from that if possible (so, thanks
again for this series).

So, most likely we will want to provide a review from UNH and test this, so
that it can be merged to DTS (technically I was granted the authority to
apply patches to the legacy repo).

I am adding Cody to this thread as I think he can assist me on this. Cody I
am going to provide some comments inline as an initial review for this
series, and then we should apply this series locally on the traffic
generator/test engine for the CN10K (Intel-1 in our lab), dry run the test
which Gnanesh has added, and report back to him. If all goes well, it
should be possible to merge.

Long term, we will want to circle back and add this test to the new DTS
framework, but it is not ready for this test yet, as currently new DTS is
testpmd only - there is not cryptodev support. Although, adding such
support to new DTS is something I would love to work with you on in the
next year if you have time available for development towards DPDK testing!
Anyways, comments inline:

On Wed, Oct 30, 2024 at 12:44 PM Gnanesh <gpalanethra@marvell.com> wrote:

>     Changes included in this patch:
>     -- New TestCases are added for Cn10K CPT Hardware crypto accelerator.
>     -- when Testing newly added Testcases it is found Necessary to make
> Below Changes
>           1> tests/cryptodev_common.py
>                --> bind_qat_device function is updated to generate VFs for
> the given crypto_dev_id
>                --> added New function bind_mrvl_devices to accept list of
> PCI Ids and bind it to vfio_pci driver.
>           2> framework/crb.py
>                --> updated pci_devices_information_uncached_linux Method
> to handle additional Nic Speeds for Cavium,
>                    as it was restricting only to 1 GIG NIC speed
>

Thanks, I also made this change on our local DTS project for cn10k, but the
patch was not submitted to the "legacy DTS" mailing list here as it was
unmaintained (and still is, to David's point). But, if we can do a
review it is good to get these things into mainline. Maybe I can rebase our
local patch on yours if there were any other changes, and get feedback for
cn10k specific questions.

I have a hard time understanding how this 10G NIC speed requirement got
added in the first place as it was quite peculiar and confused me for a
time as it was dropping the ports list! I'm sure you had a similar
experience... :)


>           3> framework/settings.py
>                --> Changed the default Cavium NIC driver to rvu_nicpf.
>

Thanks, we are also running from this change at the Community Lab. But, I
thought you said above you are creating VFs from the bind_qat_device
function. So, does rvu_nicvf need to be added as well?


>           4> nics/net_device.py
>                --> added Missing expect object
>           5> conf/crypto_perf_cryptodev_perf.cfg
>                --> added Marvell Cn10K configs for
> crypto_perf_cryptodev_perf
>           6> framework/ssh_pexpect.py
>                --> output.replace command's output is not set before
> returning from get_output_all method
>                    hence Unnecessarily returning Prompt along with the
> output
>           7> tests/TestSuite_crypto_perf_cryptodev_perf.py
>                -->  added Marvell CN10K Testcases
>

Again we will apply these, give them a run, and report back.


>              Updated Below Private functions
>                 6.1> _run_crypto_perf() --> receives additional KW
> arguments
>                 6.2> _parse_output()  ---> Updated to handle Marvell Cn10K
> app command output
>                 6.3> _run_crypto_perf_throughput() --> receives additional
> KW arguments
>


I don't see any modifications for the build process. Is there a need to
include "-Dplatform=cn10k" when setting up the build from meson, or is the
default build fine?



>
>
> Signed-off-by: Gnanesh <gpalanethra@marvell.com>
> ---
>  conf/crypto_perf_cryptodev_perf.cfg           | 849 ++++++++++++++++++
>  framework/crb.py                              |   6 +-
>  framework/settings.py                         |   4 +-
>  framework/ssh_pexpect.py                      |   2 +-
>  nics/net_device.py                            |   3 +
>  tests/TestSuite_crypto_perf_cryptodev_perf.py | 322 ++++++-
>  tests/cryptodev_common.py                     |  11 +-
>  7 files changed, 1172 insertions(+), 25 deletions(-)
>
> diff --git a/framework/crb.py b/framework/crb.py
> index 9e3b0a58..2fe9d147 100644
> --- a/framework/crb.py
> +++ b/framework/crb.py
> @@ -374,15 +374,15 @@ class Crb(object):
>                  pass
>
>          for i in range(len(match)):
> -            # check if device is cavium and check its linkspeed, append
> only if it is 10G
> +            # for cavium supported link speed specfied under linkspeeds
>              if "177d:" in match[i][1]:
> -                linkspeed = "10000"
> +                linkspeeds = ["40000", "10000", "25000", "50000",
> "100000"]
>

Honestly I don't understand why we have this condition at all. Other
devices in DTS do not have this check where the portlinks can be dropped in
crbs.py. What do you think about updating the accepted linkspeeds vs just
removing this step entirely? Can we expect additional linkspeeds to be
added for 177d in the future, and have this problem crop up again for a new
linkspeed, or is this list of speeds "final" for 177d?


>                  nic_linkspeed = self.send_expect(
>                      "cat /sys/bus/pci/devices/%s/net/*/speed" %
> match[i][0],
>                      "# ",
>                      alt_session=True,
>                  )
> -                if nic_linkspeed.split()[0] == linkspeed:
> +                if nic_linkspeed.split()[0] in  linkspeeds:
>                      self.pci_devices_info.append((match[i][0],
> match[i][1]))
>              else:
>                  self.pci_devices_info.append((match[i][0], match[i][1]))
> diff --git a/framework/settings.py b/framework/settings.py
> index 1a561dda..b2c371f8 100644
> --- a/framework/settings.py
> +++ b/framework/settings.py
> @@ -166,8 +166,8 @@ DRIVERS = {
>      "cavium_a034": "thunder-nicvf",
>      "cavium_0011": "thunder-nicvf",
>      "IXGBE_10G-X550EM_X_SFP": "ixgbe",
> -    "cavium_a063": "octeontx2-nicpf",
> -    "cavium_a064": "octeontx2-nicvf",
> +    "cavium_a063": "rvu_nicpf",
> +    "cavium_a064": "rvu_nicvf",
>

Okay disregard my comment from above about nicvf, I see you've added it.
Please do update the commit message to reflect that this is included. And,
while you're at it, you can update the commit message to accord with the
style guidelines for submitting DPDK patches as seen here on sections 9.6
and 9.7 https://doc.dpdk.org/guides/contributing/patches.html

:)

     "ICE_100G-E810C_QSFP": "ice",
>      "ICE_25G-E810C_SFP": "ice",
>      "ICE_25G-E823C_QSFP": "ice",
> diff --git a/framework/ssh_pexpect.py b/framework/ssh_pexpect.py
> index 2132c066..a96087d2 100644
> --- a/framework/ssh_pexpect.py
> +++ b/framework/ssh_pexpect.py
> @@ -186,7 +186,7 @@ class SSHPexpect:
>
>      def get_output_all(self):
>          output = self.session.before
> -        output.replace("[PEXPECT]", "")
> +        output = output.replace("[PEXPECT]#", "")
>

So, this is just removing the leading "#" from the output? I'm not sure we
should be making styling changes for a "deprecated" repository. If I look
at this example output, from one of the CI runs of this testsuite on a
quickassist card:

# Device 1 on lcore 40

# total operations: 30000000
# Buffer size: 1024
# Burst size: 32
#     Number of bursts: 937500
#
#                 Total   Average   Maximum   Minimum
#  enqueued    30000000        32        32        32
#  dequeued    30000000        32        32         1
#    cycles 91892876610      3063    304625       485
# time [us]  3675715064   122.520 12185.000    19.400

I imagine the DTS maintainers were aware of this and thought it was the
most clear way to print the logs. I don't think we need to override that
decision now.


>          return output
>
>      def close(self, force=False):
> diff --git a/nics/net_device.py b/nics/net_device.py
> index 0f9c1af4..dd0d0e92 100644
> --- a/nics/net_device.py
> +++ b/nics/net_device.py
> @@ -43,6 +43,7 @@ class NetDevice(object):
>          if not isinstance(crb, Crb):
>              raise Exception("  Please input the instance of Crb!!!")
>          self.crb = crb
> +        self.__send_expect = self.crb.send_expect
>          self.domain_id = domain_id
>          self.bus_id = bus_id
>          self.devfun_id = devfun_id
> @@ -727,6 +728,8 @@ class NetDevice(object):
>              "/sys/bus/pci/devices/%s:%s:%s" % (domain_id, bus_id,
> devfun_id),
>              vf_reg_file,
>          )
> +        self.__send_expect("echo 0 > %s" %
> +                           (vf_reg_path), "# ")
>

So, the pre-existing VFs need to be destroyed before a new set of VFs can
be created? Do you know if this is Marvell specific, or a universal
behavior?


>          self.__send_expect("echo %d > %s" % (int(vf_num), vf_reg_path),
> "# ")
>
>      def generate_sriov_vfs_linux_igb_uio(self, domain_id, bus_id,
> devfun_id, vf_num):
> diff --git a/tests/TestSuite_crypto_perf_cryptodev_perf.py
> b/tests/TestSuite_crypto_perf_cryptodev_perf.py
> index a3f48eee..b435cefa 100644
> --- a/tests/TestSuite_crypto_perf_cryptodev_perf.py
> +++ b/tests/TestSuite_crypto_perf_cryptodev_perf.py
> @@ -59,14 +59,74 @@ class TestCryptoPerfCryptodevPerf(TestCase):
>                  "# ",
>                  5,
>              )
> -
> -        cc.bind_qat_device(self, "vfio-pci")
> +        if self.nic == "cavium_a063":
> +            count = len(self.get_suite_cfg().get('l').split(",")) - 1
> +            cc.bind_qat_device(self, "vfio-pci", generate_vfs=True,
> vf_count=count)
> +        else:
> +            cc.bind_qat_device(self, "vfio-pci")
>

If we are going to use this function for non-QAT devices, can we please
update the function name to prevent confusion? It looks like it's a short
list:

probb@d121003:~/repos/dts> grep -r "bind_qat_d" .
./tests/TestSuite_compressdev_qat_pmd.py:        cc.bind_qat_device(self,
self.drivername)
./tests/TestSuite_crypto_perf_cryptodev_perf.py:
 cc.bind_qat_device(self, "vfio-pci")
./tests/TestSuite_fips_cryptodev.py:        cc.bind_qat_device(self,
"vfio-pci")
./tests/TestSuite_ipsec_gw_cryptodev_func.py:
 cc.bind_qat_device(self, self.drivername)
./tests/TestSuite_l2fwd_cryptodev_func.py:        cc.bind_qat_device(self,
self.drivername)
./tests/TestSuite_virtio_ipsec_cryptodev_func.py:
 cc.bind_qat_device(self, self.drivername)
./tests/TestSuite_virtio_perf_cryptodev_func.py:
 cc.bind_qat_device(self, self.drivername)
./tests/TestSuite_virtio_unit_cryptodev_func.py:
 cc.bind_qat_device(self, self.drivername)
./tests/compress_common.py:def bind_qat_device(test_case, driver="igb_uio"):
./tests/cryptodev_common.py:def bind_qat_device(test_case,
driver="igb_uio"):


>          src_files = ["dep/test_aes_cbc.data", "dep/test_aes_gcm.data"]
>          self.dut_file_dir = "/tmp"
>          for file in src_files:
>              self.dut.session.copy_file_to(file, self.dut_file_dir)
>
>      def tear_down_all(self):
> +        if self.nic == "cavium_a063":
> +            cc.bind_qat_device(self, "vfio-pci", generate_vfs=True,
> +                               vf_count=2)
> +            import pandas as pd
>

I don't have a problem with this, but I'm just wondering why this is
imported here as opposed to the top of the file?


> +            tuples = [('TestCase',), ('performance', 'failed_enq'),
> +                      ('performance', 'failed_deq'),
> +                      ('performance', 'throughput_mops'),
> +                      ('performance', 'cycle_buf'),
> +                      ('performance', 'throughput', 'value'),
> +                      ('performance', 'throughput', 'delta'),
> +                      ('parameters', 'core_num/thread_num'),
> +                      ('parameters', 'frame_size'),
> +                      ('parameters', 'burst_size'),
> +                      ('parameters', 'total_ops'), ('status',)]
> +            mindex = pd.MultiIndex.from_tuples(tuples)
> +            index = 0
> +            df = pd.DataFrame([], columns=mindex)
> +            for tname, data in self._perf_result.items():
> +                for dt in data:
> +                    for key, value in dt.items():
> +                        df.loc[index, 'TestCase'] = tname
> +                        if key == 'status':
> +                            df.loc[index, (key)] = value
> +                            continue
> +                        for item in value:
> +                            if item['name'] == 'throughput':
> +                                df.loc[index, (key, item['name'],
> 'value')] = item['value']
> +                                df.loc[index, (key, item['name'],
> 'delta')] = item['delta']
> +                            else:
> +                                df.loc[index, (key, item['name'])] =
> item['value']
> +                    index += 1
> +            perf_xl = self.logger.log_path + "/" +
> "perf_cryptodev_result.xls"
> +            writer = pd.ExcelWriter(perf_xl, engine='xlsxwriter')
> +            wb  = writer.book
> +            df.to_excel(writer, sheet_name='Sheet1')
> +            ws = writer.sheets["Sheet1"]
> +            merge_format = wb.add_format({"bold": 1,"border": 1,"align":
> "center",
> +                                          "valign": "vcenter"})
> +            ws.merge_range('B1:B3', "TestCase", merge_format)
> +            ws.merge_range('C2:C3', "failed_enq", merge_format)
> +            ws.merge_range('D2:D3', "failed_deq", merge_format)
> +            ws.merge_range('E2:E3', "throughput_mops", merge_format)
> +            ws.merge_range('F2:F3', "throughput_mops", merge_format)
> +            ws.merge_range('I2:I3', "core_num/thread_num", merge_format)
> +            ws.merge_range('J2:J3', "frame_size", merge_format)
> +            ws.merge_range('K2:K3', "burst_size", merge_format)
> +            ws.merge_range('L2:L3', "total_ops", merge_format)
> +            ws.merge_range('M1:M3', "status", merge_format)
> +
> +            format1 = wb.add_format({"bg_color": "#C6EFCE", "font_color":
> "#006100", "border": 1})
> +            format2 = wb.add_format({"bg_color": "#FFC7CE", "font_color":
> "#9C0006"})
> +            ws.conditional_format("B1:M3", {"type": "cell", "criteria":
> ">=", "value": 0 , 'format': format2})
> +            condition = 'B5:M%s'%(5+len(df)-1)
> +            ws.conditional_format(condition, {'type': 'no_blanks',
> 'format':format1})
> +            ws.autofit()
> +
> +            wb.close()
>          if self._perf_result:
>              with open(
>                  self.logger.log_path + "/" +
> "perf_cryptodev_result.json", "a"
> @@ -175,12 +235,197 @@ class TestCryptoPerfCryptodevPerf(TestCase):
>      def test_scheduler_zuc(self):
>          self._run_crypto_perf_throughput()
>
> +    # Marvell CN10K Testcases
> +    def _crypto_func_cn10k(self):
> +        """Functional tests Helper
> +        Returns:
> +           None
> +        """
> +        self._run_crypto_func()
> +
> +    def _crypto_perf_cn10k(self):
> +        """Perf tests Helper function
> +        Returns:
> +           None
> +        """
> +        self._run_crypto_perf_throughput(expected="# ",
> +                                         trim_whitespace=False)
> +    def test_perf_mrvl_aes_cbc(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_aes_gcm_encrypt(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_aes_cbc_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_aes_sha2_256_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_zuc_eea3_cipher_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_zuc_eia3_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_aes_gmac_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_null_cipher_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_snow3g_uea2_cipher_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_kasumi_f8_cipher_only(self):
> +        self._crypto_perf_cn10k() #F
> +
> +    def test_perf_mrvl_snow3g_uia2_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_kasumi_f9_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha1_hmac_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha1_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_aes_ctr_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_aes_ctr_snow_3g_uia2_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_aes_cmac_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_3des_cbc_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha1_hmac_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha2_224_hmac_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha2_256_hmac_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha2_384_hmac_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_aes_ctr_snow_3g_uia2_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_aes_ctr_zuc_eia3_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_aes_ctr_aes_cmac_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_zuc_eea3_snow3g_uia2_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_zuc_eea3_zuc_eia3_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_zuc_eea3_aes_cmac_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_snow3g_uea2_snow3g_uia2_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_snow3g_uea2_zuc_eia3_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_snow3g_uea2_aes_cmac_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_md5_hmac_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_aes_ctr_cipher_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_3des_cbc_cipher_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha2_224_hmac_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha2_256_hmac_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha2_384_hmac_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha2_512_hmac_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_aes_xts__cipher_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_des_cbc_cipher_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_md5_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha512_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha384_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha256_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha224_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_snow3g_uea2_snow3g_uia2(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha1_hmac_armv8(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha2_hmac_armv8(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_mrvl_ptest(self):
> +        self._crypto_func_cn10k()
> +
> +    def test_mrvl_burst(self):
> +        self._crypto_func_cn10k()
> +
> +    def test_mrvl_devtype(self):
> +        self._crypto_func_cn10k()
> +
> +    def test_mrvl_crypto_sha1_hmac_armv8(self):
> +        self._crypto_func_cn10k()
> +
> +    def test_mrvl_chacha20_poly1305_aead(self):
> +        self._crypto_func_cn10k()
> +
> +    def test_mrvl_crypto_sha2_hmac_armv8(self):
> +        self._crypto_func_cn10k()
> +
> +    def test_mrvl_buffer(self):
> +        self._crypto_func_cn10k()
> +
>      # Private functions
>      def _run_crypto_func(self):
> +        """ Runs Crypto functional test.
> +        Returns:
> +            None
> +        """
> +
>          if cc.is_test_skip(self):
>              return
> -
> -        cores = ",".join(self.dut.get_core_list("1S/2C/1T"))
> +        core_list = self.get_suite_cfg().get('core_list',"1S/2C/1T")
> +        cores = ",".join(self.dut.get_core_list(core_list))
>          config = {"l": cores}
>          devices = self._get_crypto_device(1)
>          if not devices:
> @@ -206,12 +451,22 @@ class TestCryptoPerfCryptodevPerf(TestCase):
>
>          out = self.dut.send_command(
>              "cat %s/%s.txt" % (self.dut_file_dir, self.running_case), 30
> -        )
> -
> +        ).strip()
>          self.verify("Error" not in out, "Test function failed")
>          self.verify("failed" not in out, "Test function failed")
> -
> -    def _run_crypto_perf(self):
> +        assert (out != ""), "No output"
> +
> +    def _run_crypto_perf(self, **kwargs):
> +        """ Runs Crypto Performance test.
> +        Args:
> +          **kwargs:
> +              expected: send_expect match prompt
> +              trim_whitespace: trime whitespace from command output
> +        Returns:
> +            None
> +        """
> +        expected = kwargs.get("expected", "#")
> +        trim_whitespace = kwargs.get("trim_whitespace", True)
>          if cc.is_test_skip(self):
>              return "skip"
>
> @@ -222,13 +477,14 @@ class TestCryptoPerfCryptodevPerf(TestCase):
>              return "skip"
>
>          eal_opt_str = cc.get_eal_opt_str(self, devices)
> -        crypto_perf_opt_str = self._get_crypto_perf_opt_str()
> +        crypto_perf_opt_str = self._get_crypto_perf_opt_str(**kwargs)
>
>          cmd_str = cc.get_dpdk_app_cmd_str(
>              self._app_path, eal_opt_str, crypto_perf_opt_str
>          )
>          try:
> -            out = self.dut.send_expect(cmd_str, "#", 600)
> +            out = self.dut.send_expect(cmd_str, expected, 600,
> +                                       trim_whitespace=trim_whitespace)
>          except Exception as ex:
>              self.logger.error(ex)
>              raise ex
> @@ -237,25 +493,44 @@ class TestCryptoPerfCryptodevPerf(TestCase):
>
>          return results
>
> -    def _get_crypto_perf_opt_str(self, override_crypto_perf_opts={}):
> +    def _get_crypto_perf_opt_str(self, **kwargs):
> +        """get crypto perf app option string
> +        args:
> +            **kwargs:
> +                override_crypto_perf_opts- Suite/Case specfic perf
> +                app config to overwrite defaults
> +        Returns:
> +            crypto perf option string
> +        """
> +        override_crypto_perf_opts =
> kwargs.get("override_crypto_perf_opts", {})
>          return cc.get_opt_str(
>              self, self._default_crypto_perf_opts,
> override_crypto_perf_opts
>          )
>
>      def _parse_output(self, output):
>          try:
> +            dtype = self.get_case_cfg().get('devtype')
> +            match_str = r"    lcore id|#lcore id"
>              lines = output.split("\r\n")
>              line_nb = len(lines)
>              self.logger.debug("Total output lines: " + str(line_nb))
>
>              for line_index in range(line_nb):
> -                if lines[line_index].startswith("    lcore id"):
> +                if re.match(match_str, lines[line_index]):
>                      self.logger.debug("data output line from: " +
> str(line_index))
>                      break
> -            data_line = line_index - 1
> +            if dtype == 'crypto_cn10k':
> +                data_line = line_index + 2
> +            else:
> +                data_line = line_index - 1
> +
> +            if len(lines[data_line].split(","))>1:
> +                pattern = re.compile(r',')
> +            else:
> +                pattern = re.compile(r'\s+')
>
>              results = []
> -            pattern = re.compile(r"\s+")
> +
>              for line in lines[data_line:-1]:
>                  print(line)
>                  result = {}
> @@ -334,9 +609,15 @@ class TestCryptoPerfCryptodevPerf(TestCase):
>              cpu_info[key] = value.strip()
>          core, thread = 0, 0
>          lcores = self.get_case_cfg()["l"].split(",")
> +        if 'Core(s) per cluster' in out:
> +            cl_soc1 = 'Core(s) per cluster'
> +            cl_soc2 = 'Cluster(s)'
> +        else:
> +            cl_soc1 = 'Core(s) per socket'
> +            cl_soc2 = 'Socket(s)'
>          for lcore in lcores[1:]:
> -            if int(lcore.strip()) < int(cpu_info["Core(s) per socket"]) *
> int(
> -                cpu_info["Socket(s)"]
> +            if int(lcore.strip()) < int(cpu_info[cl_soc1]) * int(
> +                cpu_info[cl_soc2]
>              ):
>                  core += 1
>                  thread += 1
> @@ -362,6 +643,11 @@ class TestCryptoPerfCryptodevPerf(TestCase):
>              dev = "crypto_snow3g"
>          elif self.get_case_cfg()["devtype"] == "crypto_zuc":
>              dev = "crypto_zuc"
> +        elif self.get_case_cfg()["devtype"] == "crypto_cn10k":
> +            dev = "crypto_cn10k"
> +            vf = cc.get_qat_devices(self, cpm_num=1, num=num)
> +            device["a"] = ' -a '.join(vf)
> +            device["vdev"] = None
>          elif self.get_case_cfg()["devtype"] == "crypto_scheduler":
>              dev = "crypto_scheduler"
>              w = cc.get_qat_devices(self, cpm_num=3, num=num * 3)
> @@ -388,8 +674,8 @@ class TestCryptoPerfCryptodevPerf(TestCase):
>
>          return device
>
> -    def _run_crypto_perf_throughput(self):
> -        results = self._run_crypto_perf()
> +    def _run_crypto_perf_throughput(self, **kwargs):
> +        results = self._run_crypto_perf(**kwargs)
>          if results == "skip":
>              return
>          self.verify(results, "test results is none, Test Failed")
> diff --git a/tests/cryptodev_common.py b/tests/cryptodev_common.py
> index b550b468..91067b05 100644
> --- a/tests/cryptodev_common.py
> +++ b/tests/cryptodev_common.py
> @@ -8,7 +8,9 @@ from nics.net_device import GetNicObj
>  conf = SuiteConf("cryptodev_sample")
>
>
> -def bind_qat_device(test_case, driver="igb_uio"):
> +def bind_qat_device(test_case, driver="igb_uio", generate_vfs=False,
> +                    vf_count=2, drvr_type='generic'):
> +
>      if driver == "vfio-pci":
>          test_case.dut.send_expect("modprobe vfio", "#", 10)
>          test_case.dut.send_expect("modprobe vfio-pci", "#", 10)
> @@ -38,6 +40,8 @@ def bind_qat_device(test_case, driver="igb_uio"):
>          devfun_id = addr_array[2]
>          pf_port = GetNicObj(test_case.dut, domain_id, bus_id, devfun_id)
>
> +        if generate_vfs:
> +            pf_port.generate_sriov_vfs_linux(domain_id, bus_id,
> devfun_id, vf_count, drvr_type)
>

Is this different from calling generate_sriov_vfs(), which I assume is the
"correct" behavior. There appears to be a common practice in legacy DTS of
templating out function names to call instead of using inheritance, so you
will have a base function like bind_port(), and then that templates out
calls to bind_port_linux(). bind_port_freebsd(), bind_port_windows() etc.
It's a little strange but it is what it is and we should probably stick
with the existing system if we are going to make additions. But, if there
is a practical reason why you did it this way, then okay. :)


>          sriov_vfs_pci = pf_port.get_sriov_vfs_pci()
>          if not sriov_vfs_pci:
>              raise Exception("can not get vf pci")
> @@ -51,6 +55,11 @@ def bind_qat_device(test_case, driver="igb_uio"):
>
>      test_case.dev = dev
>
> +def bind_mrvl_devices(test_case, driver='vfio-pci', **kwargs):
> +    pci_list = kwargs.get("pci_list").split()
> +    for pci in pci_list:
> +        test_case.dut.bind_eventdev_port(driver, pci)
> +    return pci_list
>
>  def get_qat_devices(test_case, cpm_num=None, num=1):
>      if not cpm_num:
> --
> 2.25.1
>
>
  
Gnanesh Kambalu Palanethra Nov. 14, 2024, 7:22 p.m. UTC | #3
Hi Patrick,

Please find my responses inline.  I am submitting  an updated patch based on your comments.


Regards,
Gnanesh
________________________________
From: Patrick Robb <probb@iol.unh.edu>
Sent: Friday, November 1, 2024 3:45 AM
To: Gnanesh Kambalu Palanethra <gpalanethra@marvell.com>
Cc: dts@dpdk.org <dts@dpdk.org>; JogaRao Nartu <njogarao@marvell.com>; Bharath Rajendra <brajendra@marvell.com>; Cody Cheng <ccheng@iol.unh.edu>; Hiral Shah <hshah@marvell.com>; Jerin Jacob <jerinj@marvell.com>
Subject: [EXTERNAL] Re: [PATCH 1/1] Cn10K crypto Tests: Added Marvell Cn10K specific crypto Tests to TestCryptoPerfCryptodevPerf

Prioritize security for external emails:
Confirm sender and content safety before clicking links or opening attachments
Report Suspicious<https://us-phishalarm-ewt.proofpoint.com/EWT/v1/CRVmXkqW!uK3X-9Acabm2WiDRsTBS4rIwIvRbVjipps-cHL1rcSsoqDSFNMcXqr8BOINoBweI4pZ27nz0d8jq5OKeirZIB_d-hHWPT7z2gNmL_lMlHhGml7DuPSnMefWOSbKR7Q$>

Hi Gnanesh, thanks for sending this. My name is Patrick Robb, and I'm a CI Testing Lab manager at the DPDK Community Lab hosted at UNH, and a maintainer for the "new" DTS framework which exists within the DPDK repo and should be replacing the legacy DTS project in the long term(what this patch is for). We do have a Marvell CN10K board at the lab, and I would love to begin running some crypto testing from that if possible (so, thanks again for this series).

So, most likely we will want to provide a review from UNH and test this, so that it can be merged to DTS (technically I was granted the authority to apply patches to the legacy repo).

I am adding Cody to this thread as I think he can assist me on this. Cody I am going to provide some comments inline as an initial review for this series, and then we should apply this series locally on the traffic generator/test engine for the CN10K (Intel-1 in our lab), dry run the test which Gnanesh has added, and report back to him. If all goes well, it should be possible to merge.

Long term, we will want to circle back and add this test to the new DTS framework, but it is not ready for this test yet, as currently new DTS is testpmd only - there is not cryptodev support. Although, adding such support to new DTS is something I would love to work with you on in the next year if you have time available for development towards DPDK testing! Anyways, comments inline:

On Wed, Oct 30, 2024 at 12:44 PM Gnanesh <gpalanethra@marvell.com<mailto:gpalanethra@marvell.com>> wrote:
    Changes included in this patch:
    -- New TestCases are added for Cn10K CPT Hardware crypto accelerator.
    -- when Testing newly added Testcases it is found Necessary to make Below Changes
          1> tests/cryptodev_common.py
               --> bind_qat_device function is updated to generate VFs for the given crypto_dev_id
               --> added New function bind_mrvl_devices to accept list of PCI Ids and bind it to vfio_pci driver.
          2> framework/crb.py
               --> updated pci_devices_information_uncached_linux Method to handle additional Nic Speeds for Cavium,
                   as it was restricting only to 1 GIG NIC speed

Thanks, I also made this change on our local DTS project for cn10k, but the patch was not submitted to the "legacy DTS" mailing list here as it was unmaintained (and still is, to David's point). But, if we can do a review it is good to get these things into mainline. Maybe I can rebase our local patch on yours if there were any other changes, and get feedback for cn10k specific questions.

I have a hard time understanding how this 10G NIC speed requirement got added in the first place as it was quite peculiar and confused me for a time as it was dropping the ports list! I'm sure you had a similar experience... :)

          3> framework/settings.py
               --> Changed the default Cavium NIC driver to rvu_nicpf.

Thanks, we are also running from this change at the Community Lab. But, I thought you said above you are creating VFs from the bind_qat_device function. So, does rvu_nicvf need to be added as well?

          4> nics/net_device.py
               --> added Missing expect object
          5> conf/crypto_perf_cryptodev_perf.cfg
               --> added Marvell Cn10K configs for crypto_perf_cryptodev_perf
          6> framework/ssh_pexpect.py
               --> output.replace command's output is not set before returning from get_output_all method
                   hence Unnecessarily returning Prompt along with the output
          7> tests/TestSuite_crypto_perf_cryptodev_perf.py
               -->  added Marvell CN10K Testcases

Again we will apply these, give them a run, and report back.

             Updated Below Private functions
                6.1> _run_crypto_perf() --> receives additional KW arguments
                6.2> _parse_output()  ---> Updated to handle Marvell Cn10K app command output
                6.3> _run_crypto_perf_throughput() --> receives additional KW arguments


I don't see any modifications for the build process. Is there a need to include "-Dplatform=cn10k" when setting up the build from meson, or is the default build fine?

 right now there is no change to build procedure, we are cross compiling DPDK on x86 and copying the application on to DUT.





Signed-off-by: Gnanesh <gpalanethra@marvell.com<mailto:gpalanethra@marvell.com>>
---
 conf/crypto_perf_cryptodev_perf.cfg           | 849 ++++++++++++++++++
 framework/crb.py                              |   6 +-
 framework/settings.py                         |   4 +-
 framework/ssh_pexpect.py                      |   2 +-
 nics/net_device.py                            |   3 +
 tests/TestSuite_crypto_perf_cryptodev_perf.py | 322 ++++++-
 tests/cryptodev_common.py                     |  11 +-
 7 files changed, 1172 insertions(+), 25 deletions(-)

diff --git a/framework/crb.py b/framework/crb.py
index 9e3b0a58..2fe9d147 100644
--- a/framework/crb.py
+++ b/framework/crb.py
@@ -374,15 +374,15 @@ class Crb(object):
                 pass

         for i in range(len(match)):
-            # check if device is cavium and check its linkspeed, append only if it is 10G
+            # for cavium supported link speed specfied under linkspeeds
             if "177d:" in match[i][1]:
-                linkspeed = "10000"
+                linkspeeds = ["40000", "10000", "25000", "50000", "100000"]

Honestly I don't understand why we have this condition at all. Other devices in DTS do not have this check where the portlinks can be dropped in crbs.py. What do you think about updating the accepted linkspeeds vs just removing this step entirely? Can we expect additional linkspeeds to be added for 177d in the future, and have this problem crop up again for a new linkspeed, or is this list of speeds "final" for 177d?

This check is now completely Removed in the updated patch.


                 nic_linkspeed = self.send_expect(
                     "cat /sys/bus/pci/devices/%s/net/*/speed" % match[i][0],
                     "# ",
                     alt_session=True,
                 )
-                if nic_linkspeed.split()[0] == linkspeed:
+                if nic_linkspeed.split()[0] in  linkspeeds:
                     self.pci_devices_info.append((match[i][0], match[i][1]))
             else:
                 self.pci_devices_info.append((match[i][0], match[i][1]))
diff --git a/framework/settings.py b/framework/settings.py
index 1a561dda..b2c371f8 100644
--- a/framework/settings.py
+++ b/framework/settings.py
@@ -166,8 +166,8 @@ DRIVERS = {
     "cavium_a034": "thunder-nicvf",
     "cavium_0011": "thunder-nicvf",
     "IXGBE_10G-X550EM_X_SFP": "ixgbe",
-    "cavium_a063": "octeontx2-nicpf",
-    "cavium_a064": "octeontx2-nicvf",
+    "cavium_a063": "rvu_nicpf",
+    "cavium_a064": "rvu_nicvf",

Okay disregard my comment from above about nicvf, I see you've added it. Please do update the commit message to reflect that this is included. And, while you're at it, you can update the commit message to accord with the style guidelines for submitting DPDK patches as seen here on sections 9.6 and 9.7 https://doc.dpdk.org/guides/contributing/patches.html<https://urldefense.proofpoint.com/v2/url?u=https-3A__doc.dpdk.org_guides_contributing_patches.html&d=DwMFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=Ulaoc1LiTkE5RG69cLkMa7P9xGTL0rP6LosuDS8KOe4&m=yMQuTxirxlDswEOgMtDRf6ZbKP5WHuwmlCh4L0OD2_SRTpFWN13r2mHGaVw79Yld&s=0zOdBmtI06ggYRLG6hM42PANpPCDOtwFmFz5EWUKVAw&e=>

This change message is already part of the commit message.
Regarding the style guideline, I have tried to incorporate in the updated patch, please help me in case further changes are needed


:)

     "ICE_100G-E810C_QSFP": "ice",
     "ICE_25G-E810C_SFP": "ice",
     "ICE_25G-E823C_QSFP": "ice",
diff --git a/framework/ssh_pexpect.py b/framework/ssh_pexpect.py
index 2132c066..a96087d2 100644
--- a/framework/ssh_pexpect.py
+++ b/framework/ssh_pexpect.py
@@ -186,7 +186,7 @@ class SSHPexpect:

     def get_output_all(self):
         output = self.session.before
-        output.replace("[PEXPECT]", "")
+        output = output.replace("[PEXPECT]#", "")

So, this is just removing the leading "#" from the output? I'm not sure we should be making styling changes for a "deprecated" repository. If I look at this example output, from one of the CI runs of this testsuite on a quickassist card:

# Device 1 on lcore 40

# total operations: 30000000
# Buffer size: 1024
# Burst size: 32
#     Number of bursts: 937500
#
#                 Total   Average   Maximum   Minimum
#  enqueued    30000000        32        32        32
#  dequeued    30000000        32        32         1
#    cycles 91892876610      3063    304625       485
# time [us]  3675715064   122.520 12185.000    19.400

I imagine the DTS maintainers were aware of this and thought it was the most clear way to print the logs. I don't think we need to override that decision now.

This is not a styling change.
output.replace will not replace the special prompt "[PEXPECT]" directly on the variable.
output variable needs to be overwritten to have the output strip of the prompt [I am overwriting output ]
special prompt variable is "[PEXPECT]#", not the "[PEXPECT]". hence output will have unwanted "#" string.
      


         return output

     def close(self, force=False):
diff --git a/nics/net_device.py b/nics/net_device.py
index 0f9c1af4..dd0d0e92 100644
--- a/nics/net_device.py
+++ b/nics/net_device.py
@@ -43,6 +43,7 @@ class NetDevice(object):
         if not isinstance(crb, Crb):
             raise Exception("  Please input the instance of Crb!!!")
         self.crb = crb
+        self.__send_expect = self.crb.send_expect
         self.domain_id = domain_id
         self.bus_id = bus_id
         self.devfun_id = devfun_id
@@ -727,6 +728,8 @@ class NetDevice(object):
             "/sys/bus/pci/devices/%s:%s:%s" % (domain_id, bus_id, devfun_id),
             vf_reg_file,
         )
+        self.__send_expect("echo 0 > %s" %
+                           (vf_reg_path), "# ")

So, the pre-existing VFs need to be destroyed before a new set of VFs can be created? Do you know if this is Marvell specific, or a universal behavior?

I am not aware of the other boards, but it is the clean way of creating the VFs.

         self.__send_expect("echo %d > %s" % (int(vf_num), vf_reg_path), "# ")

     def generate_sriov_vfs_linux_igb_uio(self, domain_id, bus_id, devfun_id, vf_num):
diff --git a/tests/TestSuite_crypto_perf_cryptodev_perf.py b/tests/TestSuite_crypto_perf_cryptodev_perf.py
index a3f48eee..b435cefa 100644
--- a/tests/TestSuite_crypto_perf_cryptodev_perf.py
+++ b/tests/TestSuite_crypto_perf_cryptodev_perf.py
@@ -59,14 +59,74 @@ class TestCryptoPerfCryptodevPerf(TestCase):
                 "# ",
                 5,
             )
-
-        cc.bind_qat_device(self, "vfio-pci")
+        if self.nic == "cavium_a063":
+            count = len(self.get_suite_cfg().get('l').split(",")) - 1
+            cc.bind_qat_device(self, "vfio-pci", generate_vfs=True, vf_count=count)
+        else:
+            cc.bind_qat_device(self, "vfio-pci")

If we are going to use this function for non-QAT devices, can we please update the function name to prevent confusion? It looks like it's a short list:

Addressed this in  the updated patch

probb@d121003:~/repos/dts> grep -r "bind_qat_d" .
./tests/TestSuite_compressdev_qat_pmd.py:        cc.bind_qat_device(self, self.drivername)
./tests/TestSuite_crypto_perf_cryptodev_perf.py:        cc.bind_qat_device(self, "vfio-pci")
./tests/TestSuite_fips_cryptodev.py:        cc.bind_qat_device(self, "vfio-pci")
./tests/TestSuite_ipsec_gw_cryptodev_func.py:        cc.bind_qat_device(self, self.drivername)
./tests/TestSuite_l2fwd_cryptodev_func.py:        cc.bind_qat_device(self, self.drivername)
./tests/TestSuite_virtio_ipsec_cryptodev_func.py:        cc.bind_qat_device(self, self.drivername)
./tests/TestSuite_virtio_perf_cryptodev_func.py:        cc.bind_qat_device(self, self.drivername)
./tests/TestSuite_virtio_unit_cryptodev_func.py:        cc.bind_qat_device(self, self.drivername)
./tests/compress_common.py:def bind_qat_device(test_case, driver="igb_uio"):
./tests/cryptodev_common.py:def bind_qat_device(test_case, driver="igb_uio"):



         src_files = ["dep/test_aes_cbc.data", "dep/test_aes_gcm.data"]
         self.dut_file_dir = "/tmp"
         for file in src_files:
             self.dut.session.copy_file_to(file, self.dut_file_dir)

     def tear_down_all(self):
+        if self.nic == "cavium_a063":
+            cc.bind_qat_device(self, "vfio-pci", generate_vfs=True,
+                               vf_count=2)
+            import pandas as pd

I don't have a problem with this, but I'm just wondering why this is imported here as opposed to the top of the file?

wrt to pandas import, as we are generating this report only in Marvell case. this import should not affect others

+            tuples = [('TestCase',), ('performance', 'failed_enq'),
+                      ('performance', 'failed_deq'),
+                      ('performance', 'throughput_mops'),
+                      ('performance', 'cycle_buf'),
+                      ('performance', 'throughput', 'value'),
+                      ('performance', 'throughput', 'delta'),
+                      ('parameters', 'core_num/thread_num'),
+                      ('parameters', 'frame_size'),
+                      ('parameters', 'burst_size'),
+                      ('parameters', 'total_ops'), ('status',)]
+            mindex = pd.MultiIndex.from_tuples(tuples)
+            index = 0
+            df = pd.DataFrame([], columns=mindex)
+            for tname, data in self._perf_result.items():
+                for dt in data:
+                    for key, value in dt.items():
+                        df.loc[index, 'TestCase'] = tname
+                        if key == 'status':
+                            df.loc[index, (key)] = value
+                            continue
+                        for item in value:
+                            if item['name'] == 'throughput':
+                                df.loc[index, (key, item['name'], 'value')] = item['value']
+                                df.loc[index, (key, item['name'], 'delta')] = item['delta']
+                            else:
+                                df.loc[index, (key, item['name'])] = item['value']
+                    index += 1
+            perf_xl = self.logger.log_path + "/" + "perf_cryptodev_result.xls"
+            writer = pd.ExcelWriter(perf_xl, engine='xlsxwriter')
+            wb  = writer.book
+            df.to_excel(writer, sheet_name='Sheet1')
+            ws = writer.sheets["Sheet1"]
+            merge_format = wb.add_format({"bold": 1,"border": 1,"align": "center",
+                                          "valign": "vcenter"})
+            ws.merge_range('B1:B3', "TestCase", merge_format)
+            ws.merge_range('C2:C3', "failed_enq", merge_format)
+            ws.merge_range('D2:D3', "failed_deq", merge_format)
+            ws.merge_range('E2:E3', "throughput_mops", merge_format)
+            ws.merge_range('F2:F3', "throughput_mops", merge_format)
+            ws.merge_range('I2:I3', "core_num/thread_num", merge_format)
+            ws.merge_range('J2:J3', "frame_size", merge_format)
+            ws.merge_range('K2:K3', "burst_size", merge_format)
+            ws.merge_range('L2:L3', "total_ops", merge_format)
+            ws.merge_range('M1:M3', "status", merge_format)
+
+            format1 = wb.add_format({"bg_color": "#C6EFCE", "font_color": "#006100", "border": 1})
+            format2 = wb.add_format({"bg_color": "#FFC7CE", "font_color": "#9C0006"})
+            ws.conditional_format("B1:M3", {"type": "cell", "criteria": ">=", "value": 0 , 'format': format2})
+            condition = 'B5:M%s'%(5+len(df)-1)
+            ws.conditional_format(condition, {'type': 'no_blanks', 'format':format1})
+            ws.autofit()
+
+            wb.close()
         if self._perf_result:
             with open(
                 self.logger.log_path + "/" + "perf_cryptodev_result.json", "a"
@@ -175,12 +235,197 @@ class TestCryptoPerfCryptodevPerf(TestCase):
     def test_scheduler_zuc(self):
         self._run_crypto_perf_throughput()

+    # Marvell CN10K Testcases
+    def _crypto_func_cn10k(self):
+        """Functional tests Helper
+        Returns:
+           None
+        """
+        self._run_crypto_func()
+
+    def _crypto_perf_cn10k(self):
+        """Perf tests Helper function
+        Returns:
+           None
+        """
+        self._run_crypto_perf_throughput(expected="# ",
+                                         trim_whitespace=False)
+    def test_perf_mrvl_aes_cbc(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_aes_gcm_encrypt(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_aes_cbc_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_aes_sha2_256_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_zuc_eea3_cipher_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_zuc_eia3_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_aes_gmac_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_null_cipher_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_snow3g_uea2_cipher_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_kasumi_f8_cipher_only(self):
+        self._crypto_perf_cn10k() #F
+
+    def test_perf_mrvl_snow3g_uia2_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_kasumi_f9_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha1_hmac_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha1_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_aes_ctr_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_aes_ctr_snow_3g_uia2_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_aes_cmac_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_3des_cbc_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha1_hmac_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha2_224_hmac_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha2_256_hmac_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha2_384_hmac_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_aes_ctr_snow_3g_uia2_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_aes_ctr_zuc_eia3_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_aes_ctr_aes_cmac_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_zuc_eea3_snow3g_uia2_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_zuc_eea3_zuc_eia3_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_zuc_eea3_aes_cmac_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_snow3g_uea2_snow3g_uia2_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_snow3g_uea2_zuc_eia3_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_snow3g_uea2_aes_cmac_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_md5_hmac_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_aes_ctr_cipher_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_3des_cbc_cipher_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha2_224_hmac_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha2_256_hmac_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha2_384_hmac_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha2_512_hmac_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_aes_xts__cipher_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_des_cbc_cipher_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_md5_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha512_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha384_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha256_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha224_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_snow3g_uea2_snow3g_uia2(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha1_hmac_armv8(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha2_hmac_armv8(self):
+        self._crypto_perf_cn10k()
+
+    def test_mrvl_ptest(self):
+        self._crypto_func_cn10k()
+
+    def test_mrvl_burst(self):
+        self._crypto_func_cn10k()
+
+    def test_mrvl_devtype(self):
+        self._crypto_func_cn10k()
+
+    def test_mrvl_crypto_sha1_hmac_armv8(self):
+        self._crypto_func_cn10k()
+
+    def test_mrvl_chacha20_poly1305_aead(self):
+        self._crypto_func_cn10k()
+
+    def test_mrvl_crypto_sha2_hmac_armv8(self):
+        self._crypto_func_cn10k()
+
+    def test_mrvl_buffer(self):
+        self._crypto_func_cn10k()
+
     # Private functions
     def _run_crypto_func(self):
+        """ Runs Crypto functional test.
+        Returns:
+            None
+        """
+
         if cc.is_test_skip(self):
             return
-
-        cores = ",".join(self.dut.get_core_list("1S/2C/1T"))
+        core_list = self.get_suite_cfg().get('core_list',"1S/2C/1T")
+        cores = ",".join(self.dut.get_core_list(core_list))
         config = {"l": cores}
         devices = self._get_crypto_device(1)
         if not devices:
@@ -206,12 +451,22 @@ class TestCryptoPerfCryptodevPerf(TestCase):

         out = self.dut.send_command(
             "cat %s/%s.txt" % (self.dut_file_dir, self.running_case), 30
-        )
-
+        ).strip()
         self.verify("Error" not in out, "Test function failed")
         self.verify("failed" not in out, "Test function failed")
-
-    def _run_crypto_perf(self):
+        assert (out != ""), "No output"
+
+    def _run_crypto_perf(self, **kwargs):
+        """ Runs Crypto Performance test.
+        Args:
+          **kwargs:
+              expected: send_expect match prompt
+              trim_whitespace: trime whitespace from command output
+        Returns:
+            None
+        """
+        expected = kwargs.get("expected", "#")
+        trim_whitespace = kwargs.get("trim_whitespace", True)
         if cc.is_test_skip(self):
             return "skip"

@@ -222,13 +477,14 @@ class TestCryptoPerfCryptodevPerf(TestCase):
             return "skip"

         eal_opt_str = cc.get_eal_opt_str(self, devices)
-        crypto_perf_opt_str = self._get_crypto_perf_opt_str()
+        crypto_perf_opt_str = self._get_crypto_perf_opt_str(**kwargs)

         cmd_str = cc.get_dpdk_app_cmd_str(
             self._app_path, eal_opt_str, crypto_perf_opt_str
         )
         try:
-            out = self.dut.send_expect(cmd_str, "#", 600)
+            out = self.dut.send_expect(cmd_str, expected, 600,
+                                       trim_whitespace=trim_whitespace)
         except Exception as ex:
             self.logger.error(ex)
             raise ex
@@ -237,25 +493,44 @@ class TestCryptoPerfCryptodevPerf(TestCase):

         return results

-    def _get_crypto_perf_opt_str(self, override_crypto_perf_opts={}):
+    def _get_crypto_perf_opt_str(self, **kwargs):
+        """get crypto perf app option string
+        args:
+            **kwargs:
+                override_crypto_perf_opts- Suite/Case specfic perf
+                app config to overwrite defaults
+        Returns:
+            crypto perf option string
+        """
+        override_crypto_perf_opts = kwargs.get("override_crypto_perf_opts", {})
         return cc.get_opt_str(
             self, self._default_crypto_perf_opts, override_crypto_perf_opts
         )

     def _parse_output(self, output):
         try:
+            dtype = self.get_case_cfg().get('devtype')
+            match_str = r"    lcore id|#lcore id"
             lines = output.split("\r\n")
             line_nb = len(lines)
             self.logger.debug("Total output lines: " + str(line_nb))

             for line_index in range(line_nb):
-                if lines[line_index].startswith("    lcore id"):
+                if re.match(match_str, lines[line_index]):
                     self.logger.debug("data output line from: " + str(line_index))
                     break
-            data_line = line_index - 1
+            if dtype == 'crypto_cn10k':
+                data_line = line_index + 2
+            else:
+                data_line = line_index - 1
+
+            if len(lines[data_line].split(","))>1:
+                pattern = re.compile(r',')
+            else:
+                pattern = re.compile(r'\s+')

             results = []
-            pattern = re.compile(r"\s+")
+
             for line in lines[data_line:-1]:
                 print(line)
                 result = {}
@@ -334,9 +609,15 @@ class TestCryptoPerfCryptodevPerf(TestCase):
             cpu_info[key] = value.strip()
         core, thread = 0, 0
         lcores = self.get_case_cfg()["l"].split(",")
+        if 'Core(s) per cluster' in out:
+            cl_soc1 = 'Core(s) per cluster'
+            cl_soc2 = 'Cluster(s)'
+        else:
+            cl_soc1 = 'Core(s) per socket'
+            cl_soc2 = 'Socket(s)'
         for lcore in lcores[1:]:
-            if int(lcore.strip()) < int(cpu_info["Core(s) per socket"]) * int(
-                cpu_info["Socket(s)"]
+            if int(lcore.strip()) < int(cpu_info[cl_soc1]) * int(
+                cpu_info[cl_soc2]
             ):
                 core += 1
                 thread += 1
@@ -362,6 +643,11 @@ class TestCryptoPerfCryptodevPerf(TestCase):
             dev = "crypto_snow3g"
         elif self.get_case_cfg()["devtype"] == "crypto_zuc":
             dev = "crypto_zuc"
+        elif self.get_case_cfg()["devtype"] == "crypto_cn10k":
+            dev = "crypto_cn10k"
+            vf = cc.get_qat_devices(self, cpm_num=1, num=num)
+            device["a"] = ' -a '.join(vf)
+            device["vdev"] = None
         elif self.get_case_cfg()["devtype"] == "crypto_scheduler":
             dev = "crypto_scheduler"
             w = cc.get_qat_devices(self, cpm_num=3, num=num * 3)
@@ -388,8 +674,8 @@ class TestCryptoPerfCryptodevPerf(TestCase):

         return device

-    def _run_crypto_perf_throughput(self):
-        results = self._run_crypto_perf()
+    def _run_crypto_perf_throughput(self, **kwargs):
+        results = self._run_crypto_perf(**kwargs)
         if results == "skip":
             return
         self.verify(results, "test results is none, Test Failed")
diff --git a/tests/cryptodev_common.py b/tests/cryptodev_common.py
index b550b468..91067b05 100644
--- a/tests/cryptodev_common.py
+++ b/tests/cryptodev_common.py
@@ -8,7 +8,9 @@ from nics.net_device import GetNicObj
 conf = SuiteConf("cryptodev_sample")


-def bind_qat_device(test_case, driver="igb_uio"):
+def bind_qat_device(test_case, driver="igb_uio", generate_vfs=False,
+                    vf_count=2, drvr_type='generic'):
+
     if driver == "vfio-pci":
         test_case.dut.send_expect("modprobe vfio", "#", 10)
         test_case.dut.send_expect("modprobe vfio-pci", "#", 10)
@@ -38,6 +40,8 @@ def bind_qat_device(test_case, driver="igb_uio"):
         devfun_id = addr_array[2]
         pf_port = GetNicObj(test_case.dut, domain_id, bus_id, devfun_id)

+        if generate_vfs:
+            pf_port.generate_sriov_vfs_linux(domain_id, bus_id, devfun_id, vf_count, drvr_type)

Is this different from calling generate_sriov_vfs(), which I assume is the "correct" behavior. There appears to be a common practice in legacy DTS of templating out function names to call instead of using inheritance, so you will have a base function like bind_port(), and then that templates out calls to bind_port_linux(). bind_port_freebsd(), bind_port_windows() etc. It's a little strange but it is what it is and we should probably stick with the existing system if we are going to make additions. But, if there is a practical reason why you did it this way, then okay. :)

I missed it initially, addressed it in updated patch.

         sriov_vfs_pci = pf_port.get_sriov_vfs_pci()
         if not sriov_vfs_pci:
             raise Exception("can not get vf pci")
@@ -51,6 +55,11 @@ def bind_qat_device(test_case, driver="igb_uio"):

     test_case.dev<https://urldefense.proofpoint.com/v2/url?u=http-3A__test-5Fcase.dev&d=DwMFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=Ulaoc1LiTkE5RG69cLkMa7P9xGTL0rP6LosuDS8KOe4&m=yMQuTxirxlDswEOgMtDRf6ZbKP5WHuwmlCh4L0OD2_SRTpFWN13r2mHGaVw79Yld&s=B1TWMhgRG8QFE8zQcOleQ4B7kPHb7NYeTCL2QLkTPaA&e=> = dev

+def bind_mrvl_devices(test_case, driver='vfio-pci', **kwargs):
+    pci_list = kwargs.get("pci_list").split()
+    for pci in pci_list:
+        test_case.dut.bind_eventdev_port(driver, pci)
+    return pci_list

 def get_qat_devices(test_case, cpm_num=None, num=1):
     if not cpm_num:
--
2.25.1
  
Cody Cheng Nov. 26, 2024, 12:07 a.m. UTC | #4
Hey Gnanesh,

I'm currently attempting to run the new test cases on our CN10K board,
but I am encountering a few issues that I need help resolving.

For setup, I created a VF and bound it with the following commands:

echo 1 > /sys/bus/pci/devices/0002:20:00.0/sriov_numvfs
./usertools/dpdk-devbind.py -u 0002:20:00.1
./usertools/dpdk-devbind.py -b vfio-pci 0002:20:00.1

When I run dts, I am getting the error below:


18/11/2024 17:51:03               dut.172.18.0.254:
arm64-native-linuxapp-gcc/app/dpdk-test-crypto-perf  -l 9,10 -a
0002:20:00.1 --socket-mem 2048,0 -n 2  -- --ptest throughput --silent
True --pool-sz 16384 --total-ops 1000000 --burst-sz 32 --buffer-sz 64
--devtype crypto_cn10k --optype auth-then-cipher --cipher-algo
snow3g-uea2 --cipher-op encrypt --cipher-key-sz 16 --cipher-iv-sz 16
--auth-algo aes-cmac --auth-op generate --auth-key-sz 16 --aead-iv-sz
0 --digest-sz 4 --csv-friendly True
18/11/2024 17:51:04               dut.172.18.0.254: EAL: Detected
C/resultPU lcores: 24^M
EAL: Detected NUMA nodes: 1^M
EAL: Detected static linkage of DPDK^M
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket^M
EAL: Selected IOVA mode 'VA'^M
EAL: VFIO support initialized^M
EAL: Using IOMMU type 1 (Type 1)^M
CNXK: RoC Model: cn10ka_a1 (HW_PLATFORM)^M
CRYPTODEV: Creating cryptodev 0002:20:00.1^M
CRYPTODEV: Initialisation parameters - name: 0002:20:00.1,socket id:
0, max queue pairs: 0^M
CNXK: dev_init():1560 Failed to validate LMT line^M
CNXK: roc_cpt_dev_init():763 Failed to init roc device^M
CNXK: cn10k_cpt_pci_probe():80 Failed to initialize roc cpt rc=-14^M
CRYPTODEV: Closing crypto device 0002:20:00.1^M
CNXK: cn10k_cpt_pci_probe():116 Could not create device (vendor_id:
0x177d device_id: 0xa0f3)^M
PCI_BUS: Requested device 0002:20:00.1 cannot be used^M
EAL: Bus (pci) probe failed.^M
No crypto devices type crypto_cn10k available^M
USER1: Failed to initialise requested crypto device type^M


I observed that if I modify the DPDK build process by adding
`-Dmax_lcores=24` to the meson arguments, the initial error
disappears. Unfortunately, this introduces a new issue:


TestCryptoPerfCryptodevPerf: Test Case
test_mrvl_chacha20_poly1305_aead Result FAILED: TIMEOUT on
arm64-native-linuxapp-gcc/app/dpdk-test-crypto-perf  -l 1,2 -a
0002:20:00.1 --socket-mem 2048,0 -n 2  -- --ptest throughput --silent
True --pool-sz 16384 --total-ops 10000000 --burst-sz 32 --buffer-sz
64,128,256,512,1024,2048 --devtype crypto_cn10k --optype aead
--aead-algo chacha20-poly1305 --aead-op encrypt --aead-key-sz 32
--aead-iv-sz 12 --aead-aad-sz 32 --digest-sz 16 --csv-friendly True
>/tmp/test_mrvl_chacha20_poly1305_aead.txt
   TestCryptoPerfCryptodevPerf: EAL: Detected CPU lcores: 24
EAL: Detected NUMA nodes: 1
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: VFIO support initialized
EAL: Using IOMMU type 1 (Type 1)
CNXK: RoC Model: cn10ka_a1 (HW_PLATFORM)
CRYPTODEV: Creating cryptodev 0002:20:00.1
CRYPTODEV: Initialisation parameters - name: 0002:20:00.1,socket id:
0, max queue pairs: 0
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cpt_lf_misc_irq():58 Err_irq=0x42 pf=31, vf=0
CNXK:     CPT_LF_Q_BASE: 000000019fbd5400
CNXK:     CPT_LF_Q_SIZE: 000000000000003d
CNXK:     CPT_LF_Q_INST_PTR: 0000006200000062
CNXK:     CPT_LF_Q_GRP_PTR: 0000000100000001
CNXK:     CPT_LF_CTL: 0000000000000052
CNXK:     CPT_LF_MISC_INT_ENA_W1S: 000000000000006e
CNXK:     CPT_LF_MISC_INT: 0000000000000042
CNXK:     CPT_LF_INPROG: 0000100080010000
CNXK: Count registers for CPT LF0:
CNXK:     Encrypted byte count: 0
CNXK:     Encrypted packet count: 0
CNXK:     Decrypted byte count: 0
CNXK:     Decrypted packet count: 0
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cpt_lf_misc_irq():58 Err_irq=0x2 pf=31, vf=0
CNXK:     CPT_LF_Q_BASE: 000000019fbd5400
CNXK:     CPT_LF_Q_SIZE: 000000000000003d
CNXK:     CPT_LF_Q_INST_PTR: 0000006200000062
CNXK:     CPT_LF_Q_GRP_PTR: 0000000100000001
CNXK:     CPT_LF_CTL: 0000000000000052
CNXK:     CPT_LF_MISC_INT_ENA_W1S: 000000000000006e
CNXK:     CPT_LF_MISC_INT: 0000000000000002
CNXK:     CPT_LF_INPROG: 0000100080010000
CNXK: Count registers for CPT LF0:
CNXK:     Encrypted byte count: 0
CNXK:     Encrypted packet count: 0
CNXK:     Decrypted byte count: 0
CNXK:     Decrypted packet count: 0
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
CNXK: cpt_lf_misc_irq():58 Err_irq=0x2 pf=31, vf=0
CNXK:     CPT_LF_Q_BASE: 000000019fbd5400
CNXK:     CPT_LF_Q_SIZE: 000000000000003d
CNXK:     CPT_LF_Q_INST_PTR: 0000006200000062
CNXK:     CPT_LF_Q_GRP_PTR: 0000000100000001
CNXK:     CPT_LF_CTL: 0000000000000052
CNXK:     CPT_LF_MISC_INT_ENA_W1S: 000000000000006e
CNXK:     CPT_LF_MISC_INT: 0000000000000002
CNXK:     CPT_LF_INPROG: 0000100080010000
CNXK: Count registers for CPT LF0:
CNXK:     Encrypted byte count: 0
CNXK:     Encrypted packet count: 0
CNXK:     Decrypted byte count: 0
CNXK:     Decrypted packet count: 0
CNXK: cpt_lf_misc_irq():58 Err_irq=0x2 pf=31, vf=0
CNXK:     CPT_LF_Q_BASE: 000000019fbd5400
CNXK:     CPT_LF_Q_SIZE: 000000000000003d
CNXK:     CPT_LF_Q_INST_PTR: 0000006200000062
CNXK:     CPT_LF_Q_GRP_PTR: 0000000100000001
CNXK:     CPT_LF_CTL: 0000000000000052
CNXK:     CPT_LF_MISC_INT_ENA_W1S: 000000000000006e
CNXK:     CPT_LF_MISC_INT: 0000000000000002
CNXK:     CPT_LF_INPROG: 0000100080010000
CNXK: Count registers for CPT LF0:
CNXK:     Encrypted byte count: 0
CNXK:     Encrypted packet count: 0
CNXK:     Decrypted byte count: 0
CNXK:     Decrypted packet count: 0
CNXK: cpt_lf_misc_irq():58 Err_irq=0x2 pf=31, vf=0
CNXK:     CPT_LF_Q_BASE: 000000019fbd5400
CNXK:     CPT_LF_Q_SIZE: 000000000000003d
CNXK:     CPT_LF_Q_INST_PTR: 0000006200000062
CNXK:     CPT_LF_Q_GRP_PTR: 0000000100000001
CNXK:     CPT_LF_CTL: 0000000000000052
CNXK:     CPT_LF_MISC_INT_ENA_W1S: 000000000000006e
CNXK:     CPT_LF_MISC_INT: 0000000000000002
CNXK:     CPT_LF_INPROG: 0000100080010000
CNXK: Count registers for CPT LF0:
CNXK:     Encrypted byte count: 0
CNXK:     Encrypted packet count: 0
CNXK:     Decrypted byte count: 0
CNXK:     Decrypted packet count: 0
CNXK: cpt_lf_misc_irq():58 Err_irq=0x2 pf=31, vf=0
CNXK:     CPT_LF_Q_BASE: 000000019fbd5400
CNXK:     CPT_LF_Q_SIZE: 000000000000003d
CNXK:     CPT_LF_Q_INST_PTR: 0000006200000062
CNXK:     CPT_LF_Q_GRP_PTR: 0000000100000001
CNXK:     CPT_LF_CTL: 0000000000000052
CNXK:     CPT_LF_MISC_INT_ENA_W1S: 000000000000006e
CNXK:     CPT_LF_MISC_INT: 0000000000000002
CNXK:     CPT_LF_INPROG: 0000100080010000
CNXK: Count registers for CPT LF0:
CNXK:     Encrypted byte count: 0
CNXK:     Encrypted packet count: 0
CNXK:     Decrypted byte count: 0
CNXK:     Decrypted packet count: 0
CNXK: cpt_lf_misc_irq():58 Err_irq=0x2 pf=31, vf=0
CNXK:     CPT_LF_Q_BASE: 000000019fbd5400
CNXK:     CPT_LF_Q_SIZE: 000000000000003d
CNXK:     CPT_LF_Q_INST_PTR: 0000006200000062
CNXK:     CPT_LF_Q_GRP_PTR: 0000000100000001
CNXK:     CPT_LF_CTL: 0000000000000052
CNXK:     CPT_LF_MISC_INT_ENA_W1S: 000000000000006e
CNXK:     CPT_LF_MISC_INT: 0000000000000002
CNXK:     CPT_LF_INPROG: 0000100080010000
CNXK: Count registers for CPT LF0:
CNXK:     Encrypted byte count: 0
CNXK:     Encrypted packet count: 0
CNXK:     Decrypted byte count: 0
CNXK:     Decrypted packet count: 0
CNXK: cn10k_cpt_dequeue_burst():1325 Request timed out
CNXK: Lcore ID: 2, LF/QP ID: 0
CNXK:
CNXK: S/w pending queue:
CNXK: Head: 95
CNXK: Tail: 96
CNXK: Mask: 0x7ff
CNXK: Inflight count: 2047
CNXK:
CNXK: H/w pending queue:
CNXK: Inflight in engines: 0
CNXK: NQ ptr: 0x62
CNXK: DQ ptr: 0x62
CNXK: Insts waiting in CPT: 0
CNXK:
CNXK: CPT AF registers:
CNXK:     CPT_AF_LF0_CTL: 0x0007000000040001
CNXK:     CPT_AF_LF0_CTL2: 0x0000000000000002
CNXK:     inst_req_pc: 0x0000000000000163
CNXK:     inst_lat_pc: 0x00000000002e1a2e
CNXK:     rd_req_pc: 0x0000000000001641
CNXK:     rd_lat_pc: 0x00000000001146b4
CNXK:     rd_uc_pc: 0x0000000000000fd0
CNXK:     active_cycles_pc: 0x0000000000f978e3
CNXK:     ctx_mis_pc: 0x0000000000000005
CNXK:     ctx_hit_pc: 0x000000000000015b
CNXK:     ctx_aop_pc: 0x0000000000000000
CNXK:     ctx_aop_lat_pc: 0x0000000000000000
CNXK:     ctx_ifetch_pc: 0x000000000000000a
CNXK:     ctx_ifetch_lat_pc: 0x00000000000009be
CNXK:     ctx_ffetch_pc: 0x0000000000000160
CNXK:     ctx_ffetch_lat_pc: 0x0000000000000992
CNXK:     ctx_wback_pc: 0x0000000000000160
CNXK:     ctx_wback_lat_pc: 0x0000000000000992
CNXK:     ctx_psh_pc: 0x0000000000000160
CNXK:     ctx_psh_lat_pc: 0x0000000000000992
CNXK:     ctx_err: 0x0000000000000000
CNXK:     ctx_enc_id: 0x0000000000000000
CNXK:     ctx_flush_timer: 0x000000000002faf0
CNXK:     rxc_time: 0x000000000003873a
CNXK:     rxc_time_cfg: 0x0000000000000000
CNXK:     rxc_active_sts: 0x0000000000000000
CNXK:     rxc_zombie_sts: 0x0000000000000000
CNXK:     rxc_dfrg: 0x0000000000000000
CNXK:     x2p_link_cfg0: 0x0000000000040000
CNXK:     x2p_link_cfg1: 0x0000000000040000
CNXK:     busy_sts_ae: 0x0000000000000000
CNXK:     free_sts_ae: 0x0000000000ffffff
CNXK:     busy_sts_se: 0x0000000000000000
CNXK:     free_sts_se: 0xffffffffffffffff
CNXK:     busy_sts_ie: 0x0000000000000000
CNXK:     free_sts_ie: 0x00ffffffffffffff
CNXK:     exe_err_info: 0x0000000000000000
CNXK:     cptclk_cnt: 0x000000169fa8634c
CNXK:     diag: 0x00000000010000ff
CNXK: cn10k_cpt_dequeue_burst():1325 Request timed out
CNXK: Lcore ID: 2, LF/QP ID: 0
CNXK:
CNXK: S/w pending queue:
CNXK: Head: 95
CNXK: Tail: 96
CNXK: Mask: 0x7ff
CNXK: Inflight count: 2047
CNXK:
CNXK: H/w pending queue:
CNXK: Inflight in engines: 0
CNXK: NQ ptr: 0x62
CNXK: DQ ptr: 0x62
CNXK: Insts waiting in CPT: 0
CNXK:
CNXK: CPT AF registers:
CNXK:     CPT_AF_LF0_CTL: 0x0007000000040001
CNXK:     CPT_AF_LF0_CTL2: 0x0000000000000002
CNXK:     inst_req_pc: 0x0000000000000163
CNXK:     inst_lat_pc: 0x00000000002e1a2e
CNXK:     rd_req_pc: 0x0000000000001641
CNXK:     rd_lat_pc: 0x00000000001146b4
CNXK:     rd_uc_pc: 0x0000000000000fd0
CNXK:     active_cycles_pc: 0x0000000000f9b66c
CNXK:     ctx_mis_pc: 0x0000000000000005
CNXK:     ctx_hit_pc: 0x000000000000015b
CNXK:     ctx_aop_pc: 0x0000000000000000
CNXK:     ctx_aop_lat_pc: 0x0000000000000000
CNXK:     ctx_ifetch_pc: 0x000000000000000a
CNXK:     ctx_ifetch_lat_pc: 0x00000000000009be
CNXK:     ctx_ffetch_pc: 0x0000000000000160
CNXK:     ctx_ffetch_lat_pc: 0x0000000000000992
CNXK:     ctx_wback_pc: 0x0000000000000160
CNXK:     ctx_wback_lat_pc: 0x0000000000000992
CNXK:     ctx_psh_pc: 0x0000000000000160
CNXK:     ctx_psh_lat_pc: 0x0000000000000992
CNXK:     ctx_err: 0x0000000000000000
CNXK:     ctx_enc_id: 0x0000000000000000
CNXK:     ctx_flush_timer: 0x000000000002faf0
CNXK:     rxc_time: 0x0000000000009ce5
CNXK:     rxc_time_cfg: 0x0000000000000000
CNXK:     rxc_active_sts: 0x0000000000000000
CNXK:     rxc_zombie_sts: 0x0000000000000000
CNXK:     rxc_dfrg: 0x0000000000000000
CNXK:     x2p_link_cfg0: 0x0000000000040000
CNXK:     x2p_link_cfg1: 0x0000000000040000
CNXK:     busy_sts_ae: 0x0000000000000000
CNXK:     free_sts_ae: 0x0000000000ffffff
CNXK:     busy_sts_se: 0x0000000000000000
CNXK:     free_sts_se: 0xffffffffffffffff
CNXK:     busy_sts_ie: 0x0000000000000000
CNXK:     free_sts_ie: 0x00ffffffffffffff
CNXK:     exe_err_info: 0x0000000000000000
CNXK:     cptclk_cnt: 0x00000016a6eb1638
CNXK:     diag: 0x00000000010000ff
CNXK: cn10k_cpt_dequeue_burst():1325 Request timed out
CNXK: Lcore ID: 2, LF/QP ID: 0

This `cn10k_cpt_dequeue_burst():1325 Request timed out` error will
keep repeating until DTS times out.

Could you please take a look and let me know if there's any
configuration or additional step that I might have missed to resolve
these issues?
I'd be happy to provide more details if needed.

Thank you for your help!

Best Regards,
Cody Cheng

On Thu, Nov 14, 2024 at 2:23 PM Gnanesh Kambalu Palanethra
<gpalanethra@marvell.com> wrote:
>
>
> Hi Patrick,
>
> Please find my responses inline.  I am submitting  an updated patch based on your comments.
>
>
> Regards,
> Gnanesh
> ________________________________
> From: Patrick Robb <probb@iol.unh.edu>
> Sent: Friday, November 1, 2024 3:45 AM
> To: Gnanesh Kambalu Palanethra <gpalanethra@marvell.com>
> Cc: dts@dpdk.org <dts@dpdk.org>; JogaRao Nartu <njogarao@marvell.com>; Bharath Rajendra <brajendra@marvell.com>; Cody Cheng <ccheng@iol.unh.edu>; Hiral Shah <hshah@marvell.com>; Jerin Jacob <jerinj@marvell.com>
> Subject: [EXTERNAL] Re: [PATCH 1/1] Cn10K crypto Tests: Added Marvell Cn10K specific crypto Tests to TestCryptoPerfCryptodevPerf
>
> Prioritize security for external emails:
> Confirm sender and content safety before clicking links or opening attachments
> Report Suspicious
>
> Hi Gnanesh, thanks for sending this. My name is Patrick Robb, and I'm a CI Testing Lab manager at the DPDK Community Lab hosted at UNH, and a maintainer for the "new" DTS framework which exists within the DPDK repo and should be replacing the legacy DTS project in the long term(what this patch is for). We do have a Marvell CN10K board at the lab, and I would love to begin running some crypto testing from that if possible (so, thanks again for this series).
>
> So, most likely we will want to provide a review from UNH and test this, so that it can be merged to DTS (technically I was granted the authority to apply patches to the legacy repo).
>
> I am adding Cody to this thread as I think he can assist me on this. Cody I am going to provide some comments inline as an initial review for this series, and then we should apply this series locally on the traffic generator/test engine for the CN10K (Intel-1 in our lab), dry run the test which Gnanesh has added, and report back to him. If all goes well, it should be possible to merge.
>
> Long term, we will want to circle back and add this test to the new DTS framework, but it is not ready for this test yet, as currently new DTS is testpmd only - there is not cryptodev support. Although, adding such support to new DTS is something I would love to work with you on in the next year if you have time available for development towards DPDK testing! Anyways, comments inline:
>
> On Wed, Oct 30, 2024 at 12:44 PM Gnanesh <gpalanethra@marvell.com> wrote:
>
>     Changes included in this patch:
>     -- New TestCases are added for Cn10K CPT Hardware crypto accelerator.
>     -- when Testing newly added Testcases it is found Necessary to make Below Changes
>           1> tests/cryptodev_common.py
>                --> bind_qat_device function is updated to generate VFs for the given crypto_dev_id
>                --> added New function bind_mrvl_devices to accept list of PCI Ids and bind it to vfio_pci driver.
>           2> framework/crb.py
>                --> updated pci_devices_information_uncached_linux Method to handle additional Nic Speeds for Cavium,
>                    as it was restricting only to 1 GIG NIC speed
>
>
> Thanks, I also made this change on our local DTS project for cn10k, but the patch was not submitted to the "legacy DTS" mailing list here as it was unmaintained (and still is, to David's point). But, if we can do a review it is good to get these things into mainline. Maybe I can rebase our local patch on yours if there were any other changes, and get feedback for cn10k specific questions.
>
> I have a hard time understanding how this 10G NIC speed requirement got added in the first place as it was quite peculiar and confused me for a time as it was dropping the ports list! I'm sure you had a similar experience... :)
>
>
>           3> framework/settings.py
>                --> Changed the default Cavium NIC driver to rvu_nicpf.
>
>
> Thanks, we are also running from this change at the Community Lab. But, I thought you said above you are creating VFs from the bind_qat_device function. So, does rvu_nicvf need to be added as well?
>
>
>           4> nics/net_device.py
>                --> added Missing expect object
>           5> conf/crypto_perf_cryptodev_perf.cfg
>                --> added Marvell Cn10K configs for crypto_perf_cryptodev_perf
>           6> framework/ssh_pexpect.py
>                --> output.replace command's output is not set before returning from get_output_all method
>                    hence Unnecessarily returning Prompt along with the output
>           7> tests/TestSuite_crypto_perf_cryptodev_perf.py
>                -->  added Marvell CN10K Testcases
>
>
> Again we will apply these, give them a run, and report back.
>
>
>              Updated Below Private functions
>                 6.1> _run_crypto_perf() --> receives additional KW arguments
>                 6.2> _parse_output()  ---> Updated to handle Marvell Cn10K app command output
>                 6.3> _run_crypto_perf_throughput() --> receives additional KW arguments
>
>
>
> I don't see any modifications for the build process. Is there a need to include "-Dplatform=cn10k" when setting up the build from meson, or is the default build fine?
>
>  right now there is no change to build procedure, we are cross compiling DPDK on x86 and copying the application on to DUT.
>
>
>
>
>
>
> Signed-off-by: Gnanesh <gpalanethra@marvell.com>
> ---
>  conf/crypto_perf_cryptodev_perf.cfg           | 849 ++++++++++++++++++
>  framework/crb.py                              |   6 +-
>  framework/settings.py                         |   4 +-
>  framework/ssh_pexpect.py                      |   2 +-
>  nics/net_device.py                            |   3 +
>  tests/TestSuite_crypto_perf_cryptodev_perf.py | 322 ++++++-
>  tests/cryptodev_common.py                     |  11 +-
>  7 files changed, 1172 insertions(+), 25 deletions(-)
>
> diff --git a/framework/crb.py b/framework/crb.py
> index 9e3b0a58..2fe9d147 100644
> --- a/framework/crb.py
> +++ b/framework/crb.py
> @@ -374,15 +374,15 @@ class Crb(object):
>                  pass
>
>          for i in range(len(match)):
> -            # check if device is cavium and check its linkspeed, append only if it is 10G
> +            # for cavium supported link speed specfied under linkspeeds
>              if "177d:" in match[i][1]:
> -                linkspeed = "10000"
> +                linkspeeds = ["40000", "10000", "25000", "50000", "100000"]
>
>
> Honestly I don't understand why we have this condition at all. Other devices in DTS do not have this check where the portlinks can be dropped in crbs.py. What do you think about updating the accepted linkspeeds vs just removing this step entirely? Can we expect additional linkspeeds to be added for 177d in the future, and have this problem crop up again for a new linkspeed, or is this list of speeds "final" for 177d?
>
> This check is now completely Removed in the updated patch.
>
>
>
>                  nic_linkspeed = self.send_expect(
>                      "cat /sys/bus/pci/devices/%s/net/*/speed" % match[i][0],
>                      "# ",
>                      alt_session=True,
>                  )
> -                if nic_linkspeed.split()[0] == linkspeed:
> +                if nic_linkspeed.split()[0] in  linkspeeds:
>                      self.pci_devices_info.append((match[i][0], match[i][1]))
>              else:
>                  self.pci_devices_info.append((match[i][0], match[i][1]))
> diff --git a/framework/settings.py b/framework/settings.py
> index 1a561dda..b2c371f8 100644
> --- a/framework/settings.py
> +++ b/framework/settings.py
> @@ -166,8 +166,8 @@ DRIVERS = {
>      "cavium_a034": "thunder-nicvf",
>      "cavium_0011": "thunder-nicvf",
>      "IXGBE_10G-X550EM_X_SFP": "ixgbe",
> -    "cavium_a063": "octeontx2-nicpf",
> -    "cavium_a064": "octeontx2-nicvf",
> +    "cavium_a063": "rvu_nicpf",
> +    "cavium_a064": "rvu_nicvf",
>
>
> Okay disregard my comment from above about nicvf, I see you've added it. Please do update the commit message to reflect that this is included. And, while you're at it, you can update the commit message to accord with the style guidelines for submitting DPDK patches as seen here on sections 9.6 and 9.7 https://doc.dpdk.org/guides/contributing/patches.html
>
> This change message is already part of the commit message.
> Regarding the style guideline, I have tried to incorporate in the updated patch, please help me in case further changes are needed
>
>
> :)
>
>      "ICE_100G-E810C_QSFP": "ice",
>      "ICE_25G-E810C_SFP": "ice",
>      "ICE_25G-E823C_QSFP": "ice",
> diff --git a/framework/ssh_pexpect.py b/framework/ssh_pexpect.py
> index 2132c066..a96087d2 100644
> --- a/framework/ssh_pexpect.py
> +++ b/framework/ssh_pexpect.py
> @@ -186,7 +186,7 @@ class SSHPexpect:
>
>      def get_output_all(self):
>          output = self.session.before
> -        output.replace("[PEXPECT]", "")
> +        output = output.replace("[PEXPECT]#", "")
>
>
> So, this is just removing the leading "#" from the output? I'm not sure we should be making styling changes for a "deprecated" repository. If I look at this example output, from one of the CI runs of this testsuite on a quickassist card:
>
> # Device 1 on lcore 40
>
> # total operations: 30000000
> # Buffer size: 1024
> # Burst size: 32
> #     Number of bursts: 937500
> #
> #                 Total   Average   Maximum   Minimum
> #  enqueued    30000000        32        32        32
> #  dequeued    30000000        32        32         1
> #    cycles 91892876610      3063    304625       485
> # time [us]  3675715064   122.520 12185.000    19.400
>
> I imagine the DTS maintainers were aware of this and thought it was the most clear way to print the logs. I don't think we need to override that decision now.
>
> This is not a styling change.
> output.replace will not replace the special prompt "[PEXPECT]" directly on the variable.
> output variable needs to be overwritten to have the output strip of the prompt [I am overwriting output ]
> special prompt variable is "[PEXPECT]#", not the "[PEXPECT]". hence output will have unwanted "#" string.
>
>
>
>          return output
>
>      def close(self, force=False):
> diff --git a/nics/net_device.py b/nics/net_device.py
> index 0f9c1af4..dd0d0e92 100644
> --- a/nics/net_device.py
> +++ b/nics/net_device.py
> @@ -43,6 +43,7 @@ class NetDevice(object):
>          if not isinstance(crb, Crb):
>              raise Exception("  Please input the instance of Crb!!!")
>          self.crb = crb
> +        self.__send_expect = self.crb.send_expect
>          self.domain_id = domain_id
>          self.bus_id = bus_id
>          self.devfun_id = devfun_id
> @@ -727,6 +728,8 @@ class NetDevice(object):
>              "/sys/bus/pci/devices/%s:%s:%s" % (domain_id, bus_id, devfun_id),
>              vf_reg_file,
>          )
> +        self.__send_expect("echo 0 > %s" %
> +                           (vf_reg_path), "# ")
>
>
> So, the pre-existing VFs need to be destroyed before a new set of VFs can be created? Do you know if this is Marvell specific, or a universal behavior?
>
> I am not aware of the other boards, but it is the clean way of creating the VFs.
>
>
>          self.__send_expect("echo %d > %s" % (int(vf_num), vf_reg_path), "# ")
>
>      def generate_sriov_vfs_linux_igb_uio(self, domain_id, bus_id, devfun_id, vf_num):
> diff --git a/tests/TestSuite_crypto_perf_cryptodev_perf.py b/tests/TestSuite_crypto_perf_cryptodev_perf.py
> index a3f48eee..b435cefa 100644
> --- a/tests/TestSuite_crypto_perf_cryptodev_perf.py
> +++ b/tests/TestSuite_crypto_perf_cryptodev_perf.py
> @@ -59,14 +59,74 @@ class TestCryptoPerfCryptodevPerf(TestCase):
>                  "# ",
>                  5,
>              )
> -
> -        cc.bind_qat_device(self, "vfio-pci")
> +        if self.nic == "cavium_a063":
> +            count = len(self.get_suite_cfg().get('l').split(",")) - 1
> +            cc.bind_qat_device(self, "vfio-pci", generate_vfs=True, vf_count=count)
> +        else:
> +            cc.bind_qat_device(self, "vfio-pci")
>
>
> If we are going to use this function for non-QAT devices, can we please update the function name to prevent confusion? It looks like it's a short list:
>
> Addressed this in  the updated patch
>
> probb@d121003:~/repos/dts> grep -r "bind_qat_d" .
> ./tests/TestSuite_compressdev_qat_pmd.py:        cc.bind_qat_device(self, self.drivername)
> ./tests/TestSuite_crypto_perf_cryptodev_perf.py:        cc.bind_qat_device(self, "vfio-pci")
> ./tests/TestSuite_fips_cryptodev.py:        cc.bind_qat_device(self, "vfio-pci")
> ./tests/TestSuite_ipsec_gw_cryptodev_func.py:        cc.bind_qat_device(self, self.drivername)
> ./tests/TestSuite_l2fwd_cryptodev_func.py:        cc.bind_qat_device(self, self.drivername)
> ./tests/TestSuite_virtio_ipsec_cryptodev_func.py:        cc.bind_qat_device(self, self.drivername)
> ./tests/TestSuite_virtio_perf_cryptodev_func.py:        cc.bind_qat_device(self, self.drivername)
> ./tests/TestSuite_virtio_unit_cryptodev_func.py:        cc.bind_qat_device(self, self.drivername)
> ./tests/compress_common.py:def bind_qat_device(test_case, driver="igb_uio"):
> ./tests/cryptodev_common.py:def bind_qat_device(test_case, driver="igb_uio"):
>
>
>
>
>          src_files = ["dep/test_aes_cbc.data", "dep/test_aes_gcm.data"]
>          self.dut_file_dir = "/tmp"
>          for file in src_files:
>              self.dut.session.copy_file_to(file, self.dut_file_dir)
>
>      def tear_down_all(self):
> +        if self.nic == "cavium_a063":
> +            cc.bind_qat_device(self, "vfio-pci", generate_vfs=True,
> +                               vf_count=2)
> +            import pandas as pd
>
>
> I don't have a problem with this, but I'm just wondering why this is imported here as opposed to the top of the file?
>
> wrt to pandas import, as we are generating this report only in Marvell case. this import should not affect others
>
>
> +            tuples = [('TestCase',), ('performance', 'failed_enq'),
> +                      ('performance', 'failed_deq'),
> +                      ('performance', 'throughput_mops'),
> +                      ('performance', 'cycle_buf'),
> +                      ('performance', 'throughput', 'value'),
> +                      ('performance', 'throughput', 'delta'),
> +                      ('parameters', 'core_num/thread_num'),
> +                      ('parameters', 'frame_size'),
> +                      ('parameters', 'burst_size'),
> +                      ('parameters', 'total_ops'), ('status',)]
> +            mindex = pd.MultiIndex.from_tuples(tuples)
> +            index = 0
> +            df = pd.DataFrame([], columns=mindex)
> +            for tname, data in self._perf_result.items():
> +                for dt in data:
> +                    for key, value in dt.items():
> +                        df.loc[index, 'TestCase'] = tname
> +                        if key == 'status':
> +                            df.loc[index, (key)] = value
> +                            continue
> +                        for item in value:
> +                            if item['name'] == 'throughput':
> +                                df.loc[index, (key, item['name'], 'value')] = item['value']
> +                                df.loc[index, (key, item['name'], 'delta')] = item['delta']
> +                            else:
> +                                df.loc[index, (key, item['name'])] = item['value']
> +                    index += 1
> +            perf_xl = self.logger.log_path + "/" + "perf_cryptodev_result.xls"
> +            writer = pd.ExcelWriter(perf_xl, engine='xlsxwriter')
> +            wb  = writer.book
> +            df.to_excel(writer, sheet_name='Sheet1')
> +            ws = writer.sheets["Sheet1"]
> +            merge_format = wb.add_format({"bold": 1,"border": 1,"align": "center",
> +                                          "valign": "vcenter"})
> +            ws.merge_range('B1:B3', "TestCase", merge_format)
> +            ws.merge_range('C2:C3', "failed_enq", merge_format)
> +            ws.merge_range('D2:D3', "failed_deq", merge_format)
> +            ws.merge_range('E2:E3', "throughput_mops", merge_format)
> +            ws.merge_range('F2:F3', "throughput_mops", merge_format)
> +            ws.merge_range('I2:I3', "core_num/thread_num", merge_format)
> +            ws.merge_range('J2:J3', "frame_size", merge_format)
> +            ws.merge_range('K2:K3', "burst_size", merge_format)
> +            ws.merge_range('L2:L3', "total_ops", merge_format)
> +            ws.merge_range('M1:M3', "status", merge_format)
> +
> +            format1 = wb.add_format({"bg_color": "#C6EFCE", "font_color": "#006100", "border": 1})
> +            format2 = wb.add_format({"bg_color": "#FFC7CE", "font_color": "#9C0006"})
> +            ws.conditional_format("B1:M3", {"type": "cell", "criteria": ">=", "value": 0 , 'format': format2})
> +            condition = 'B5:M%s'%(5+len(df)-1)
> +            ws.conditional_format(condition, {'type': 'no_blanks', 'format':format1})
> +            ws.autofit()
> +
> +            wb.close()
>          if self._perf_result:
>              with open(
>                  self.logger.log_path + "/" + "perf_cryptodev_result.json", "a"
> @@ -175,12 +235,197 @@ class TestCryptoPerfCryptodevPerf(TestCase):
>      def test_scheduler_zuc(self):
>          self._run_crypto_perf_throughput()
>
> +    # Marvell CN10K Testcases
> +    def _crypto_func_cn10k(self):
> +        """Functional tests Helper
> +        Returns:
> +           None
> +        """
> +        self._run_crypto_func()
> +
> +    def _crypto_perf_cn10k(self):
> +        """Perf tests Helper function
> +        Returns:
> +           None
> +        """
> +        self._run_crypto_perf_throughput(expected="# ",
> +                                         trim_whitespace=False)
> +    def test_perf_mrvl_aes_cbc(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_aes_gcm_encrypt(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_aes_cbc_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_aes_sha2_256_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_zuc_eea3_cipher_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_zuc_eia3_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_aes_gmac_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_null_cipher_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_snow3g_uea2_cipher_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_kasumi_f8_cipher_only(self):
> +        self._crypto_perf_cn10k() #F
> +
> +    def test_perf_mrvl_snow3g_uia2_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_kasumi_f9_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha1_hmac_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha1_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_aes_ctr_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_aes_ctr_snow_3g_uia2_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_aes_cmac_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_3des_cbc_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha1_hmac_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha2_224_hmac_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha2_256_hmac_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha2_384_hmac_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_aes_ctr_snow_3g_uia2_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_aes_ctr_zuc_eia3_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_aes_ctr_aes_cmac_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_zuc_eea3_snow3g_uia2_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_zuc_eea3_zuc_eia3_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_zuc_eea3_aes_cmac_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_snow3g_uea2_snow3g_uia2_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_snow3g_uea2_zuc_eia3_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_snow3g_uea2_aes_cmac_cipher_then_auth(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_md5_hmac_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_aes_ctr_cipher_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_3des_cbc_cipher_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha2_224_hmac_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha2_256_hmac_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha2_384_hmac_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha2_512_hmac_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_aes_xts__cipher_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_des_cbc_cipher_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_md5_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha512_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha384_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha256_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha224_auth_only(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_snow3g_uea2_snow3g_uia2(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha1_hmac_armv8(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_perf_mrvl_sha2_hmac_armv8(self):
> +        self._crypto_perf_cn10k()
> +
> +    def test_mrvl_ptest(self):
> +        self._crypto_func_cn10k()
> +
> +    def test_mrvl_burst(self):
> +        self._crypto_func_cn10k()
> +
> +    def test_mrvl_devtype(self):
> +        self._crypto_func_cn10k()
> +
> +    def test_mrvl_crypto_sha1_hmac_armv8(self):
> +        self._crypto_func_cn10k()
> +
> +    def test_mrvl_chacha20_poly1305_aead(self):
> +        self._crypto_func_cn10k()
> +
> +    def test_mrvl_crypto_sha2_hmac_armv8(self):
> +        self._crypto_func_cn10k()
> +
> +    def test_mrvl_buffer(self):
> +        self._crypto_func_cn10k()
> +
>      # Private functions
>      def _run_crypto_func(self):
> +        """ Runs Crypto functional test.
> +        Returns:
> +            None
> +        """
> +
>          if cc.is_test_skip(self):
>              return
> -
> -        cores = ",".join(self.dut.get_core_list("1S/2C/1T"))
> +        core_list = self.get_suite_cfg().get('core_list',"1S/2C/1T")
> +        cores = ",".join(self.dut.get_core_list(core_list))
>          config = {"l": cores}
>          devices = self._get_crypto_device(1)
>          if not devices:
> @@ -206,12 +451,22 @@ class TestCryptoPerfCryptodevPerf(TestCase):
>
>          out = self.dut.send_command(
>              "cat %s/%s.txt" % (self.dut_file_dir, self.running_case), 30
> -        )
> -
> +        ).strip()
>          self.verify("Error" not in out, "Test function failed")
>          self.verify("failed" not in out, "Test function failed")
> -
> -    def _run_crypto_perf(self):
> +        assert (out != ""), "No output"
> +
> +    def _run_crypto_perf(self, **kwargs):
> +        """ Runs Crypto Performance test.
> +        Args:
> +          **kwargs:
> +              expected: send_expect match prompt
> +              trim_whitespace: trime whitespace from command output
> +        Returns:
> +            None
> +        """
> +        expected = kwargs.get("expected", "#")
> +        trim_whitespace = kwargs.get("trim_whitespace", True)
>          if cc.is_test_skip(self):
>              return "skip"
>
> @@ -222,13 +477,14 @@ class TestCryptoPerfCryptodevPerf(TestCase):
>              return "skip"
>
>          eal_opt_str = cc.get_eal_opt_str(self, devices)
> -        crypto_perf_opt_str = self._get_crypto_perf_opt_str()
> +        crypto_perf_opt_str = self._get_crypto_perf_opt_str(**kwargs)
>
>          cmd_str = cc.get_dpdk_app_cmd_str(
>              self._app_path, eal_opt_str, crypto_perf_opt_str
>          )
>          try:
> -            out = self.dut.send_expect(cmd_str, "#", 600)
> +            out = self.dut.send_expect(cmd_str, expected, 600,
> +                                       trim_whitespace=trim_whitespace)
>          except Exception as ex:
>              self.logger.error(ex)
>              raise ex
> @@ -237,25 +493,44 @@ class TestCryptoPerfCryptodevPerf(TestCase):
>
>          return results
>
> -    def _get_crypto_perf_opt_str(self, override_crypto_perf_opts={}):
> +    def _get_crypto_perf_opt_str(self, **kwargs):
> +        """get crypto perf app option string
> +        args:
> +            **kwargs:
> +                override_crypto_perf_opts- Suite/Case specfic perf
> +                app config to overwrite defaults
> +        Returns:
> +            crypto perf option string
> +        """
> +        override_crypto_perf_opts = kwargs.get("override_crypto_perf_opts", {})
>          return cc.get_opt_str(
>              self, self._default_crypto_perf_opts, override_crypto_perf_opts
>          )
>
>      def _parse_output(self, output):
>          try:
> +            dtype = self.get_case_cfg().get('devtype')
> +            match_str = r"    lcore id|#lcore id"
>              lines = output.split("\r\n")
>              line_nb = len(lines)
>              self.logger.debug("Total output lines: " + str(line_nb))
>
>              for line_index in range(line_nb):
> -                if lines[line_index].startswith("    lcore id"):
> +                if re.match(match_str, lines[line_index]):
>                      self.logger.debug("data output line from: " + str(line_index))
>                      break
> -            data_line = line_index - 1
> +            if dtype == 'crypto_cn10k':
> +                data_line = line_index + 2
> +            else:
> +                data_line = line_index - 1
> +
> +            if len(lines[data_line].split(","))>1:
> +                pattern = re.compile(r',')
> +            else:
> +                pattern = re.compile(r'\s+')
>
>              results = []
> -            pattern = re.compile(r"\s+")
> +
>              for line in lines[data_line:-1]:
>                  print(line)
>                  result = {}
> @@ -334,9 +609,15 @@ class TestCryptoPerfCryptodevPerf(TestCase):
>              cpu_info[key] = value.strip()
>          core, thread = 0, 0
>          lcores = self.get_case_cfg()["l"].split(",")
> +        if 'Core(s) per cluster' in out:
> +            cl_soc1 = 'Core(s) per cluster'
> +            cl_soc2 = 'Cluster(s)'
> +        else:
> +            cl_soc1 = 'Core(s) per socket'
> +            cl_soc2 = 'Socket(s)'
>          for lcore in lcores[1:]:
> -            if int(lcore.strip()) < int(cpu_info["Core(s) per socket"]) * int(
> -                cpu_info["Socket(s)"]
> +            if int(lcore.strip()) < int(cpu_info[cl_soc1]) * int(
> +                cpu_info[cl_soc2]
>              ):
>                  core += 1
>                  thread += 1
> @@ -362,6 +643,11 @@ class TestCryptoPerfCryptodevPerf(TestCase):
>              dev = "crypto_snow3g"
>          elif self.get_case_cfg()["devtype"] == "crypto_zuc":
>              dev = "crypto_zuc"
> +        elif self.get_case_cfg()["devtype"] == "crypto_cn10k":
> +            dev = "crypto_cn10k"
> +            vf = cc.get_qat_devices(self, cpm_num=1, num=num)
> +            device["a"] = ' -a '.join(vf)
> +            device["vdev"] = None
>          elif self.get_case_cfg()["devtype"] == "crypto_scheduler":
>              dev = "crypto_scheduler"
>              w = cc.get_qat_devices(self, cpm_num=3, num=num * 3)
> @@ -388,8 +674,8 @@ class TestCryptoPerfCryptodevPerf(TestCase):
>
>          return device
>
> -    def _run_crypto_perf_throughput(self):
> -        results = self._run_crypto_perf()
> +    def _run_crypto_perf_throughput(self, **kwargs):
> +        results = self._run_crypto_perf(**kwargs)
>          if results == "skip":
>              return
>          self.verify(results, "test results is none, Test Failed")
> diff --git a/tests/cryptodev_common.py b/tests/cryptodev_common.py
> index b550b468..91067b05 100644
> --- a/tests/cryptodev_common.py
> +++ b/tests/cryptodev_common.py
> @@ -8,7 +8,9 @@ from nics.net_device import GetNicObj
>  conf = SuiteConf("cryptodev_sample")
>
>
> -def bind_qat_device(test_case, driver="igb_uio"):
> +def bind_qat_device(test_case, driver="igb_uio", generate_vfs=False,
> +                    vf_count=2, drvr_type='generic'):
> +
>      if driver == "vfio-pci":
>          test_case.dut.send_expect("modprobe vfio", "#", 10)
>          test_case.dut.send_expect("modprobe vfio-pci", "#", 10)
> @@ -38,6 +40,8 @@ def bind_qat_device(test_case, driver="igb_uio"):
>          devfun_id = addr_array[2]
>          pf_port = GetNicObj(test_case.dut, domain_id, bus_id, devfun_id)
>
> +        if generate_vfs:
> +            pf_port.generate_sriov_vfs_linux(domain_id, bus_id, devfun_id, vf_count, drvr_type)
>
>
> Is this different from calling generate_sriov_vfs(), which I assume is the "correct" behavior. There appears to be a common practice in legacy DTS of templating out function names to call instead of using inheritance, so you will have a base function like bind_port(), and then that templates out calls to bind_port_linux(). bind_port_freebsd(), bind_port_windows() etc. It's a little strange but it is what it is and we should probably stick with the existing system if we are going to make additions. But, if there is a practical reason why you did it this way, then okay. :)
>
> I missed it initially, addressed it in updated patch.
>
>
>          sriov_vfs_pci = pf_port.get_sriov_vfs_pci()
>          if not sriov_vfs_pci:
>              raise Exception("can not get vf pci")
> @@ -51,6 +55,11 @@ def bind_qat_device(test_case, driver="igb_uio"):
>
>      test_case.dev = dev
>
> +def bind_mrvl_devices(test_case, driver='vfio-pci', **kwargs):
> +    pci_list = kwargs.get("pci_list").split()
> +    for pci in pci_list:
> +        test_case.dut.bind_eventdev_port(driver, pci)
> +    return pci_list
>
>  def get_qat_devices(test_case, cpm_num=None, num=1):
>      if not cpm_num:
> --
> 2.25.1
>
  
Gnanesh Kambalu Palanethra Nov. 26, 2024, 5:06 p.m. UTC | #5
Hi Cody,

 Cn10k uses single socket only.  --socket-mem 2048 should be sufficient.

We are cross compiling the DPDK sources [DPDK-24.7] on an x86 machine using marvel toolchain and then placing the application (arm64-cn10k-linux-gcc) folder in dut.

I suspect a difference in the compiled application that we are using, due to the difference in compilation procedure.

Regards,
Gnanesh
  
Patrick Robb Nov. 27, 2024, 6:01 p.m. UTC | #6
Hi Gnanesh,

Right, CN10K is a single socket system. I was unsure of why you suggested
--socket-mem 2048 from this though.

I understand that --socket-mem 2048,0 (what is in your patch) should
allocate 2048 for socket 0 (the only socket) and 0 for socket 1
(non-existent). Are you suggesting that Cody try using the cryptodevs using
--socket-mem 2048 instead of --socket-mem 2048,0? We can do so, I just fear
I am misunderstanding what you are getting at.

As far as build procedure goes, I guess I would expect that we end up with
a good application from cross compile (your workflow) and native compile on
the CN10K DUT (our workflow). If a CN10K native compile is broken for the
cryptodev application, I'm not sure the patch can be merged under those
conditions. At a minimum this cross compile requirement would have to be
documented in the test procedure.

Thanks,
Patrick

On Tue, Nov 26, 2024 at 12:06 PM Gnanesh Kambalu Palanethra <
gpalanethra@marvell.com> wrote:

> Hi Cody,
>
>  Cn10k uses single socket only.  *--socket-mem 2048 *should be sufficient.
>
> We are cross compiling the DPDK sources [DPDK-24.7] on an x86 machine
> using marvel toolchain and then placing the application
> (arm64-cn10k-linux-gcc) folder in dut.
>
> I suspect a difference in the compiled application that we are using, due
> to the difference in compilation procedure.
>
> Regards,
> Gnanesh
> ------------------------------
> *From:* Cody Cheng <ccheng@iol.unh.edu>
> *Sent:* Tuesday, November 26, 2024 5:37 AM
> *To:* Gnanesh Kambalu Palanethra <gpalanethra@marvell.com>
> *Cc:* Patrick Robb <probb@iol.unh.edu>; dts@dpdk.org <dts@dpdk.org>;
> JogaRao Nartu <njogarao@marvell.com>; Bharath Rajendra <
> brajendra@marvell.com>; Hiral Shah <hshah@marvell.com>; Jerin Jacob <
> jerinj@marvell.com>
> *Subject:* Re: [EXTERNAL] Re: [PATCH 1/1] Cn10K crypto Tests: Added
> Marvell Cn10K specific crypto Tests to TestCryptoPerfCryptodevPerf
>
> Prioritize security for external emails:
> Confirm sender and content safety before clicking links or opening
> attachments
> Report Suspicious
> <https://us-phishalarm-ewt.proofpoint.com/EWT/v1/CRVmXkqW!uK3X-9D6Drt3vG9UlJKzICAP_jmNmOoxqnprh1nFjVdo94W1uLV3BZAhmDqdXwoEJaVNDA7Cn5WQ6nS8q4OORBNlYS3XFlWDhmYWbm7B2taFaTTq7bcsyPIKCb_czw$>
>
>
> Hey Gnanesh,
>
> I'm currently attempting to run the new test cases on our CN10K board,
> but I am encountering a few issues that I need help resolving.
>
> For setup, I created a VF and bound it with the following commands:
>
> echo 1 > /sys/bus/pci/devices/0002:20:00.0/sriov_numvfs
> ./usertools/dpdk-devbind.py -u 0002:20:00.1
> ./usertools/dpdk-devbind.py -b vfio-pci 0002:20:00.1
>
> When I run dts, I am getting the error below:
>
>
> 18/11/2024 17:51:03               dut.172.18.0.254:
> arm64-native-linuxapp-gcc/app/dpdk-test-crypto-perf  -l 9,10 -a
> 0002:20:00.1 --socket-mem 2048,0 -n 2  -- --ptest throughput --silent
> True --pool-sz 16384 --total-ops 1000000 --burst-sz 32 --buffer-sz 64
> --devtype crypto_cn10k --optype auth-then-cipher --cipher-algo
> snow3g-uea2 --cipher-op encrypt --cipher-key-sz 16 --cipher-iv-sz 16
> --auth-algo aes-cmac --auth-op generate --auth-key-sz 16 --aead-iv-sz
> 0 --digest-sz 4 --csv-friendly True
> 18/11/2024 17:51:04               dut.172.18.0.254: EAL: Detected
> C/resultPU lcores: 24^M
> EAL: Detected NUMA nodes: 1^M
> EAL: Detected static linkage of DPDK^M
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket^M
> EAL: Selected IOVA mode 'VA'^M
> EAL: VFIO support initialized^M
> EAL: Using IOMMU type 1 (Type 1)^M
> CNXK: RoC Model: cn10ka_a1 (HW_PLATFORM)^M
> CRYPTODEV: Creating cryptodev 0002:20:00.1^M
> CRYPTODEV: Initialisation parameters - name: 0002:20:00.1,socket id:
> 0, max queue pairs: 0^M
> CNXK: dev_init():1560 Failed to validate LMT line^M
> CNXK: roc_cpt_dev_init():763 Failed to init roc device^M
> CNXK: cn10k_cpt_pci_probe():80 Failed to initialize roc cpt rc=-14^M
> CRYPTODEV: Closing crypto device 0002:20:00.1^M
> CNXK: cn10k_cpt_pci_probe():116 Could not create device (vendor_id:
> 0x177d device_id: 0xa0f3)^M
> PCI_BUS: Requested device 0002:20:00.1 cannot be used^M
> EAL: Bus (pci) probe failed.^M
> No crypto devices type crypto_cn10k available^M
> USER1: Failed to initialise requested crypto device type^M
>
>
> I observed that if I modify the DPDK build process by adding
> `-Dmax_lcores=24` to the meson arguments, the initial error
> disappears. Unfortunately, this introduces a new issue:
>
>
> TestCryptoPerfCryptodevPerf: Test Case
> test_mrvl_chacha20_poly1305_aead Result FAILED: TIMEOUT on
> arm64-native-linuxapp-gcc/app/dpdk-test-crypto-perf  -l 1,2 -a
> 0002:20:00.1 --socket-mem 2048,0 -n 2  -- --ptest throughput --silent
> True --pool-sz 16384 --total-ops 10000000 --burst-sz 32 --buffer-sz
> 64,128,256,512,1024,2048 --devtype crypto_cn10k --optype aead
> --aead-algo chacha20-poly1305 --aead-op encrypt --aead-key-sz 32
> --aead-iv-sz 12 --aead-aad-sz 32 --digest-sz 16 --csv-friendly True
> >/tmp/test_mrvl_chacha20_poly1305_aead.txt
>    TestCryptoPerfCryptodevPerf: EAL: Detected CPU lcores: 24
> EAL: Detected NUMA nodes: 1
> EAL: Detected static linkage of DPDK
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: Selected IOVA mode 'VA'
> EAL: VFIO support initialized
> EAL: Using IOMMU type 1 (Type 1)
> CNXK: RoC Model: cn10ka_a1 (HW_PLATFORM)
> CRYPTODEV: Creating cryptodev 0002:20:00.1
> CRYPTODEV: Initialisation parameters - name: 0002:20:00.1,socket id:
> 0, max queue pairs: 0
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cpt_lf_misc_irq():58 Err_irq=0x42 pf=31, vf=0
> CNXK:     CPT_LF_Q_BASE: 000000019fbd5400
> CNXK:     CPT_LF_Q_SIZE: 000000000000003d
> CNXK:     CPT_LF_Q_INST_PTR: 0000006200000062
> CNXK:     CPT_LF_Q_GRP_PTR: 0000000100000001
> CNXK:     CPT_LF_CTL: 0000000000000052
> CNXK:     CPT_LF_MISC_INT_ENA_W1S: 000000000000006e
> CNXK:     CPT_LF_MISC_INT: 0000000000000042
> CNXK:     CPT_LF_INPROG: 0000100080010000
> CNXK: Count registers for CPT LF0:
> CNXK:     Encrypted byte count: 0
> CNXK:     Encrypted packet count: 0
> CNXK:     Decrypted byte count: 0
> CNXK:     Decrypted packet count: 0
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cpt_lf_misc_irq():58 Err_irq=0x2 pf=31, vf=0
> CNXK:     CPT_LF_Q_BASE: 000000019fbd5400
> CNXK:     CPT_LF_Q_SIZE: 000000000000003d
> CNXK:     CPT_LF_Q_INST_PTR: 0000006200000062
> CNXK:     CPT_LF_Q_GRP_PTR: 0000000100000001
> CNXK:     CPT_LF_CTL: 0000000000000052
> CNXK:     CPT_LF_MISC_INT_ENA_W1S: 000000000000006e
> CNXK:     CPT_LF_MISC_INT: 0000000000000002
> CNXK:     CPT_LF_INPROG: 0000100080010000
> CNXK: Count registers for CPT LF0:
> CNXK:     Encrypted byte count: 0
> CNXK:     Encrypted packet count: 0
> CNXK:     Decrypted byte count: 0
> CNXK:     Decrypted packet count: 0
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
> CNXK: cpt_lf_misc_irq():58 Err_irq=0x2 pf=31, vf=0
> CNXK:     CPT_LF_Q_BASE: 000000019fbd5400
> CNXK:     CPT_LF_Q_SIZE: 000000000000003d
> CNXK:     CPT_LF_Q_INST_PTR: 0000006200000062
> CNXK:     CPT_LF_Q_GRP_PTR: 0000000100000001
> CNXK:     CPT_LF_CTL: 0000000000000052
> CNXK:     CPT_LF_MISC_INT_ENA_W1S: 000000000000006e
> CNXK:     CPT_LF_MISC_INT: 0000000000000002
> CNXK:     CPT_LF_INPROG: 0000100080010000
> CNXK: Count registers for CPT LF0:
> CNXK:     Encrypted byte count: 0
> CNXK:     Encrypted packet count: 0
> CNXK:     Decrypted byte count: 0
> CNXK:     Decrypted packet count: 0
> CNXK: cpt_lf_misc_irq():58 Err_irq=0x2 pf=31, vf=0
> CNXK:     CPT_LF_Q_BASE: 000000019fbd5400
> CNXK:     CPT_LF_Q_SIZE: 000000000000003d
> CNXK:     CPT_LF_Q_INST_PTR: 0000006200000062
> CNXK:     CPT_LF_Q_GRP_PTR: 0000000100000001
> CNXK:     CPT_LF_CTL: 0000000000000052
> CNXK:     CPT_LF_MISC_INT_ENA_W1S: 000000000000006e
> CNXK:     CPT_LF_MISC_INT: 0000000000000002
> CNXK:     CPT_LF_INPROG: 0000100080010000
> CNXK: Count registers for CPT LF0:
> CNXK:     Encrypted byte count: 0
> CNXK:     Encrypted packet count: 0
> CNXK:     Decrypted byte count: 0
> CNXK:     Decrypted packet count: 0
> CNXK: cpt_lf_misc_irq():58 Err_irq=0x2 pf=31, vf=0
> CNXK:     CPT_LF_Q_BASE: 000000019fbd5400
> CNXK:     CPT_LF_Q_SIZE: 000000000000003d
> CNXK:     CPT_LF_Q_INST_PTR: 0000006200000062
> CNXK:     CPT_LF_Q_GRP_PTR: 0000000100000001
> CNXK:     CPT_LF_CTL: 0000000000000052
> CNXK:     CPT_LF_MISC_INT_ENA_W1S: 000000000000006e
> CNXK:     CPT_LF_MISC_INT: 0000000000000002
> CNXK:     CPT_LF_INPROG: 0000100080010000
> CNXK: Count registers for CPT LF0:
> CNXK:     Encrypted byte count: 0
> CNXK:     Encrypted packet count: 0
> CNXK:     Decrypted byte count: 0
> CNXK:     Decrypted packet count: 0
> CNXK: cpt_lf_misc_irq():58 Err_irq=0x2 pf=31, vf=0
> CNXK:     CPT_LF_Q_BASE: 000000019fbd5400
> CNXK:     CPT_LF_Q_SIZE: 000000000000003d
> CNXK:     CPT_LF_Q_INST_PTR: 0000006200000062
> CNXK:     CPT_LF_Q_GRP_PTR: 0000000100000001
> CNXK:     CPT_LF_CTL: 0000000000000052
> CNXK:     CPT_LF_MISC_INT_ENA_W1S: 000000000000006e
> CNXK:     CPT_LF_MISC_INT: 0000000000000002
> CNXK:     CPT_LF_INPROG: 0000100080010000
> CNXK: Count registers for CPT LF0:
> CNXK:     Encrypted byte count: 0
> CNXK:     Encrypted packet count: 0
> CNXK:     Decrypted byte count: 0
> CNXK:     Decrypted packet count: 0
> CNXK: cpt_lf_misc_irq():58 Err_irq=0x2 pf=31, vf=0
> CNXK:     CPT_LF_Q_BASE: 000000019fbd5400
> CNXK:     CPT_LF_Q_SIZE: 000000000000003d
> CNXK:     CPT_LF_Q_INST_PTR: 0000006200000062
> CNXK:     CPT_LF_Q_GRP_PTR: 0000000100000001
> CNXK:     CPT_LF_CTL: 0000000000000052
> CNXK:     CPT_LF_MISC_INT_ENA_W1S: 000000000000006e
> CNXK:     CPT_LF_MISC_INT: 0000000000000002
> CNXK:     CPT_LF_INPROG: 0000100080010000
> CNXK: Count registers for CPT LF0:
> CNXK:     Encrypted byte count: 0
> CNXK:     Encrypted packet count: 0
> CNXK:     Decrypted byte count: 0
> CNXK:     Decrypted packet count: 0
> CNXK: cn10k_cpt_dequeue_burst():1325 Request timed out
> CNXK: Lcore ID: 2, LF/QP ID: 0
> CNXK:
> CNXK: S/w pending queue:
> CNXK: Head: 95
> CNXK: Tail: 96
> CNXK: Mask: 0x7ff
> CNXK: Inflight count: 2047
> CNXK:
> CNXK: H/w pending queue:
> CNXK: Inflight in engines: 0
> CNXK: NQ ptr: 0x62
> CNXK: DQ ptr: 0x62
> CNXK: Insts waiting in CPT: 0
> CNXK:
> CNXK: CPT AF registers:
> CNXK:     CPT_AF_LF0_CTL: 0x0007000000040001
> CNXK:     CPT_AF_LF0_CTL2: 0x0000000000000002
> CNXK:     inst_req_pc: 0x0000000000000163
> CNXK:     inst_lat_pc: 0x00000000002e1a2e
> CNXK:     rd_req_pc: 0x0000000000001641
> CNXK:     rd_lat_pc: 0x00000000001146b4
> CNXK:     rd_uc_pc: 0x0000000000000fd0
> CNXK:     active_cycles_pc: 0x0000000000f978e3
> CNXK:     ctx_mis_pc: 0x0000000000000005
> CNXK:     ctx_hit_pc: 0x000000000000015b
> CNXK:     ctx_aop_pc: 0x0000000000000000
> CNXK:     ctx_aop_lat_pc: 0x0000000000000000
> CNXK:     ctx_ifetch_pc: 0x000000000000000a
> CNXK:     ctx_ifetch_lat_pc: 0x00000000000009be
> CNXK:     ctx_ffetch_pc: 0x0000000000000160
> CNXK:     ctx_ffetch_lat_pc: 0x0000000000000992
> CNXK:     ctx_wback_pc: 0x0000000000000160
> CNXK:     ctx_wback_lat_pc: 0x0000000000000992
> CNXK:     ctx_psh_pc: 0x0000000000000160
> CNXK:     ctx_psh_lat_pc: 0x0000000000000992
> CNXK:     ctx_err: 0x0000000000000000
> CNXK:     ctx_enc_id: 0x0000000000000000
> CNXK:     ctx_flush_timer: 0x000000000002faf0
> CNXK:     rxc_time: 0x000000000003873a
> CNXK:     rxc_time_cfg: 0x0000000000000000
> CNXK:     rxc_active_sts: 0x0000000000000000
> CNXK:     rxc_zombie_sts: 0x0000000000000000
> CNXK:     rxc_dfrg: 0x0000000000000000
> CNXK:     x2p_link_cfg0: 0x0000000000040000
> CNXK:     x2p_link_cfg1: 0x0000000000040000
> CNXK:     busy_sts_ae: 0x0000000000000000
> CNXK:     free_sts_ae: 0x0000000000ffffff
> CNXK:     busy_sts_se: 0x0000000000000000
> CNXK:     free_sts_se: 0xffffffffffffffff
> CNXK:     busy_sts_ie: 0x0000000000000000
> CNXK:     free_sts_ie: 0x00ffffffffffffff
> CNXK:     exe_err_info: 0x0000000000000000
> CNXK:     cptclk_cnt: 0x000000169fa8634c
> CNXK:     diag: 0x00000000010000ff
> CNXK: cn10k_cpt_dequeue_burst():1325 Request timed out
> CNXK: Lcore ID: 2, LF/QP ID: 0
> CNXK:
> CNXK: S/w pending queue:
> CNXK: Head: 95
> CNXK: Tail: 96
> CNXK: Mask: 0x7ff
> CNXK: Inflight count: 2047
> CNXK:
> CNXK: H/w pending queue:
> CNXK: Inflight in engines: 0
> CNXK: NQ ptr: 0x62
> CNXK: DQ ptr: 0x62
> CNXK: Insts waiting in CPT: 0
> CNXK:
> CNXK: CPT AF registers:
> CNXK:     CPT_AF_LF0_CTL: 0x0007000000040001
> CNXK:     CPT_AF_LF0_CTL2: 0x0000000000000002
> CNXK:     inst_req_pc: 0x0000000000000163
> CNXK:     inst_lat_pc: 0x00000000002e1a2e
> CNXK:     rd_req_pc: 0x0000000000001641
> CNXK:     rd_lat_pc: 0x00000000001146b4
> CNXK:     rd_uc_pc: 0x0000000000000fd0
> CNXK:     active_cycles_pc: 0x0000000000f9b66c
> CNXK:     ctx_mis_pc: 0x0000000000000005
> CNXK:     ctx_hit_pc: 0x000000000000015b
> CNXK:     ctx_aop_pc: 0x0000000000000000
> CNXK:     ctx_aop_lat_pc: 0x0000000000000000
> CNXK:     ctx_ifetch_pc: 0x000000000000000a
> CNXK:     ctx_ifetch_lat_pc: 0x00000000000009be
> CNXK:     ctx_ffetch_pc: 0x0000000000000160
> CNXK:     ctx_ffetch_lat_pc: 0x0000000000000992
> CNXK:     ctx_wback_pc: 0x0000000000000160
> CNXK:     ctx_wback_lat_pc: 0x0000000000000992
> CNXK:     ctx_psh_pc: 0x0000000000000160
> CNXK:     ctx_psh_lat_pc: 0x0000000000000992
> CNXK:     ctx_err: 0x0000000000000000
> CNXK:     ctx_enc_id: 0x0000000000000000
> CNXK:     ctx_flush_timer: 0x000000000002faf0
> CNXK:     rxc_time: 0x0000000000009ce5
> CNXK:     rxc_time_cfg: 0x0000000000000000
> CNXK:     rxc_active_sts: 0x0000000000000000
> CNXK:     rxc_zombie_sts: 0x0000000000000000
> CNXK:     rxc_dfrg: 0x0000000000000000
> CNXK:     x2p_link_cfg0: 0x0000000000040000
> CNXK:     x2p_link_cfg1: 0x0000000000040000
> CNXK:     busy_sts_ae: 0x0000000000000000
> CNXK:     free_sts_ae: 0x0000000000ffffff
> CNXK:     busy_sts_se: 0x0000000000000000
> CNXK:     free_sts_se: 0xffffffffffffffff
> CNXK:     busy_sts_ie: 0x0000000000000000
> CNXK:     free_sts_ie: 0x00ffffffffffffff
> CNXK:     exe_err_info: 0x0000000000000000
> CNXK:     cptclk_cnt: 0x00000016a6eb1638
> CNXK:     diag: 0x00000000010000ff
> CNXK: cn10k_cpt_dequeue_burst():1325 Request timed out
> CNXK: Lcore ID: 2, LF/QP ID: 0
>
> This `cn10k_cpt_dequeue_burst():1325 Request timed out` error will
> keep repeating until DTS times out.
>
> Could you please take a look and let me know if there's any
> configuration or additional step that I might have missed to resolve
> these issues?
> I'd be happy to provide more details if needed.
>
> Thank you for your help!
>
> Best Regards,
> Cody Cheng
>
> On Thu, Nov 14, 2024 at 2:23 PM Gnanesh Kambalu Palanethra
> <gpalanethra@marvell.com> wrote:
> >
> >
> > Hi Patrick,
> >
> > Please find my responses inline.  I am submitting  an updated patch based on your comments.
> >
> >
> > Regards,
> > Gnanesh
> > ________________________________
> > From: Patrick Robb <probb@iol.unh.edu>
> > Sent: Friday, November 1, 2024 3:45 AM
> > To: Gnanesh Kambalu Palanethra <gpalanethra@marvell.com>
> > Cc: dts@dpdk.org <dts@dpdk.org>; JogaRao Nartu <njogarao@marvell.com>; Bharath Rajendra <brajendra@marvell.com>; Cody Cheng <ccheng@iol.unh.edu>; Hiral Shah <hshah@marvell.com>; Jerin Jacob <jerinj@marvell.com>
> > Subject: [EXTERNAL] Re: [PATCH 1/1] Cn10K crypto Tests: Added Marvell Cn10K specific crypto Tests to TestCryptoPerfCryptodevPerf
> >
> > Prioritize security for external emails:
> > Confirm sender and content safety before clicking links or opening attachments
> > Report Suspicious
> >
> > Hi Gnanesh, thanks for sending this. My name is Patrick Robb, and I'm a CI Testing Lab manager at the DPDK Community Lab hosted at UNH, and a maintainer for the "new" DTS framework which exists within the DPDK repo and should be replacing the legacy DTS project in the long term(what this patch is for). We do have a Marvell CN10K board at the lab, and I would love to begin running some crypto testing from that if possible (so, thanks again for this series).
> >
> > So, most likely we will want to provide a review from UNH and test this, so that it can be merged to DTS (technically I was granted the authority to apply patches to the legacy repo).
> >
> > I am adding Cody to this thread as I think he can assist me on this. Cody I am going to provide some comments inline as an initial review for this series, and then we should apply this series locally on the traffic generator/test engine for the CN10K (Intel-1 in our lab), dry run the test which Gnanesh has added, and report back to him. If all goes well, it should be possible to merge.
> >
> > Long term, we will want to circle back and add this test to the new DTS framework, but it is not ready for this test yet, as currently new DTS is testpmd only - there is not cryptodev support. Although, adding such support to new DTS is something I would love to work with you on in the next year if you have time available for development towards DPDK testing! Anyways, comments inline:
> >
> > On Wed, Oct 30, 2024 at 12:44 PM Gnanesh <gpalanethra@marvell.com> wrote:
> >
> >     Changes included in this patch:
> >     -- New TestCases are added for Cn10K CPT Hardware crypto accelerator.
> >     -- when Testing newly added Testcases it is found Necessary to make Below Changes
> >           1> tests/cryptodev_common.py
> >                --> bind_qat_device function is updated to generate VFs for the given crypto_dev_id
> >                --> added New function bind_mrvl_devices to accept list of PCI Ids and bind it to vfio_pci driver.
> >           2> framework/crb.py
> >                --> updated pci_devices_information_uncached_linux Method to handle additional Nic Speeds for Cavium,
> >                    as it was restricting only to 1 GIG NIC speed
> >
> >
> > Thanks, I also made this change on our local DTS project for cn10k, but the patch was not submitted to the "legacy DTS" mailing list here as it was unmaintained (and still is, to David's point). But, if we can do a review it is good to get these things into mainline. Maybe I can rebase our local patch on yours if there were any other changes, and get feedback for cn10k specific questions.
> >
> > I have a hard time understanding how this 10G NIC speed requirement got added in the first place as it was quite peculiar and confused me for a time as it was dropping the ports list! I'm sure you had a similar experience... :)
> >
> >
> >           3> framework/settings.py
> >                --> Changed the default Cavium NIC driver to rvu_nicpf.
> >
> >
> > Thanks, we are also running from this change at the Community Lab. But, I thought you said above you are creating VFs from the bind_qat_device function. So, does rvu_nicvf need to be added as well?
> >
> >
> >           4> nics/net_device.py
> >                --> added Missing expect object
> >           5> conf/crypto_perf_cryptodev_perf.cfg
> >                --> added Marvell Cn10K configs for crypto_perf_cryptodev_perf
> >           6> framework/ssh_pexpect.py
> >                --> output.replace command's output is not set before returning from get_output_all method
> >                    hence Unnecessarily returning Prompt along with the output
> >           7> tests/TestSuite_crypto_perf_cryptodev_perf.py
> >                -->  added Marvell CN10K Testcases
> >
> >
> > Again we will apply these, give them a run, and report back.
> >
> >
> >              Updated Below Private functions
> >                 6.1> _run_crypto_perf() --> receives additional KW arguments
> >                 6.2> _parse_output()  ---> Updated to handle Marvell Cn10K app command output
> >                 6.3> _run_crypto_perf_throughput() --> receives additional KW arguments
> >
> >
> >
> > I don't see any modifications for the build process. Is there a need to include "-Dplatform=cn10k" when setting up the build from meson, or is the default build fine?
> >
> >  right now there is no change to build procedure, we are cross compiling DPDK on x86 and copying the application on to DUT.
> >
> >
> >
> >
> >
> >
> > Signed-off-by: Gnanesh <gpalanethra@marvell.com>
> > ---
> >  conf/crypto_perf_cryptodev_perf.cfg           | 849 ++++++++++++++++++
> >  framework/crb.py                              |   6 +-
> >  framework/settings.py                         |   4 +-
> >  framework/ssh_pexpect.py                      |   2 +-
> >  nics/net_device.py                            |   3 +
> >  tests/TestSuite_crypto_perf_cryptodev_perf.py | 322 ++++++-
> >  tests/cryptodev_common.py                     |  11 +-
> >  7 files changed, 1172 insertions(+), 25 deletions(-)
> >
> > diff --git a/framework/crb.py b/framework/crb.py
> > index 9e3b0a58..2fe9d147 100644
> > --- a/framework/crb.py
> > +++ b/framework/crb.py
> > @@ -374,15 +374,15 @@ class Crb(object):
> >                  pass
> >
> >          for i in range(len(match)):
> > -            # check if device is cavium and check its linkspeed, append only if it is 10G
> > +            # for cavium supported link speed specfied under linkspeeds
> >              if "177d:" in match[i][1]:
> > -                linkspeed = "10000"
> > +                linkspeeds = ["40000", "10000", "25000", "50000", "100000"]
> >
> >
> > Honestly I don't understand why we have this condition at all. Other devices in DTS do not have this check where the portlinks can be dropped in crbs.py. What do you think about updating the accepted linkspeeds vs just removing this step entirely? Can we expect additional linkspeeds to be added for 177d in the future, and have this problem crop up again for a new linkspeed, or is this list of speeds "final" for 177d?
> >
> > This check is now completely Removed in the updated patch.
> >
> >
> >
> >                  nic_linkspeed = self.send_expect(
> >                      "cat /sys/bus/pci/devices/%s/net/*/speed" % match[i][0],
> >                      "# ",
> >                      alt_session=True,
> >                  )
> > -                if nic_linkspeed.split()[0] == linkspeed:
> > +                if nic_linkspeed.split()[0] in  linkspeeds:
> >                      self.pci_devices_info.append((match[i][0], match[i][1]))
> >              else:
> >                  self.pci_devices_info.append((match[i][0], match[i][1]))
> > diff --git a/framework/settings.py b/framework/settings.py
> > index 1a561dda..b2c371f8 100644
> > --- a/framework/settings.py
> > +++ b/framework/settings.py
> > @@ -166,8 +166,8 @@ DRIVERS = {
> >      "cavium_a034": "thunder-nicvf",
> >      "cavium_0011": "thunder-nicvf",
> >      "IXGBE_10G-X550EM_X_SFP": "ixgbe",
> > -    "cavium_a063": "octeontx2-nicpf",
> > -    "cavium_a064": "octeontx2-nicvf",
> > +    "cavium_a063": "rvu_nicpf",
> > +    "cavium_a064": "rvu_nicvf",
> >
> >
> > Okay disregard my comment from above about nicvf, I see you've added it. Please do update the commit message to reflect that this is included. And, while you're at it, you can update the commit message to accord with the style guidelines for submitting DPDK patches as seen here on sections 9.6 and 9.7 https://urldefense.proofpoint.com/v2/url?u=https-3A__doc.dpdk.org_guides_contributing_patches.html&d=DwIFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=Ulaoc1LiTkE5RG69cLkMa7P9xGTL0rP6LosuDS8KOe4&m=vVns1KNfhQSJf3ilhnQOFPpnXVYyIzN1csUdc6rFfMlREYfLTRuin-dnxy8CVeSj&s=IHrV3dmsSA_rqBs3IDZTGA2ce2WeX5SjR8lTna-RWYU&e=
> >
> > This change message is already part of the commit message.
> > Regarding the style guideline, I have tried to incorporate in the updated patch, please help me in case further changes are needed
> >
> >
> > :)
> >
> >      "ICE_100G-E810C_QSFP": "ice",
> >      "ICE_25G-E810C_SFP": "ice",
> >      "ICE_25G-E823C_QSFP": "ice",
> > diff --git a/framework/ssh_pexpect.py b/framework/ssh_pexpect.py
> > index 2132c066..a96087d2 100644
> > --- a/framework/ssh_pexpect.py
> > +++ b/framework/ssh_pexpect.py
> > @@ -186,7 +186,7 @@ class SSHPexpect:
> >
> >      def get_output_all(self):
> >          output = self.session.before
> > -        output.replace("[PEXPECT]", "")
> > +        output = output.replace("[PEXPECT]#", "")
> >
> >
> > So, this is just removing the leading "#" from the output? I'm not sure we should be making styling changes for a "deprecated" repository. If I look at this example output, from one of the CI runs of this testsuite on a quickassist card:
> >
> > # Device 1 on lcore 40
> >
> > # total operations: 30000000
> > # Buffer size: 1024
> > # Burst size: 32
> > #     Number of bursts: 937500
> > #
> > #                 Total   Average   Maximum   Minimum
> > #  enqueued    30000000        32        32        32
> > #  dequeued    30000000        32        32         1
> > #    cycles 91892876610      3063    304625       485
> > # time [us]  3675715064   122.520 12185.000    19.400
> >
> > I imagine the DTS maintainers were aware of this and thought it was the most clear way to print the logs. I don't think we need to override that decision now.
> >
> > This is not a styling change.
> > output.replace will not replace the special prompt "[PEXPECT]" directly on the variable.
> > output variable needs to be overwritten to have the output strip of the prompt [I am overwriting output ]
> > special prompt variable is "[PEXPECT]#", not the "[PEXPECT]". hence output will have unwanted "#" string.
> >
> >
> >
> >          return output
> >
> >      def close(self, force=False):
> > diff --git a/nics/net_device.py b/nics/net_device.py
> > index 0f9c1af4..dd0d0e92 100644
> > --- a/nics/net_device.py
> > +++ b/nics/net_device.py
> > @@ -43,6 +43,7 @@ class NetDevice(object):
> >          if not isinstance(crb, Crb):
> >              raise Exception("  Please input the instance of Crb!!!")
> >          self.crb = crb
> > +        self.__send_expect = self.crb.send_expect
> >          self.domain_id = domain_id
> >          self.bus_id = bus_id
> >          self.devfun_id = devfun_id
> > @@ -727,6 +728,8 @@ class NetDevice(object):
> >              "/sys/bus/pci/devices/%s:%s:%s" % (domain_id, bus_id, devfun_id),
> >              vf_reg_file,
> >          )
> > +        self.__send_expect("echo 0 > %s" %
> > +                           (vf_reg_path), "# ")
> >
> >
> > So, the pre-existing VFs need to be destroyed before a new set of VFs can be created? Do you know if this is Marvell specific, or a universal behavior?
> >
> > I am not aware of the other boards, but it is the clean way of creating the VFs.
> >
> >
> >          self.__send_expect("echo %d > %s" % (int(vf_num), vf_reg_path), "# ")
> >
> >      def generate_sriov_vfs_linux_igb_uio(self, domain_id, bus_id, devfun_id, vf_num):
> > diff --git a/tests/TestSuite_crypto_perf_cryptodev_perf.py b/tests/TestSuite_crypto_perf_cryptodev_perf.py
> > index a3f48eee..b435cefa 100644
> > --- a/tests/TestSuite_crypto_perf_cryptodev_perf.py
> > +++ b/tests/TestSuite_crypto_perf_cryptodev_perf.py
> > @@ -59,14 +59,74 @@ class TestCryptoPerfCryptodevPerf(TestCase):
> >                  "# ",
> >                  5,
> >              )
> > -
> > -        cc.bind_qat_device(self, "vfio-pci")
> > +        if self.nic == "cavium_a063":
> > +            count = len(self.get_suite_cfg().get('l').split(",")) - 1
> > +            cc.bind_qat_device(self, "vfio-pci", generate_vfs=True, vf_count=count)
> > +        else:
> > +            cc.bind_qat_device(self, "vfio-pci")
> >
> >
> > If we are going to use this function for non-QAT devices, can we please update the function name to prevent confusion? It looks like it's a short list:
> >
> > Addressed this in  the updated patch
> >
> > probb@d121003:~/repos/dts> grep -r "bind_qat_d" .
> > ./tests/TestSuite_compressdev_qat_pmd.py:        cc.bind_qat_device(self, self.drivername)
> > ./tests/TestSuite_crypto_perf_cryptodev_perf.py:        cc.bind_qat_device(self, "vfio-pci")
> > ./tests/TestSuite_fips_cryptodev.py:        cc.bind_qat_device(self, "vfio-pci")
> > ./tests/TestSuite_ipsec_gw_cryptodev_func.py:        cc.bind_qat_device(self, self.drivername)
> > ./tests/TestSuite_l2fwd_cryptodev_func.py:        cc.bind_qat_device(self, self.drivername)
> > ./tests/TestSuite_virtio_ipsec_cryptodev_func.py:        cc.bind_qat_device(self, self.drivername)
> > ./tests/TestSuite_virtio_perf_cryptodev_func.py:        cc.bind_qat_device(self, self.drivername)
> > ./tests/TestSuite_virtio_unit_cryptodev_func.py:        cc.bind_qat_device(self, self.drivername)
> > ./tests/compress_common.py:def bind_qat_device(test_case, driver="igb_uio"):
> > ./tests/cryptodev_common.py:def bind_qat_device(test_case, driver="igb_uio"):
> >
> >
> >
> >
> >          src_files = ["dep/test_aes_cbc.data", "dep/test_aes_gcm.data"]
> >          self.dut_file_dir = "/tmp"
> >          for file in src_files:
> >              self.dut.session.copy_file_to(file, self.dut_file_dir)
> >
> >      def tear_down_all(self):
> > +        if self.nic == "cavium_a063":
> > +            cc.bind_qat_device(self, "vfio-pci", generate_vfs=True,
> > +                               vf_count=2)
> > +            import pandas as pd
> >
> >
> > I don't have a problem with this, but I'm just wondering why this is imported here as opposed to the top of the file?
> >
> > wrt to pandas import, as we are generating this report only in Marvell case. this import should not affect others
> >
> >
> > +            tuples = [('TestCase',), ('performance', 'failed_enq'),
> > +                      ('performance', 'failed_deq'),
> > +                      ('performance', 'throughput_mops'),
> > +                      ('performance', 'cycle_buf'),
> > +                      ('performance', 'throughput', 'value'),
> > +                      ('performance', 'throughput', 'delta'),
> > +                      ('parameters', 'core_num/thread_num'),
> > +                      ('parameters', 'frame_size'),
> > +                      ('parameters', 'burst_size'),
> > +                      ('parameters', 'total_ops'), ('status',)]
> > +            mindex = pd.MultiIndex.from_tuples(tuples)
> > +            index = 0
> > +            df = pd.DataFrame([], columns=mindex)
> > +            for tname, data in self._perf_result.items():
> > +                for dt in data:
> > +                    for key, value in dt.items():
> > +                        df.loc[index, 'TestCase'] = tname
> > +                        if key == 'status':
> > +                            df.loc[index, (key)] = value
> > +                            continue
> > +                        for item in value:
> > +                            if item['name'] == 'throughput':
> > +                                df.loc[index, (key, item['name'], 'value')] = item['value']
> > +                                df.loc[index, (key, item['name'], 'delta')] = item['delta']
> > +                            else:
> > +                                df.loc[index, (key, item['name'])] = item['value']
> > +                    index += 1
> > +            perf_xl = self.logger.log_path + "/" + "perf_cryptodev_result.xls"
> > +            writer = pd.ExcelWriter(perf_xl, engine='xlsxwriter')
> > +            wb  = writer.book
> > +            df.to_excel(writer, sheet_name='Sheet1')
> > +            ws = writer.sheets["Sheet1"]
> > +            merge_format = wb.add_format({"bold": 1,"border": 1,"align": "center",
> > +                                          "valign": "vcenter"})
> > +            ws.merge_range('B1:B3', "TestCase", merge_format)
> > +            ws.merge_range('C2:C3', "failed_enq", merge_format)
> > +            ws.merge_range('D2:D3', "failed_deq", merge_format)
> > +            ws.merge_range('E2:E3', "throughput_mops", merge_format)
> > +            ws.merge_range('F2:F3', "throughput_mops", merge_format)
> > +            ws.merge_range('I2:I3', "core_num/thread_num", merge_format)
> > +            ws.merge_range('J2:J3', "frame_size", merge_format)
> > +            ws.merge_range('K2:K3', "burst_size", merge_format)
> > +            ws.merge_range('L2:L3', "total_ops", merge_format)
> > +            ws.merge_range('M1:M3', "status", merge_format)
> > +
> > +            format1 = wb.add_format({"bg_color": "#C6EFCE", "font_color": "#006100", "border": 1})
> > +            format2 = wb.add_format({"bg_color": "#FFC7CE", "font_color": "#9C0006"})
> > +            ws.conditional_format("B1:M3", {"type": "cell", "criteria": ">=", "value": 0 , 'format': format2})
> > +            condition = 'B5:M%s'%(5+len(df)-1)
> > +            ws.conditional_format(condition, {'type': 'no_blanks', 'format':format1})
> > +            ws.autofit()
> > +
> > +            wb.close()
> >          if self._perf_result:
> >              with open(
> >                  self.logger.log_path + "/" + "perf_cryptodev_result.json", "a"
> > @@ -175,12 +235,197 @@ class TestCryptoPerfCryptodevPerf(TestCase):
> >      def test_scheduler_zuc(self):
> >          self._run_crypto_perf_throughput()
> >
> > +    # Marvell CN10K Testcases
> > +    def _crypto_func_cn10k(self):
> > +        """Functional tests Helper
> > +        Returns:
> > +           None
> > +        """
> > +        self._run_crypto_func()
> > +
> > +    def _crypto_perf_cn10k(self):
> > +        """Perf tests Helper function
> > +        Returns:
> > +           None
> > +        """
> > +        self._run_crypto_perf_throughput(expected="# ",
> > +                                         trim_whitespace=False)
> > +    def test_perf_mrvl_aes_cbc(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_aes_gcm_encrypt(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_aes_cbc_cipher_then_auth(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_aes_sha2_256_cipher_then_auth(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_zuc_eea3_cipher_only(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_zuc_eia3_auth_only(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_aes_gmac_auth_only(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_null_cipher_only(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_snow3g_uea2_cipher_only(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_kasumi_f8_cipher_only(self):
> > +        self._crypto_perf_cn10k() #F
> > +
> > +    def test_perf_mrvl_snow3g_uia2_auth_only(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_kasumi_f9_auth_only(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_sha1_hmac_auth_only(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_sha1_auth_only(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_aes_ctr_cipher_then_auth(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_aes_ctr_snow_3g_uia2_cipher_then_auth(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_aes_cmac_cipher_then_auth(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_3des_cbc_cipher_then_auth(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_sha1_hmac_cipher_then_auth(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_sha2_224_hmac_cipher_then_auth(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_sha2_256_hmac_cipher_then_auth(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_sha2_384_hmac_cipher_then_auth(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_aes_ctr_snow_3g_uia2_cipher_then_auth(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_aes_ctr_zuc_eia3_cipher_then_auth(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_aes_ctr_aes_cmac_cipher_then_auth(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_zuc_eea3_snow3g_uia2_cipher_then_auth(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_zuc_eea3_zuc_eia3_cipher_then_auth(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_zuc_eea3_aes_cmac_cipher_then_auth(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_snow3g_uea2_snow3g_uia2_cipher_then_auth(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_snow3g_uea2_zuc_eia3_cipher_then_auth(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_snow3g_uea2_aes_cmac_cipher_then_auth(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_md5_hmac_auth_only(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_aes_ctr_cipher_only(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_3des_cbc_cipher_only(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_sha2_224_hmac_auth_only(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_sha2_256_hmac_auth_only(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_sha2_384_hmac_auth_only(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_sha2_512_hmac_auth_only(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_aes_xts__cipher_only(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_des_cbc_cipher_only(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_md5_auth_only(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_sha512_auth_only(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_sha384_auth_only(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_sha256_auth_only(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_sha224_auth_only(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_snow3g_uea2_snow3g_uia2(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_sha1_hmac_armv8(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_perf_mrvl_sha2_hmac_armv8(self):
> > +        self._crypto_perf_cn10k()
> > +
> > +    def test_mrvl_ptest(self):
> > +        self._crypto_func_cn10k()
> > +
> > +    def test_mrvl_burst(self):
> > +        self._crypto_func_cn10k()
> > +
> > +    def test_mrvl_devtype(self):
> > +        self._crypto_func_cn10k()
> > +
> > +    def test_mrvl_crypto_sha1_hmac_armv8(self):
> > +        self._crypto_func_cn10k()
> > +
> > +    def test_mrvl_chacha20_poly1305_aead(self):
> > +        self._crypto_func_cn10k()
> > +
> > +    def test_mrvl_crypto_sha2_hmac_armv8(self):
> > +        self._crypto_func_cn10k()
> > +
> > +    def test_mrvl_buffer(self):
> > +        self._crypto_func_cn10k()
> > +
> >      # Private functions
> >      def _run_crypto_func(self):
> > +        """ Runs Crypto functional test.
> > +        Returns:
> > +            None
> > +        """
> > +
> >          if cc.is_test_skip(self):
> >              return
> > -
> > -        cores = ",".join(self.dut.get_core_list("1S/2C/1T"))
> > +        core_list = self.get_suite_cfg().get('core_list',"1S/2C/1T")
> > +        cores = ",".join(self.dut.get_core_list(core_list))
> >          config = {"l": cores}
> >          devices = self._get_crypto_device(1)
> >          if not devices:
> > @@ -206,12 +451,22 @@ class TestCryptoPerfCryptodevPerf(TestCase):
> >
> >          out = self.dut.send_command(
> >              "cat %s/%s.txt" % (self.dut_file_dir, self.running_case), 30
> > -        )
> > -
> > +        ).strip()
> >          self.verify("Error" not in out, "Test function failed")
> >          self.verify("failed" not in out, "Test function failed")
> > -
> > -    def _run_crypto_perf(self):
> > +        assert (out != ""), "No output"
> > +
> > +    def _run_crypto_perf(self, **kwargs):
> > +        """ Runs Crypto Performance test.
> > +        Args:
> > +          **kwargs:
> > +              expected: send_expect match prompt
> > +              trim_whitespace: trime whitespace from command output
> > +        Returns:
> > +            None
> > +        """
> > +        expected = kwargs.get("expected", "#")
> > +        trim_whitespace = kwargs.get("trim_whitespace", True)
> >          if cc.is_test_skip(self):
> >              return "skip"
> >
> > @@ -222,13 +477,14 @@ class TestCryptoPerfCryptodevPerf(TestCase):
> >              return "skip"
> >
> >          eal_opt_str = cc.get_eal_opt_str(self, devices)
> > -        crypto_perf_opt_str = self._get_crypto_perf_opt_str()
> > +        crypto_perf_opt_str = self._get_crypto_perf_opt_str(**kwargs)
> >
> >          cmd_str = cc.get_dpdk_app_cmd_str(
> >              self._app_path, eal_opt_str, crypto_perf_opt_str
> >          )
> >          try:
> > -            out = self.dut.send_expect(cmd_str, "#", 600)
> > +            out = self.dut.send_expect(cmd_str, expected, 600,
> > +                                       trim_whitespace=trim_whitespace)
> >          except Exception as ex:
> >              self.logger.error(ex)
> >              raise ex
> > @@ -237,25 +493,44 @@ class TestCryptoPerfCryptodevPerf(TestCase):
> >
> >          return results
> >
> > -    def _get_crypto_perf_opt_str(self, override_crypto_perf_opts={}):
> > +    def _get_crypto_perf_opt_str(self, **kwargs):
> > +        """get crypto perf app option string
> > +        args:
> > +            **kwargs:
> > +                override_crypto_perf_opts- Suite/Case specfic perf
> > +                app config to overwrite defaults
> > +        Returns:
> > +            crypto perf option string
> > +        """
> > +        override_crypto_perf_opts = kwargs.get("override_crypto_perf_opts", {})
> >          return cc.get_opt_str(
> >              self, self._default_crypto_perf_opts, override_crypto_perf_opts
> >          )
> >
> >      def _parse_output(self, output):
> >          try:
> > +            dtype = self.get_case_cfg().get('devtype')
> > +            match_str = r"    lcore id|#lcore id"
> >              lines = output.split("\r\n")
> >              line_nb = len(lines)
> >              self.logger.debug("Total output lines: " + str(line_nb))
> >
> >              for line_index in range(line_nb):
> > -                if lines[line_index].startswith("    lcore id"):
> > +                if re.match(match_str, lines[line_index]):
> >                      self.logger.debug("data output line from: " + str(line_index))
> >                      break
> > -            data_line = line_index - 1
> > +            if dtype == 'crypto_cn10k':
> > +                data_line = line_index + 2
> > +            else:
> > +                data_line = line_index - 1
> > +
> > +            if len(lines[data_line].split(","))>1:
> > +                pattern = re.compile(r',')
> > +            else:
> > +                pattern = re.compile(r'\s+')
> >
> >              results = []
> > -            pattern = re.compile(r"\s+")
> > +
> >              for line in lines[data_line:-1]:
> >                  print(line)
> >                  result = {}
> > @@ -334,9 +609,15 @@ class TestCryptoPerfCryptodevPerf(TestCase):
> >              cpu_info[key] = value.strip()
> >          core, thread = 0, 0
> >          lcores = self.get_case_cfg()["l"].split(",")
> > +        if 'Core(s) per cluster' in out:
> > +            cl_soc1 = 'Core(s) per cluster'
> > +            cl_soc2 = 'Cluster(s)'
> > +        else:
> > +            cl_soc1 = 'Core(s) per socket'
> > +            cl_soc2 = 'Socket(s)'
> >          for lcore in lcores[1:]:
> > -            if int(lcore.strip()) < int(cpu_info["Core(s) per socket"]) * int(
> > -                cpu_info["Socket(s)"]
> > +            if int(lcore.strip()) < int(cpu_info[cl_soc1]) * int(
> > +                cpu_info[cl_soc2]
> >              ):
> >                  core += 1
> >                  thread += 1
> > @@ -362,6 +643,11 @@ class TestCryptoPerfCryptodevPerf(TestCase):
> >              dev = "crypto_snow3g"
> >          elif self.get_case_cfg()["devtype"] == "crypto_zuc":
> >              dev = "crypto_zuc"
> > +        elif self.get_case_cfg()["devtype"] == "crypto_cn10k":
> > +            dev = "crypto_cn10k"
> > +            vf = cc.get_qat_devices(self, cpm_num=1, num=num)
> > +            device["a"] = ' -a '.join(vf)
> > +            device["vdev"] = None
> >          elif self.get_case_cfg()["devtype"] == "crypto_scheduler":
> >              dev = "crypto_scheduler"
> >              w = cc.get_qat_devices(self, cpm_num=3, num=num * 3)
> > @@ -388,8 +674,8 @@ class TestCryptoPerfCryptodevPerf(TestCase):
> >
> >          return device
> >
> > -    def _run_crypto_perf_throughput(self):
> > -        results = self._run_crypto_perf()
> > +    def _run_crypto_perf_throughput(self, **kwargs):
> > +        results = self._run_crypto_perf(**kwargs)
> >          if results == "skip":
> >              return
> >          self.verify(results, "test results is none, Test Failed")
> > diff --git a/tests/cryptodev_common.py b/tests/cryptodev_common.py
> > index b550b468..91067b05 100644
> > --- a/tests/cryptodev_common.py
> > +++ b/tests/cryptodev_common.py
> > @@ -8,7 +8,9 @@ from nics.net_device import GetNicObj
> >  conf = SuiteConf("cryptodev_sample")
> >
> >
> > -def bind_qat_device(test_case, driver="igb_uio"):
> > +def bind_qat_device(test_case, driver="igb_uio", generate_vfs=False,
> > +                    vf_count=2, drvr_type='generic'):
> > +
> >      if driver == "vfio-pci":
> >          test_case.dut.send_expect("modprobe vfio", "#", 10)
> >          test_case.dut.send_expect("modprobe vfio-pci", "#", 10)
> > @@ -38,6 +40,8 @@ def bind_qat_device(test_case, driver="igb_uio"):
> >          devfun_id = addr_array[2]
> >          pf_port = GetNicObj(test_case.dut, domain_id, bus_id, devfun_id)
> >
> > +        if generate_vfs:
> > +            pf_port.generate_sriov_vfs_linux(domain_id, bus_id, devfun_id, vf_count, drvr_type)
> >
> >
> > Is this different from calling generate_sriov_vfs(), which I assume is the "correct" behavior. There appears to be a common practice in legacy DTS of templating out function names to call instead of using inheritance, so you will have a base function like bind_port(), and then that templates out calls to bind_port_linux(). bind_port_freebsd(), bind_port_windows() etc. It's a little strange but it is what it is and we should probably stick with the existing system if we are going to make additions. But, if there is a practical reason why you did it this way, then okay. :)
> >
> > I missed it initially, addressed it in updated patch.
> >
> >
> >          sriov_vfs_pci = pf_port.get_sriov_vfs_pci()
> >          if not sriov_vfs_pci:
> >              raise Exception("can not get vf pci")
> > @@ -51,6 +55,11 @@ def bind_qat_device(test_case, driver="igb_uio"):
> >
> >      test_case.dev = dev
> >
> > +def bind_mrvl_devices(test_case, driver='vfio-pci', **kwargs):
> > +    pci_list = kwargs.get("pci_list").split()
> > +    for pci in pci_list:
> > +        test_case.dut.bind_eventdev_port(driver, pci)
> > +    return pci_list
> >
> >  def get_qat_devices(test_case, cpm_num=None, num=1):
> >      if not cpm_num:
> > --
> > 2.25.1
> >
>
>
  
Patrick Robb Nov. 27, 2024, 8:22 p.m. UTC | #7
Quick update: Cody tried the cross compile workflow but ended up getting
the same behavior as he describes in his most recent email, so perhaps this
is unrelated to native vs cross compile. We will have to re-assess next
week, as we will be out of office tomorrow and Friday for Thanksgiving.
But, we will try to be in touch quickly.

Thanks,
Patrick

On Wed, Nov 27, 2024 at 1:01 PM Patrick Robb <probb@iol.unh.edu> wrote:

> Hi Gnanesh,
>
> Right, CN10K is a single socket system. I was unsure of why you suggested
> --socket-mem 2048 from this though.
>
> I understand that --socket-mem 2048,0 (what is in your patch) should
> allocate 2048 for socket 0 (the only socket) and 0 for socket 1
> (non-existent). Are you suggesting that Cody try using the cryptodevs using
> --socket-mem 2048 instead of --socket-mem 2048,0? We can do so, I just fear
> I am misunderstanding what you are getting at.
>
> As far as build procedure goes, I guess I would expect that we end up with
> a good application from cross compile (your workflow) and native compile on
> the CN10K DUT (our workflow). If a CN10K native compile is broken for the
> cryptodev application, I'm not sure the patch can be merged under those
> conditions. At a minimum this cross compile requirement would have to be
> documented in the test procedure.
>
> Thanks,
> Patrick
>
> On Tue, Nov 26, 2024 at 12:06 PM Gnanesh Kambalu Palanethra <
> gpalanethra@marvell.com> wrote:
>
>> Hi Cody,
>>
>>  Cn10k uses single socket only.  *--socket-mem 2048 *should be
>> sufficient.
>>
>> We are cross compiling the DPDK sources [DPDK-24.7] on an x86 machine
>> using marvel toolchain and then placing the application
>> (arm64-cn10k-linux-gcc) folder in dut.
>>
>> I suspect a difference in the compiled application that we are using, due
>> to the difference in compilation procedure.
>>
>> Regards,
>> Gnanesh
>> ------------------------------
>> *From:* Cody Cheng <ccheng@iol.unh.edu>
>> *Sent:* Tuesday, November 26, 2024 5:37 AM
>> *To:* Gnanesh Kambalu Palanethra <gpalanethra@marvell.com>
>> *Cc:* Patrick Robb <probb@iol.unh.edu>; dts@dpdk.org <dts@dpdk.org>;
>> JogaRao Nartu <njogarao@marvell.com>; Bharath Rajendra <
>> brajendra@marvell.com>; Hiral Shah <hshah@marvell.com>; Jerin Jacob <
>> jerinj@marvell.com>
>> *Subject:* Re: [EXTERNAL] Re: [PATCH 1/1] Cn10K crypto Tests: Added
>> Marvell Cn10K specific crypto Tests to TestCryptoPerfCryptodevPerf
>>
>> Prioritize security for external emails:
>> Confirm sender and content safety before clicking links or opening
>> attachments
>> Report Suspicious
>> <https://us-phishalarm-ewt.proofpoint.com/EWT/v1/CRVmXkqW!uK3X-9D6Drt3vG9UlJKzICAP_jmNmOoxqnprh1nFjVdo94W1uLV3BZAhmDqdXwoEJaVNDA7Cn5WQ6nS8q4OORBNlYS3XFlWDhmYWbm7B2taFaTTq7bcsyPIKCb_czw$>
>>
>>
>> Hey Gnanesh,
>>
>> I'm currently attempting to run the new test cases on our CN10K board,
>> but I am encountering a few issues that I need help resolving.
>>
>> For setup, I created a VF and bound it with the following commands:
>>
>> echo 1 > /sys/bus/pci/devices/0002:20:00.0/sriov_numvfs
>> ./usertools/dpdk-devbind.py -u 0002:20:00.1
>> ./usertools/dpdk-devbind.py -b vfio-pci 0002:20:00.1
>>
>> When I run dts, I am getting the error below:
>>
>>
>> 18/11/2024 17:51:03               dut.172.18.0.254:
>> arm64-native-linuxapp-gcc/app/dpdk-test-crypto-perf  -l 9,10 -a
>> 0002:20:00.1 --socket-mem 2048,0 -n 2  -- --ptest throughput --silent
>> True --pool-sz 16384 --total-ops 1000000 --burst-sz 32 --buffer-sz 64
>> --devtype crypto_cn10k --optype auth-then-cipher --cipher-algo
>> snow3g-uea2 --cipher-op encrypt --cipher-key-sz 16 --cipher-iv-sz 16
>> --auth-algo aes-cmac --auth-op generate --auth-key-sz 16 --aead-iv-sz
>> 0 --digest-sz 4 --csv-friendly True
>> 18/11/2024 17:51:04               dut.172.18.0.254: EAL: Detected
>> C/resultPU lcores: 24^M
>> EAL: Detected NUMA nodes: 1^M
>> EAL: Detected static linkage of DPDK^M
>> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket^M
>> EAL: Selected IOVA mode 'VA'^M
>> EAL: VFIO support initialized^M
>> EAL: Using IOMMU type 1 (Type 1)^M
>> CNXK: RoC Model: cn10ka_a1 (HW_PLATFORM)^M
>> CRYPTODEV: Creating cryptodev 0002:20:00.1^M
>> CRYPTODEV: Initialisation parameters - name: 0002:20:00.1,socket id:
>> 0, max queue pairs: 0^M
>> CNXK: dev_init():1560 Failed to validate LMT line^M
>> CNXK: roc_cpt_dev_init():763 Failed to init roc device^M
>> CNXK: cn10k_cpt_pci_probe():80 Failed to initialize roc cpt rc=-14^M
>> CRYPTODEV: Closing crypto device 0002:20:00.1^M
>> CNXK: cn10k_cpt_pci_probe():116 Could not create device (vendor_id:
>> 0x177d device_id: 0xa0f3)^M
>> PCI_BUS: Requested device 0002:20:00.1 cannot be used^M
>> EAL: Bus (pci) probe failed.^M
>> No crypto devices type crypto_cn10k available^M
>> USER1: Failed to initialise requested crypto device type^M
>>
>>
>> I observed that if I modify the DPDK build process by adding
>> `-Dmax_lcores=24` to the meson arguments, the initial error
>> disappears. Unfortunately, this introduces a new issue:
>>
>>
>> TestCryptoPerfCryptodevPerf: Test Case
>> test_mrvl_chacha20_poly1305_aead Result FAILED: TIMEOUT on
>> arm64-native-linuxapp-gcc/app/dpdk-test-crypto-perf  -l 1,2 -a
>> 0002:20:00.1 --socket-mem 2048,0 -n 2  -- --ptest throughput --silent
>> True --pool-sz 16384 --total-ops 10000000 --burst-sz 32 --buffer-sz
>> 64,128,256,512,1024,2048 --devtype crypto_cn10k --optype aead
>> --aead-algo chacha20-poly1305 --aead-op encrypt --aead-key-sz 32
>> --aead-iv-sz 12 --aead-aad-sz 32 --digest-sz 16 --csv-friendly True
>> >/tmp/test_mrvl_chacha20_poly1305_aead.txt
>>    TestCryptoPerfCryptodevPerf: EAL: Detected CPU lcores: 24
>> EAL: Detected NUMA nodes: 1
>> EAL: Detected static linkage of DPDK
>> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
>> EAL: Selected IOVA mode 'VA'
>> EAL: VFIO support initialized
>> EAL: Using IOMMU type 1 (Type 1)
>> CNXK: RoC Model: cn10ka_a1 (HW_PLATFORM)
>> CRYPTODEV: Creating cryptodev 0002:20:00.1
>> CRYPTODEV: Initialisation parameters - name: 0002:20:00.1,socket id:
>> 0, max queue pairs: 0
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cpt_lf_misc_irq():58 Err_irq=0x42 pf=31, vf=0
>> CNXK:     CPT_LF_Q_BASE: 000000019fbd5400
>> CNXK:     CPT_LF_Q_SIZE: 000000000000003d
>> CNXK:     CPT_LF_Q_INST_PTR: 0000006200000062
>> CNXK:     CPT_LF_Q_GRP_PTR: 0000000100000001
>> CNXK:     CPT_LF_CTL: 0000000000000052
>> CNXK:     CPT_LF_MISC_INT_ENA_W1S: 000000000000006e
>> CNXK:     CPT_LF_MISC_INT: 0000000000000042
>> CNXK:     CPT_LF_INPROG: 0000100080010000
>> CNXK: Count registers for CPT LF0:
>> CNXK:     Encrypted byte count: 0
>> CNXK:     Encrypted packet count: 0
>> CNXK:     Decrypted byte count: 0
>> CNXK:     Decrypted packet count: 0
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cpt_lf_misc_irq():58 Err_irq=0x2 pf=31, vf=0
>> CNXK:     CPT_LF_Q_BASE: 000000019fbd5400
>> CNXK:     CPT_LF_Q_SIZE: 000000000000003d
>> CNXK:     CPT_LF_Q_INST_PTR: 0000006200000062
>> CNXK:     CPT_LF_Q_GRP_PTR: 0000000100000001
>> CNXK:     CPT_LF_CTL: 0000000000000052
>> CNXK:     CPT_LF_MISC_INT_ENA_W1S: 000000000000006e
>> CNXK:     CPT_LF_MISC_INT: 0000000000000002
>> CNXK:     CPT_LF_INPROG: 0000100080010000
>> CNXK: Count registers for CPT LF0:
>> CNXK:     Encrypted byte count: 0
>> CNXK:     Encrypted packet count: 0
>> CNXK:     Decrypted byte count: 0
>> CNXK:     Decrypted packet count: 0
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cn10k_cpt_dequeue_post_process():1200 HW completion code 0x2
>> CNXK: cn10k_cpt_dequeue_post_process():1207 Request failed with DMA fault
>> CNXK: cpt_lf_misc_irq():58 Err_irq=0x2 pf=31, vf=0
>> CNXK:     CPT_LF_Q_BASE: 000000019fbd5400
>> CNXK:     CPT_LF_Q_SIZE: 000000000000003d
>> CNXK:     CPT_LF_Q_INST_PTR: 0000006200000062
>> CNXK:     CPT_LF_Q_GRP_PTR: 0000000100000001
>> CNXK:     CPT_LF_CTL: 0000000000000052
>> CNXK:     CPT_LF_MISC_INT_ENA_W1S: 000000000000006e
>> CNXK:     CPT_LF_MISC_INT: 0000000000000002
>> CNXK:     CPT_LF_INPROG: 0000100080010000
>> CNXK: Count registers for CPT LF0:
>> CNXK:     Encrypted byte count: 0
>> CNXK:     Encrypted packet count: 0
>> CNXK:     Decrypted byte count: 0
>> CNXK:     Decrypted packet count: 0
>> CNXK: cpt_lf_misc_irq():58 Err_irq=0x2 pf=31, vf=0
>> CNXK:     CPT_LF_Q_BASE: 000000019fbd5400
>> CNXK:     CPT_LF_Q_SIZE: 000000000000003d
>> CNXK:     CPT_LF_Q_INST_PTR: 0000006200000062
>> CNXK:     CPT_LF_Q_GRP_PTR: 0000000100000001
>> CNXK:     CPT_LF_CTL: 0000000000000052
>> CNXK:     CPT_LF_MISC_INT_ENA_W1S: 000000000000006e
>> CNXK:     CPT_LF_MISC_INT: 0000000000000002
>> CNXK:     CPT_LF_INPROG: 0000100080010000
>> CNXK: Count registers for CPT LF0:
>> CNXK:     Encrypted byte count: 0
>> CNXK:     Encrypted packet count: 0
>> CNXK:     Decrypted byte count: 0
>> CNXK:     Decrypted packet count: 0
>> CNXK: cpt_lf_misc_irq():58 Err_irq=0x2 pf=31, vf=0
>> CNXK:     CPT_LF_Q_BASE: 000000019fbd5400
>> CNXK:     CPT_LF_Q_SIZE: 000000000000003d
>> CNXK:     CPT_LF_Q_INST_PTR: 0000006200000062
>> CNXK:     CPT_LF_Q_GRP_PTR: 0000000100000001
>> CNXK:     CPT_LF_CTL: 0000000000000052
>> CNXK:     CPT_LF_MISC_INT_ENA_W1S: 000000000000006e
>> CNXK:     CPT_LF_MISC_INT: 0000000000000002
>> CNXK:     CPT_LF_INPROG: 0000100080010000
>> CNXK: Count registers for CPT LF0:
>> CNXK:     Encrypted byte count: 0
>> CNXK:     Encrypted packet count: 0
>> CNXK:     Decrypted byte count: 0
>> CNXK:     Decrypted packet count: 0
>> CNXK: cpt_lf_misc_irq():58 Err_irq=0x2 pf=31, vf=0
>> CNXK:     CPT_LF_Q_BASE: 000000019fbd5400
>> CNXK:     CPT_LF_Q_SIZE: 000000000000003d
>> CNXK:     CPT_LF_Q_INST_PTR: 0000006200000062
>> CNXK:     CPT_LF_Q_GRP_PTR: 0000000100000001
>> CNXK:     CPT_LF_CTL: 0000000000000052
>> CNXK:     CPT_LF_MISC_INT_ENA_W1S: 000000000000006e
>> CNXK:     CPT_LF_MISC_INT: 0000000000000002
>> CNXK:     CPT_LF_INPROG: 0000100080010000
>> CNXK: Count registers for CPT LF0:
>> CNXK:     Encrypted byte count: 0
>> CNXK:     Encrypted packet count: 0
>> CNXK:     Decrypted byte count: 0
>> CNXK:     Decrypted packet count: 0
>> CNXK: cpt_lf_misc_irq():58 Err_irq=0x2 pf=31, vf=0
>> CNXK:     CPT_LF_Q_BASE: 000000019fbd5400
>> CNXK:     CPT_LF_Q_SIZE: 000000000000003d
>> CNXK:     CPT_LF_Q_INST_PTR: 0000006200000062
>> CNXK:     CPT_LF_Q_GRP_PTR: 0000000100000001
>> CNXK:     CPT_LF_CTL: 0000000000000052
>> CNXK:     CPT_LF_MISC_INT_ENA_W1S: 000000000000006e
>> CNXK:     CPT_LF_MISC_INT: 0000000000000002
>> CNXK:     CPT_LF_INPROG: 0000100080010000
>> CNXK: Count registers for CPT LF0:
>> CNXK:     Encrypted byte count: 0
>> CNXK:     Encrypted packet count: 0
>> CNXK:     Decrypted byte count: 0
>> CNXK:     Decrypted packet count: 0
>> CNXK: cn10k_cpt_dequeue_burst():1325 Request timed out
>> CNXK: Lcore ID: 2, LF/QP ID: 0
>> CNXK:
>> CNXK: S/w pending queue:
>> CNXK: Head: 95
>> CNXK: Tail: 96
>> CNXK: Mask: 0x7ff
>> CNXK: Inflight count: 2047
>> CNXK:
>> CNXK: H/w pending queue:
>> CNXK: Inflight in engines: 0
>> CNXK: NQ ptr: 0x62
>> CNXK: DQ ptr: 0x62
>> CNXK: Insts waiting in CPT: 0
>> CNXK:
>> CNXK: CPT AF registers:
>> CNXK:     CPT_AF_LF0_CTL: 0x0007000000040001
>> CNXK:     CPT_AF_LF0_CTL2: 0x0000000000000002
>> CNXK:     inst_req_pc: 0x0000000000000163
>> CNXK:     inst_lat_pc: 0x00000000002e1a2e
>> CNXK:     rd_req_pc: 0x0000000000001641
>> CNXK:     rd_lat_pc: 0x00000000001146b4
>> CNXK:     rd_uc_pc: 0x0000000000000fd0
>> CNXK:     active_cycles_pc: 0x0000000000f978e3
>> CNXK:     ctx_mis_pc: 0x0000000000000005
>> CNXK:     ctx_hit_pc: 0x000000000000015b
>> CNXK:     ctx_aop_pc: 0x0000000000000000
>> CNXK:     ctx_aop_lat_pc: 0x0000000000000000
>> CNXK:     ctx_ifetch_pc: 0x000000000000000a
>> CNXK:     ctx_ifetch_lat_pc: 0x00000000000009be
>> CNXK:     ctx_ffetch_pc: 0x0000000000000160
>> CNXK:     ctx_ffetch_lat_pc: 0x0000000000000992
>> CNXK:     ctx_wback_pc: 0x0000000000000160
>> CNXK:     ctx_wback_lat_pc: 0x0000000000000992
>> CNXK:     ctx_psh_pc: 0x0000000000000160
>> CNXK:     ctx_psh_lat_pc: 0x0000000000000992
>> CNXK:     ctx_err: 0x0000000000000000
>> CNXK:     ctx_enc_id: 0x0000000000000000
>> CNXK:     ctx_flush_timer: 0x000000000002faf0
>> CNXK:     rxc_time: 0x000000000003873a
>> CNXK:     rxc_time_cfg: 0x0000000000000000
>> CNXK:     rxc_active_sts: 0x0000000000000000
>> CNXK:     rxc_zombie_sts: 0x0000000000000000
>> CNXK:     rxc_dfrg: 0x0000000000000000
>> CNXK:     x2p_link_cfg0: 0x0000000000040000
>> CNXK:     x2p_link_cfg1: 0x0000000000040000
>> CNXK:     busy_sts_ae: 0x0000000000000000
>> CNXK:     free_sts_ae: 0x0000000000ffffff
>> CNXK:     busy_sts_se: 0x0000000000000000
>> CNXK:     free_sts_se: 0xffffffffffffffff
>> CNXK:     busy_sts_ie: 0x0000000000000000
>> CNXK:     free_sts_ie: 0x00ffffffffffffff
>> CNXK:     exe_err_info: 0x0000000000000000
>> CNXK:     cptclk_cnt: 0x000000169fa8634c
>> CNXK:     diag: 0x00000000010000ff
>> CNXK: cn10k_cpt_dequeue_burst():1325 Request timed out
>> CNXK: Lcore ID: 2, LF/QP ID: 0
>> CNXK:
>> CNXK: S/w pending queue:
>> CNXK: Head: 95
>> CNXK: Tail: 96
>> CNXK: Mask: 0x7ff
>> CNXK: Inflight count: 2047
>> CNXK:
>> CNXK: H/w pending queue:
>> CNXK: Inflight in engines: 0
>> CNXK: NQ ptr: 0x62
>> CNXK: DQ ptr: 0x62
>> CNXK: Insts waiting in CPT: 0
>> CNXK:
>> CNXK: CPT AF registers:
>> CNXK:     CPT_AF_LF0_CTL: 0x0007000000040001
>> CNXK:     CPT_AF_LF0_CTL2: 0x0000000000000002
>> CNXK:     inst_req_pc: 0x0000000000000163
>> CNXK:     inst_lat_pc: 0x00000000002e1a2e
>> CNXK:     rd_req_pc: 0x0000000000001641
>> CNXK:     rd_lat_pc: 0x00000000001146b4
>> CNXK:     rd_uc_pc: 0x0000000000000fd0
>> CNXK:     active_cycles_pc: 0x0000000000f9b66c
>> CNXK:     ctx_mis_pc: 0x0000000000000005
>> CNXK:     ctx_hit_pc: 0x000000000000015b
>> CNXK:     ctx_aop_pc: 0x0000000000000000
>> CNXK:     ctx_aop_lat_pc: 0x0000000000000000
>> CNXK:     ctx_ifetch_pc: 0x000000000000000a
>> CNXK:     ctx_ifetch_lat_pc: 0x00000000000009be
>> CNXK:     ctx_ffetch_pc: 0x0000000000000160
>> CNXK:     ctx_ffetch_lat_pc: 0x0000000000000992
>> CNXK:     ctx_wback_pc: 0x0000000000000160
>> CNXK:     ctx_wback_lat_pc: 0x0000000000000992
>> CNXK:     ctx_psh_pc: 0x0000000000000160
>> CNXK:     ctx_psh_lat_pc: 0x0000000000000992
>> CNXK:     ctx_err: 0x0000000000000000
>> CNXK:     ctx_enc_id: 0x0000000000000000
>> CNXK:     ctx_flush_timer: 0x000000000002faf0
>> CNXK:     rxc_time: 0x0000000000009ce5
>> CNXK:     rxc_time_cfg: 0x0000000000000000
>> CNXK:     rxc_active_sts: 0x0000000000000000
>> CNXK:     rxc_zombie_sts: 0x0000000000000000
>> CNXK:     rxc_dfrg: 0x0000000000000000
>> CNXK:     x2p_link_cfg0: 0x0000000000040000
>> CNXK:     x2p_link_cfg1: 0x0000000000040000
>> CNXK:     busy_sts_ae: 0x0000000000000000
>> CNXK:     free_sts_ae: 0x0000000000ffffff
>> CNXK:     busy_sts_se: 0x0000000000000000
>> CNXK:     free_sts_se: 0xffffffffffffffff
>> CNXK:     busy_sts_ie: 0x0000000000000000
>> CNXK:     free_sts_ie: 0x00ffffffffffffff
>> CNXK:     exe_err_info: 0x0000000000000000
>> CNXK:     cptclk_cnt: 0x00000016a6eb1638
>> CNXK:     diag: 0x00000000010000ff
>> CNXK: cn10k_cpt_dequeue_burst():1325 Request timed out
>> CNXK: Lcore ID: 2, LF/QP ID: 0
>>
>> This `cn10k_cpt_dequeue_burst():1325 Request timed out` error will
>> keep repeating until DTS times out.
>>
>> Could you please take a look and let me know if there's any
>> configuration or additional step that I might have missed to resolve
>> these issues?
>> I'd be happy to provide more details if needed.
>>
>> Thank you for your help!
>>
>> Best Regards,
>> Cody Cheng
>>
>> On Thu, Nov 14, 2024 at 2:23 PM Gnanesh Kambalu Palanethra
>> <gpalanethra@marvell.com> wrote:
>> >
>> >
>> > Hi Patrick,
>> >
>> > Please find my responses inline.  I am submitting  an updated patch based on your comments.
>> >
>> >
>> > Regards,
>> > Gnanesh
>> > ________________________________
>> > From: Patrick Robb <probb@iol.unh.edu>
>> > Sent: Friday, November 1, 2024 3:45 AM
>> > To: Gnanesh Kambalu Palanethra <gpalanethra@marvell.com>
>> > Cc: dts@dpdk.org <dts@dpdk.org>; JogaRao Nartu <njogarao@marvell.com>; Bharath Rajendra <brajendra@marvell.com>; Cody Cheng <ccheng@iol.unh.edu>; Hiral Shah <hshah@marvell.com>; Jerin Jacob <jerinj@marvell.com>
>> > Subject: [EXTERNAL] Re: [PATCH 1/1] Cn10K crypto Tests: Added Marvell Cn10K specific crypto Tests to TestCryptoPerfCryptodevPerf
>> >
>> > Prioritize security for external emails:
>> > Confirm sender and content safety before clicking links or opening attachments
>> > Report Suspicious
>> >
>> > Hi Gnanesh, thanks for sending this. My name is Patrick Robb, and I'm a CI Testing Lab manager at the DPDK Community Lab hosted at UNH, and a maintainer for the "new" DTS framework which exists within the DPDK repo and should be replacing the legacy DTS project in the long term(what this patch is for). We do have a Marvell CN10K board at the lab, and I would love to begin running some crypto testing from that if possible (so, thanks again for this series).
>> >
>> > So, most likely we will want to provide a review from UNH and test this, so that it can be merged to DTS (technically I was granted the authority to apply patches to the legacy repo).
>> >
>> > I am adding Cody to this thread as I think he can assist me on this. Cody I am going to provide some comments inline as an initial review for this series, and then we should apply this series locally on the traffic generator/test engine for the CN10K (Intel-1 in our lab), dry run the test which Gnanesh has added, and report back to him. If all goes well, it should be possible to merge.
>> >
>> > Long term, we will want to circle back and add this test to the new DTS framework, but it is not ready for this test yet, as currently new DTS is testpmd only - there is not cryptodev support. Although, adding such support to new DTS is something I would love to work with you on in the next year if you have time available for development towards DPDK testing! Anyways, comments inline:
>> >
>> > On Wed, Oct 30, 2024 at 12:44 PM Gnanesh <gpalanethra@marvell.com> wrote:
>> >
>> >     Changes included in this patch:
>> >     -- New TestCases are added for Cn10K CPT Hardware crypto accelerator.
>> >     -- when Testing newly added Testcases it is found Necessary to make Below Changes
>> >           1> tests/cryptodev_common.py
>> >                --> bind_qat_device function is updated to generate VFs for the given crypto_dev_id
>> >                --> added New function bind_mrvl_devices to accept list of PCI Ids and bind it to vfio_pci driver.
>> >           2> framework/crb.py
>> >                --> updated pci_devices_information_uncached_linux Method to handle additional Nic Speeds for Cavium,
>> >                    as it was restricting only to 1 GIG NIC speed
>> >
>> >
>> > Thanks, I also made this change on our local DTS project for cn10k, but the patch was not submitted to the "legacy DTS" mailing list here as it was unmaintained (and still is, to David's point). But, if we can do a review it is good to get these things into mainline. Maybe I can rebase our local patch on yours if there were any other changes, and get feedback for cn10k specific questions.
>> >
>> > I have a hard time understanding how this 10G NIC speed requirement got added in the first place as it was quite peculiar and confused me for a time as it was dropping the ports list! I'm sure you had a similar experience... :)
>> >
>> >
>> >           3> framework/settings.py
>> >                --> Changed the default Cavium NIC driver to rvu_nicpf.
>> >
>> >
>> > Thanks, we are also running from this change at the Community Lab. But, I thought you said above you are creating VFs from the bind_qat_device function. So, does rvu_nicvf need to be added as well?
>> >
>> >
>> >           4> nics/net_device.py
>> >                --> added Missing expect object
>> >           5> conf/crypto_perf_cryptodev_perf.cfg
>> >                --> added Marvell Cn10K configs for crypto_perf_cryptodev_perf
>> >           6> framework/ssh_pexpect.py
>> >                --> output.replace command's output is not set before returning from get_output_all method
>> >                    hence Unnecessarily returning Prompt along with the output
>> >           7> tests/TestSuite_crypto_perf_cryptodev_perf.py
>> >                -->  added Marvell CN10K Testcases
>> >
>> >
>> > Again we will apply these, give them a run, and report back.
>> >
>> >
>> >              Updated Below Private functions
>> >                 6.1> _run_crypto_perf() --> receives additional KW arguments
>> >                 6.2> _parse_output()  ---> Updated to handle Marvell Cn10K app command output
>> >                 6.3> _run_crypto_perf_throughput() --> receives additional KW arguments
>> >
>> >
>> >
>> > I don't see any modifications for the build process. Is there a need to include "-Dplatform=cn10k" when setting up the build from meson, or is the default build fine?
>> >
>> >  right now there is no change to build procedure, we are cross compiling DPDK on x86 and copying the application on to DUT.
>> >
>> >
>> >
>> >
>> >
>> >
>> > Signed-off-by: Gnanesh <gpalanethra@marvell.com>
>> > ---
>> >  conf/crypto_perf_cryptodev_perf.cfg           | 849 ++++++++++++++++++
>> >  framework/crb.py                              |   6 +-
>> >  framework/settings.py                         |   4 +-
>> >  framework/ssh_pexpect.py                      |   2 +-
>> >  nics/net_device.py                            |   3 +
>> >  tests/TestSuite_crypto_perf_cryptodev_perf.py | 322 ++++++-
>> >  tests/cryptodev_common.py                     |  11 +-
>> >  7 files changed, 1172 insertions(+), 25 deletions(-)
>> >
>> > diff --git a/framework/crb.py b/framework/crb.py
>> > index 9e3b0a58..2fe9d147 100644
>> > --- a/framework/crb.py
>> > +++ b/framework/crb.py
>> > @@ -374,15 +374,15 @@ class Crb(object):
>> >                  pass
>> >
>> >          for i in range(len(match)):
>> > -            # check if device is cavium and check its linkspeed, append only if it is 10G
>> > +            # for cavium supported link speed specfied under linkspeeds
>> >              if "177d:" in match[i][1]:
>> > -                linkspeed = "10000"
>> > +                linkspeeds = ["40000", "10000", "25000", "50000", "100000"]
>> >
>> >
>> > Honestly I don't understand why we have this condition at all. Other devices in DTS do not have this check where the portlinks can be dropped in crbs.py. What do you think about updating the accepted linkspeeds vs just removing this step entirely? Can we expect additional linkspeeds to be added for 177d in the future, and have this problem crop up again for a new linkspeed, or is this list of speeds "final" for 177d?
>> >
>> > This check is now completely Removed in the updated patch.
>> >
>> >
>> >
>> >                  nic_linkspeed = self.send_expect(
>> >                      "cat /sys/bus/pci/devices/%s/net/*/speed" % match[i][0],
>> >                      "# ",
>> >                      alt_session=True,
>> >                  )
>> > -                if nic_linkspeed.split()[0] == linkspeed:
>> > +                if nic_linkspeed.split()[0] in  linkspeeds:
>> >                      self.pci_devices_info.append((match[i][0], match[i][1]))
>> >              else:
>> >                  self.pci_devices_info.append((match[i][0], match[i][1]))
>> > diff --git a/framework/settings.py b/framework/settings.py
>> > index 1a561dda..b2c371f8 100644
>> > --- a/framework/settings.py
>> > +++ b/framework/settings.py
>> > @@ -166,8 +166,8 @@ DRIVERS = {
>> >      "cavium_a034": "thunder-nicvf",
>> >      "cavium_0011": "thunder-nicvf",
>> >      "IXGBE_10G-X550EM_X_SFP": "ixgbe",
>> > -    "cavium_a063": "octeontx2-nicpf",
>> > -    "cavium_a064": "octeontx2-nicvf",
>> > +    "cavium_a063": "rvu_nicpf",
>> > +    "cavium_a064": "rvu_nicvf",
>> >
>> >
>> > Okay disregard my comment from above about nicvf, I see you've added it. Please do update the commit message to reflect that this is included. And, while you're at it, you can update the commit message to accord with the style guidelines for submitting DPDK patches as seen here on sections 9.6 and 9.7 https://urldefense.proofpoint.com/v2/url?u=https-3A__doc.dpdk.org_guides_contributing_patches.html&d=DwIFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=Ulaoc1LiTkE5RG69cLkMa7P9xGTL0rP6LosuDS8KOe4&m=vVns1KNfhQSJf3ilhnQOFPpnXVYyIzN1csUdc6rFfMlREYfLTRuin-dnxy8CVeSj&s=IHrV3dmsSA_rqBs3IDZTGA2ce2WeX5SjR8lTna-RWYU&e=
>> >
>> > This change message is already part of the commit message.
>> > Regarding the style guideline, I have tried to incorporate in the updated patch, please help me in case further changes are needed
>> >
>> >
>> > :)
>> >
>> >      "ICE_100G-E810C_QSFP": "ice",
>> >      "ICE_25G-E810C_SFP": "ice",
>> >      "ICE_25G-E823C_QSFP": "ice",
>> > diff --git a/framework/ssh_pexpect.py b/framework/ssh_pexpect.py
>> > index 2132c066..a96087d2 100644
>> > --- a/framework/ssh_pexpect.py
>> > +++ b/framework/ssh_pexpect.py
>> > @@ -186,7 +186,7 @@ class SSHPexpect:
>> >
>> >      def get_output_all(self):
>> >          output = self.session.before
>> > -        output.replace("[PEXPECT]", "")
>> > +        output = output.replace("[PEXPECT]#", "")
>> >
>> >
>> > So, this is just removing the leading "#" from the output? I'm not sure we should be making styling changes for a "deprecated" repository. If I look at this example output, from one of the CI runs of this testsuite on a quickassist card:
>> >
>> > # Device 1 on lcore 40
>> >
>> > # total operations: 30000000
>> > # Buffer size: 1024
>> > # Burst size: 32
>> > #     Number of bursts: 937500
>> > #
>> > #                 Total   Average   Maximum   Minimum
>> > #  enqueued    30000000        32        32        32
>> > #  dequeued    30000000        32        32         1
>> > #    cycles 91892876610      3063    304625       485
>> > # time [us]  3675715064   122.520 12185.000    19.400
>> >
>> > I imagine the DTS maintainers were aware of this and thought it was the most clear way to print the logs. I don't think we need to override that decision now.
>> >
>> > This is not a styling change.
>> > output.replace will not replace the special prompt "[PEXPECT]" directly on the variable.
>> > output variable needs to be overwritten to have the output strip of the prompt [I am overwriting output ]
>> > special prompt variable is "[PEXPECT]#", not the "[PEXPECT]". hence output will have unwanted "#" string.
>> >
>> >
>> >
>> >          return output
>> >
>> >      def close(self, force=False):
>> > diff --git a/nics/net_device.py b/nics/net_device.py
>> > index 0f9c1af4..dd0d0e92 100644
>> > --- a/nics/net_device.py
>> > +++ b/nics/net_device.py
>> > @@ -43,6 +43,7 @@ class NetDevice(object):
>> >          if not isinstance(crb, Crb):
>> >              raise Exception("  Please input the instance of Crb!!!")
>> >          self.crb = crb
>> > +        self.__send_expect = self.crb.send_expect
>> >          self.domain_id = domain_id
>> >          self.bus_id = bus_id
>> >          self.devfun_id = devfun_id
>> > @@ -727,6 +728,8 @@ class NetDevice(object):
>> >              "/sys/bus/pci/devices/%s:%s:%s" % (domain_id, bus_id, devfun_id),
>> >              vf_reg_file,
>> >          )
>> > +        self.__send_expect("echo 0 > %s" %
>> > +                           (vf_reg_path), "# ")
>> >
>> >
>> > So, the pre-existing VFs need to be destroyed before a new set of VFs can be created? Do you know if this is Marvell specific, or a universal behavior?
>> >
>> > I am not aware of the other boards, but it is the clean way of creating the VFs.
>> >
>> >
>> >          self.__send_expect("echo %d > %s" % (int(vf_num), vf_reg_path), "# ")
>> >
>> >      def generate_sriov_vfs_linux_igb_uio(self, domain_id, bus_id, devfun_id, vf_num):
>> > diff --git a/tests/TestSuite_crypto_perf_cryptodev_perf.py b/tests/TestSuite_crypto_perf_cryptodev_perf.py
>> > index a3f48eee..b435cefa 100644
>> > --- a/tests/TestSuite_crypto_perf_cryptodev_perf.py
>> > +++ b/tests/TestSuite_crypto_perf_cryptodev_perf.py
>> > @@ -59,14 +59,74 @@ class TestCryptoPerfCryptodevPerf(TestCase):
>> >                  "# ",
>> >                  5,
>> >              )
>> > -
>> > -        cc.bind_qat_device(self, "vfio-pci")
>> > +        if self.nic == "cavium_a063":
>> > +            count = len(self.get_suite_cfg().get('l').split(",")) - 1
>> > +            cc.bind_qat_device(self, "vfio-pci", generate_vfs=True, vf_count=count)
>> > +        else:
>> > +            cc.bind_qat_device(self, "vfio-pci")
>> >
>> >
>> > If we are going to use this function for non-QAT devices, can we please update the function name to prevent confusion? It looks like it's a short list:
>> >
>> > Addressed this in  the updated patch
>> >
>> > probb@d121003:~/repos/dts> grep -r "bind_qat_d" .
>> > ./tests/TestSuite_compressdev_qat_pmd.py:        cc.bind_qat_device(self, self.drivername)
>> > ./tests/TestSuite_crypto_perf_cryptodev_perf.py:        cc.bind_qat_device(self, "vfio-pci")
>> > ./tests/TestSuite_fips_cryptodev.py:        cc.bind_qat_device(self, "vfio-pci")
>> > ./tests/TestSuite_ipsec_gw_cryptodev_func.py:        cc.bind_qat_device(self, self.drivername)
>> > ./tests/TestSuite_l2fwd_cryptodev_func.py:        cc.bind_qat_device(self, self.drivername)
>> > ./tests/TestSuite_virtio_ipsec_cryptodev_func.py:        cc.bind_qat_device(self, self.drivername)
>> > ./tests/TestSuite_virtio_perf_cryptodev_func.py:        cc.bind_qat_device(self, self.drivername)
>> > ./tests/TestSuite_virtio_unit_cryptodev_func.py:        cc.bind_qat_device(self, self.drivername)
>> > ./tests/compress_common.py:def bind_qat_device(test_case, driver="igb_uio"):
>> > ./tests/cryptodev_common.py:def bind_qat_device(test_case, driver="igb_uio"):
>> >
>> >
>> >
>> >
>> >          src_files = ["dep/test_aes_cbc.data", "dep/test_aes_gcm.data"]
>> >          self.dut_file_dir = "/tmp"
>> >          for file in src_files:
>> >              self.dut.session.copy_file_to(file, self.dut_file_dir)
>> >
>> >      def tear_down_all(self):
>> > +        if self.nic == "cavium_a063":
>> > +            cc.bind_qat_device(self, "vfio-pci", generate_vfs=True,
>> > +                               vf_count=2)
>> > +            import pandas as pd
>> >
>> >
>> > I don't have a problem with this, but I'm just wondering why this is imported here as opposed to the top of the file?
>> >
>> > wrt to pandas import, as we are generating this report only in Marvell case. this import should not affect others
>> >
>> >
>> > +            tuples = [('TestCase',), ('performance', 'failed_enq'),
>> > +                      ('performance', 'failed_deq'),
>> > +                      ('performance', 'throughput_mops'),
>> > +                      ('performance', 'cycle_buf'),
>> > +                      ('performance', 'throughput', 'value'),
>> > +                      ('performance', 'throughput', 'delta'),
>> > +                      ('parameters', 'core_num/thread_num'),
>> > +                      ('parameters', 'frame_size'),
>> > +                      ('parameters', 'burst_size'),
>> > +                      ('parameters', 'total_ops'), ('status',)]
>> > +            mindex = pd.MultiIndex.from_tuples(tuples)
>> > +            index = 0
>> > +            df = pd.DataFrame([], columns=mindex)
>> > +            for tname, data in self._perf_result.items():
>> > +                for dt in data:
>> > +                    for key, value in dt.items():
>> > +                        df.loc[index, 'TestCase'] = tname
>> > +                        if key == 'status':
>> > +                            df.loc[index, (key)] = value
>> > +                            continue
>> > +                        for item in value:
>> > +                            if item['name'] == 'throughput':
>> > +                                df.loc[index, (key, item['name'], 'value')] = item['value']
>> > +                                df.loc[index, (key, item['name'], 'delta')] = item['delta']
>> > +                            else:
>> > +                                df.loc[index, (key, item['name'])] = item['value']
>> > +                    index += 1
>> > +            perf_xl = self.logger.log_path + "/" + "perf_cryptodev_result.xls"
>> > +            writer = pd.ExcelWriter(perf_xl, engine='xlsxwriter')
>> > +            wb  = writer.book
>> > +            df.to_excel(writer, sheet_name='Sheet1')
>> > +            ws = writer.sheets["Sheet1"]
>> > +            merge_format = wb.add_format({"bold": 1,"border": 1,"align": "center",
>> > +                                          "valign": "vcenter"})
>> > +            ws.merge_range('B1:B3', "TestCase", merge_format)
>> > +            ws.merge_range('C2:C3', "failed_enq", merge_format)
>> > +            ws.merge_range('D2:D3', "failed_deq", merge_format)
>> > +            ws.merge_range('E2:E3', "throughput_mops", merge_format)
>> > +            ws.merge_range('F2:F3', "throughput_mops", merge_format)
>> > +            ws.merge_range('I2:I3', "core_num/thread_num", merge_format)
>> > +            ws.merge_range('J2:J3', "frame_size", merge_format)
>> > +            ws.merge_range('K2:K3', "burst_size", merge_format)
>> > +            ws.merge_range('L2:L3', "total_ops", merge_format)
>> > +            ws.merge_range('M1:M3', "status", merge_format)
>> > +
>> > +            format1 = wb.add_format({"bg_color": "#C6EFCE", "font_color": "#006100", "border": 1})
>> > +            format2 = wb.add_format({"bg_color": "#FFC7CE", "font_color": "#9C0006"})
>> > +            ws.conditional_format("B1:M3", {"type": "cell", "criteria": ">=", "value": 0 , 'format': format2})
>> > +            condition = 'B5:M%s'%(5+len(df)-1)
>> > +            ws.conditional_format(condition, {'type': 'no_blanks', 'format':format1})
>> > +            ws.autofit()
>> > +
>> > +            wb.close()
>> >          if self._perf_result:
>> >              with open(
>> >                  self.logger.log_path + "/" + "perf_cryptodev_result.json", "a"
>> > @@ -175,12 +235,197 @@ class TestCryptoPerfCryptodevPerf(TestCase):
>> >      def test_scheduler_zuc(self):
>> >          self._run_crypto_perf_throughput()
>> >
>> > +    # Marvell CN10K Testcases
>> > +    def _crypto_func_cn10k(self):
>> > +        """Functional tests Helper
>> > +        Returns:
>> > +           None
>> > +        """
>> > +        self._run_crypto_func()
>> > +
>> > +    def _crypto_perf_cn10k(self):
>> > +        """Perf tests Helper function
>> > +        Returns:
>> > +           None
>> > +        """
>> > +        self._run_crypto_perf_throughput(expected="# ",
>> > +                                         trim_whitespace=False)
>> > +    def test_perf_mrvl_aes_cbc(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_aes_gcm_encrypt(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_aes_cbc_cipher_then_auth(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_aes_sha2_256_cipher_then_auth(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_zuc_eea3_cipher_only(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_zuc_eia3_auth_only(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_aes_gmac_auth_only(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_null_cipher_only(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_snow3g_uea2_cipher_only(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_kasumi_f8_cipher_only(self):
>> > +        self._crypto_perf_cn10k() #F
>> > +
>> > +    def test_perf_mrvl_snow3g_uia2_auth_only(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_kasumi_f9_auth_only(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_sha1_hmac_auth_only(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_sha1_auth_only(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_aes_ctr_cipher_then_auth(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_aes_ctr_snow_3g_uia2_cipher_then_auth(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_aes_cmac_cipher_then_auth(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_3des_cbc_cipher_then_auth(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_sha1_hmac_cipher_then_auth(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_sha2_224_hmac_cipher_then_auth(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_sha2_256_hmac_cipher_then_auth(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_sha2_384_hmac_cipher_then_auth(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_aes_ctr_snow_3g_uia2_cipher_then_auth(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_aes_ctr_zuc_eia3_cipher_then_auth(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_aes_ctr_aes_cmac_cipher_then_auth(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_zuc_eea3_snow3g_uia2_cipher_then_auth(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_zuc_eea3_zuc_eia3_cipher_then_auth(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_zuc_eea3_aes_cmac_cipher_then_auth(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_snow3g_uea2_snow3g_uia2_cipher_then_auth(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_snow3g_uea2_zuc_eia3_cipher_then_auth(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_snow3g_uea2_aes_cmac_cipher_then_auth(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_md5_hmac_auth_only(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_aes_ctr_cipher_only(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_3des_cbc_cipher_only(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_sha2_224_hmac_auth_only(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_sha2_256_hmac_auth_only(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_sha2_384_hmac_auth_only(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_sha2_512_hmac_auth_only(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_aes_xts__cipher_only(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_des_cbc_cipher_only(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_md5_auth_only(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_sha512_auth_only(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_sha384_auth_only(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_sha256_auth_only(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_sha224_auth_only(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_snow3g_uea2_snow3g_uia2(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_sha1_hmac_armv8(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_perf_mrvl_sha2_hmac_armv8(self):
>> > +        self._crypto_perf_cn10k()
>> > +
>> > +    def test_mrvl_ptest(self):
>> > +        self._crypto_func_cn10k()
>> > +
>> > +    def test_mrvl_burst(self):
>> > +        self._crypto_func_cn10k()
>> > +
>> > +    def test_mrvl_devtype(self):
>> > +        self._crypto_func_cn10k()
>> > +
>> > +    def test_mrvl_crypto_sha1_hmac_armv8(self):
>> > +        self._crypto_func_cn10k()
>> > +
>> > +    def test_mrvl_chacha20_poly1305_aead(self):
>> > +        self._crypto_func_cn10k()
>> > +
>> > +    def test_mrvl_crypto_sha2_hmac_armv8(self):
>> > +        self._crypto_func_cn10k()
>> > +
>> > +    def test_mrvl_buffer(self):
>> > +        self._crypto_func_cn10k()
>> > +
>> >      # Private functions
>> >      def _run_crypto_func(self):
>> > +        """ Runs Crypto functional test.
>> > +        Returns:
>> > +            None
>> > +        """
>> > +
>> >          if cc.is_test_skip(self):
>> >              return
>> > -
>> > -        cores = ",".join(self.dut.get_core_list("1S/2C/1T"))
>> > +        core_list = self.get_suite_cfg().get('core_list',"1S/2C/1T")
>> > +        cores = ",".join(self.dut.get_core_list(core_list))
>> >          config = {"l": cores}
>> >          devices = self._get_crypto_device(1)
>> >          if not devices:
>> > @@ -206,12 +451,22 @@ class TestCryptoPerfCryptodevPerf(TestCase):
>> >
>> >          out = self.dut.send_command(
>> >              "cat %s/%s.txt" % (self.dut_file_dir, self.running_case), 30
>> > -        )
>> > -
>> > +        ).strip()
>> >          self.verify("Error" not in out, "Test function failed")
>> >          self.verify("failed" not in out, "Test function failed")
>> > -
>> > -    def _run_crypto_perf(self):
>> > +        assert (out != ""), "No output"
>> > +
>> > +    def _run_crypto_perf(self, **kwargs):
>> > +        """ Runs Crypto Performance test.
>> > +        Args:
>> > +          **kwargs:
>> > +              expected: send_expect match prompt
>> > +              trim_whitespace: trime whitespace from command output
>> > +        Returns:
>> > +            None
>> > +        """
>> > +        expected = kwargs.get("expected", "#")
>> > +        trim_whitespace = kwargs.get("trim_whitespace", True)
>> >          if cc.is_test_skip(self):
>> >              return "skip"
>> >
>> > @@ -222,13 +477,14 @@ class TestCryptoPerfCryptodevPerf(TestCase):
>> >              return "skip"
>> >
>> >          eal_opt_str = cc.get_eal_opt_str(self, devices)
>> > -        crypto_perf_opt_str = self._get_crypto_perf_opt_str()
>> > +        crypto_perf_opt_str = self._get_crypto_perf_opt_str(**kwargs)
>> >
>> >          cmd_str = cc.get_dpdk_app_cmd_str(
>> >              self._app_path, eal_opt_str, crypto_perf_opt_str
>> >          )
>> >          try:
>> > -            out = self.dut.send_expect(cmd_str, "#", 600)
>> > +            out = self.dut.send_expect(cmd_str, expected, 600,
>> > +                                       trim_whitespace=trim_whitespace)
>> >          except Exception as ex:
>> >              self.logger.error(ex)
>> >              raise ex
>> > @@ -237,25 +493,44 @@ class TestCryptoPerfCryptodevPerf(TestCase):
>> >
>> >          return results
>> >
>> > -    def _get_crypto_perf_opt_str(self, override_crypto_perf_opts={}):
>> > +    def _get_crypto_perf_opt_str(self, **kwargs):
>> > +        """get crypto perf app option string
>> > +        args:
>> > +            **kwargs:
>> > +                override_crypto_perf_opts- Suite/Case specfic perf
>> > +                app config to overwrite defaults
>> > +        Returns:
>> > +            crypto perf option string
>> > +        """
>> > +        override_crypto_perf_opts = kwargs.get("override_crypto_perf_opts", {})
>> >          return cc.get_opt_str(
>> >              self, self._default_crypto_perf_opts, override_crypto_perf_opts
>> >          )
>> >
>> >      def _parse_output(self, output):
>> >          try:
>> > +            dtype = self.get_case_cfg().get('devtype')
>> > +            match_str = r"    lcore id|#lcore id"
>> >              lines = output.split("\r\n")
>> >              line_nb = len(lines)
>> >              self.logger.debug("Total output lines: " + str(line_nb))
>> >
>> >              for line_index in range(line_nb):
>> > -                if lines[line_index].startswith("    lcore id"):
>> > +                if re.match(match_str, lines[line_index]):
>> >                      self.logger.debug("data output line from: " + str(line_index))
>> >                      break
>> > -            data_line = line_index - 1
>> > +            if dtype == 'crypto_cn10k':
>> > +                data_line = line_index + 2
>> > +            else:
>> > +                data_line = line_index - 1
>> > +
>> > +            if len(lines[data_line].split(","))>1:
>> > +                pattern = re.compile(r',')
>> > +            else:
>> > +                pattern = re.compile(r'\s+')
>> >
>> >              results = []
>> > -            pattern = re.compile(r"\s+")
>> > +
>> >              for line in lines[data_line:-1]:
>> >                  print(line)
>> >                  result = {}
>> > @@ -334,9 +609,15 @@ class TestCryptoPerfCryptodevPerf(TestCase):
>> >              cpu_info[key] = value.strip()
>> >          core, thread = 0, 0
>> >          lcores = self.get_case_cfg()["l"].split(",")
>> > +        if 'Core(s) per cluster' in out:
>> > +            cl_soc1 = 'Core(s) per cluster'
>> > +            cl_soc2 = 'Cluster(s)'
>> > +        else:
>> > +            cl_soc1 = 'Core(s) per socket'
>> > +            cl_soc2 = 'Socket(s)'
>> >          for lcore in lcores[1:]:
>> > -            if int(lcore.strip()) < int(cpu_info["Core(s) per socket"]) * int(
>> > -                cpu_info["Socket(s)"]
>> > +            if int(lcore.strip()) < int(cpu_info[cl_soc1]) * int(
>> > +                cpu_info[cl_soc2]
>> >              ):
>> >                  core += 1
>> >                  thread += 1
>> > @@ -362,6 +643,11 @@ class TestCryptoPerfCryptodevPerf(TestCase):
>> >              dev = "crypto_snow3g"
>> >          elif self.get_case_cfg()["devtype"] == "crypto_zuc":
>> >              dev = "crypto_zuc"
>> > +        elif self.get_case_cfg()["devtype"] == "crypto_cn10k":
>> > +            dev = "crypto_cn10k"
>> > +            vf = cc.get_qat_devices(self, cpm_num=1, num=num)
>> > +            device["a"] = ' -a '.join(vf)
>> > +            device["vdev"] = None
>> >          elif self.get_case_cfg()["devtype"] == "crypto_scheduler":
>> >              dev = "crypto_scheduler"
>> >              w = cc.get_qat_devices(self, cpm_num=3, num=num * 3)
>> > @@ -388,8 +674,8 @@ class TestCryptoPerfCryptodevPerf(TestCase):
>> >
>> >          return device
>> >
>> > -    def _run_crypto_perf_throughput(self):
>> > -        results = self._run_crypto_perf()
>> > +    def _run_crypto_perf_throughput(self, **kwargs):
>> > +        results = self._run_crypto_perf(**kwargs)
>> >          if results == "skip":
>> >              return
>> >          self.verify(results, "test results is none, Test Failed")
>> > diff --git a/tests/cryptodev_common.py b/tests/cryptodev_common.py
>> > index b550b468..91067b05 100644
>> > --- a/tests/cryptodev_common.py
>> > +++ b/tests/cryptodev_common.py
>> > @@ -8,7 +8,9 @@ from nics.net_device import GetNicObj
>> >  conf = SuiteConf("cryptodev_sample")
>> >
>> >
>> > -def bind_qat_device(test_case, driver="igb_uio"):
>> > +def bind_qat_device(test_case, driver="igb_uio", generate_vfs=False,
>> > +                    vf_count=2, drvr_type='generic'):
>> > +
>> >      if driver == "vfio-pci":
>> >          test_case.dut.send_expect("modprobe vfio", "#", 10)
>> >          test_case.dut.send_expect("modprobe vfio-pci", "#", 10)
>> > @@ -38,6 +40,8 @@ def bind_qat_device(test_case, driver="igb_uio"):
>> >          devfun_id = addr_array[2]
>> >          pf_port = GetNicObj(test_case.dut, domain_id, bus_id, devfun_id)
>> >
>> > +        if generate_vfs:
>> > +            pf_port.generate_sriov_vfs_linux(domain_id, bus_id, devfun_id, vf_count, drvr_type)
>> >
>> >
>> > Is this different from calling generate_sriov_vfs(), which I assume is the "correct" behavior. There appears to be a common practice in legacy DTS of templating out function names to call instead of using inheritance, so you will have a base function like bind_port(), and then that templates out calls to bind_port_linux(). bind_port_freebsd(), bind_port_windows() etc. It's a little strange but it is what it is and we should probably stick with the existing system if we are going to make additions. But, if there is a practical reason why you did it this way, then okay. :)
>> >
>> > I missed it initially, addressed it in updated patch.
>> >
>> >
>> >          sriov_vfs_pci = pf_port.get_sriov_vfs_pci()
>> >          if not sriov_vfs_pci:
>> >              raise Exception("can not get vf pci")
>> > @@ -51,6 +55,11 @@ def bind_qat_device(test_case, driver="igb_uio"):
>> >
>> >      test_case.dev = dev
>> >
>> > +def bind_mrvl_devices(test_case, driver='vfio-pci', **kwargs):
>> > +    pci_list = kwargs.get("pci_list").split()
>> > +    for pci in pci_list:
>> > +        test_case.dut.bind_eventdev_port(driver, pci)
>> > +    return pci_list
>> >
>> >  def get_qat_devices(test_case, cpm_num=None, num=1):
>> >      if not cpm_num:
>> > --
>> > 2.25.1
>> >
>>
>>
  

Patch

diff --git a/conf/crypto_perf_cryptodev_perf.cfg b/conf/crypto_perf_cryptodev_perf.cfg
index f0c86a45..c152ad69 100644
--- a/conf/crypto_perf_cryptodev_perf.cfg
+++ b/conf/crypto_perf_cryptodev_perf.cfg
@@ -365,3 +365,852 @@  auth-op="generate"
 auth-key-sz=16
 auth-iv-sz=16
 digest-sz=4
+
+# Marvell Cn10K configs
+[test_perf_mrvl_aes_cbc]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="cipher-only"
+cipher-algo="aes-cbc"
+cipher-op="encrypt"
+cipher-key-sz="16"
+cipher-iv-sz="16"
+csv-friendly="True"
+
+[test_perf_mrvl_aes_gcm_encrypt]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="aead"
+aead-algo="aes-gcm"
+aead-op="encrypt"
+aead-key-sz="16"
+aead-iv-sz="12"
+aead-aad-sz="16"
+digest-sz="16"
+csv-friendly="True"
+
+[test_perf_mrvl_aes_cbc_cipher_then_auth]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="cipher-then-auth"
+cipher-algo="aes-cbc"
+cipher-op="encrypt"
+cipher-key-sz="32"
+cipher-iv-sz="16"
+auth-algo="sha1-hmac"
+auth-op="generate"
+auth-key-sz="64"
+digest-sz="20"
+csv-friendly="True"
+
+[test_perf_mrvl_aes_sha2_256_cipher_then_auth]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="cipher-then-auth"
+cipher-algo="aes-cbc"
+cipher-op="encrypt"
+cipher-key-sz="32"
+cipher-iv-sz="16"
+auth-algo="sha2-256"
+auth-op="generate"
+auth-key-sz="0"
+digest-sz="32"
+csv-friendly="True"
+
+[test_perf_mrvl_zuc_eea3_cipher_only]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="cipher-only"
+cipher-algo="zuc-eea3"
+cipher-op="encrypt"
+cipher-key-sz="16"
+cipher-iv-sz="16"
+csv-friendly="True"
+
+[test_perf_mrvl_zuc_eia3_auth_only]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="auth-only"
+auth-algo="zuc-eia3"
+auth-op="generate"
+auth-key-sz="16"
+auth-iv-sz="16"
+digest-sz="4"
+csv-friendly="True"
+
+[test_perf_mrvl_aes_gmac_auth_only]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="auth-only"
+auth-algo="aes-gmac"
+auth-op="generate"
+auth-key-sz="32"
+auth-iv-sz="12"
+digest-sz="16"
+csv-friendly="True"
+
+[test_perf_mrvl_null_cipher_only]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="cipher-only"
+cipher-algo="null"
+cipher-op="encrypt"
+cipher-key-sz="0"
+cipher-iv-sz="0"
+csv-friendly="True"
+
+[test_perf_mrvl_snow3g_uea2_cipher_only]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="cipher-only"
+cipher-algo="snow3g-uea2"
+cipher-op="encrypt"
+cipher-key-sz="16"
+cipher-iv-sz="16"
+csv-friendly="True"
+
+[test_perf_mrvl_kasumi_f8_cipher_only]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="cipher-only"
+cipher-algo="kasumi-f8"
+cipher-op="encrypt"
+cipher-key-sz="16"
+cipher-iv-sz="8"
+csv-friendly="True"
+
+[test_perf_mrvl_snow3g_uia2_auth_only]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="auth-only"
+auth-algo="snow3g-uia2"
+auth-op="generate"
+auth-key-sz="16"
+auth-iv-sz="16"
+digest-sz="4"
+csv-friendly="True"
+
+[test_perf_mrvl_kasumi_f9_auth_only]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="auth-only"
+auth-algo="kasumi-f9"
+auth-op="generate"
+auth-key-sz="16"
+aead-aad-sz="0"
+digest-sz="4"
+csv-friendly="True"
+
+[test_perf_mrvl_sha1_hmac_auth_only]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="auth-only"
+auth-algo="sha1-hmac"
+auth-op="generate"
+auth-key-sz="64"
+aead-aad-sz="0"
+digest-sz="20"
+csv-friendly="True"
+
+[test_perf_mrvl_sha1_auth_only]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="auth-only"
+auth-algo="sha1"
+auth-op="generate"
+auth-key-sz="0"
+aead-aad-sz="0"
+digest-sz="20"
+csv-friendly="True"
+
+[test_perf_mrvl_aes_ctr_cipher_then_auth]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="cipher-then-auth"
+cipher-algo="aes-ctr"
+cipher-op="encrypt"
+cipher-key-sz="16"
+cipher-iv-sz="16"
+auth-algo="sha1-hmac"
+auth-op="generate"
+auth-key-sz="64"
+digest-sz="20"
+csv-friendly="True"
+
+[test_perf_mrvl_3des_cbc_cipher_then_auth]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="cipher-then-auth"
+cipher-algo="3des-cbc"
+cipher-op="encrypt"
+cipher-key-sz="24"
+cipher-iv-sz="16"
+auth-algo="sha1-hmac"
+auth-op="generate"
+auth-key-sz="64"
+digest-sz="20"
+csv-friendly="True"
+
+[test_perf_mrvl_sha1_hmac_cipher_then_auth]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="cipher-then-auth"
+cipher-algo="3des-cbc"
+cipher-op="encrypt"
+cipher-key-sz="24"
+cipher-iv-sz="16"
+auth-algo="sha1-hmac"
+auth-op="generate"
+auth-key-sz="64"
+digest-sz="20"
+csv-friendly="True"
+
+[test_perf_mrvl_sha2_224_hmac_cipher_then_auth]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="cipher-then-auth"
+cipher-algo="aes-ctr"
+cipher-op="encrypt"
+cipher-key-sz="16"
+cipher-iv-sz="16"
+auth-algo="sha2-224-hmac"
+auth-op="generate"
+auth-key-sz="64"
+digest-sz="28"
+csv-friendly="True"
+
+[test_perf_mrvl_sha2_256_hmac_cipher_then_auth]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="cipher-then-auth"
+cipher-algo="aes-ctr"
+cipher-op="encrypt"
+cipher-key-sz="16"
+cipher-iv-sz="16"
+auth-algo="sha2-256-hmac"
+auth-op="generate"
+auth-key-sz="64"
+digest-sz="32"
+csv-friendly="True"
+
+[test_perf_mrvl_sha2_384_hmac_cipher_then_auth]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="cipher-then-auth"
+cipher-algo="aes-ctr"
+cipher-op="encrypt"
+cipher-key-sz="16"
+cipher-iv-sz="16"
+auth-algo="sha2-384-hmac"
+auth-op="generate"
+auth-key-sz="64"
+digest-sz="48"
+csv-friendly="True"
+
+[test_perf_mrvl_aes_cmac_cipher_then_auth]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="1000000"
+burst-sz="32"
+buffer-sz="64"
+optype="auth-then-cipher"
+cipher-algo="snow3g-uea2"
+cipher-op="encrypt"
+cipher-key-sz="16"
+cipher-iv-sz="16"
+auth-algo="aes-cmac"
+auth-op="generate"
+auth-key-sz="16"
+aead-iv-sz="0"
+digest-sz="4"
+csv-friendly="True"
+
+[test_perf_mrvl_aes_ctr_snow_3g_uia2_cipher_then_auth]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="1000000"
+burst-sz="32"
+buffer-sz="64"
+optype="auth-then-cipher"
+cipher-algo="aes-ctr"
+cipher-op="encrypt"
+cipher-key-sz="16"
+cipher-iv-sz="16"
+auth-algo="snow3g-uia2"
+auth-op="generate"
+auth-key-sz="16"
+auth-iv-sz="16"
+digest-sz="4"
+csv-friendly="True"
+
+[test_perf_mrvl_aes_ctr_zuc_eia3_cipher_then_auth]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="1000000"
+burst-sz="32"
+buffer-sz="64"
+optype="auth-then-cipher"
+cipher-algo="aes-ctr"
+cipher-op="encrypt"
+cipher-key-sz="16"
+cipher-iv-sz="16"
+auth-algo="zuc-eia3"
+auth-op="generate"
+auth-key-sz="16"
+auth-iv-sz="16"
+digest-sz="4"
+csv-friendly="True"
+
+[test_perf_mrvl_aes_ctr_aes_cmac_cipher_then_auth]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="1000000"
+burst-sz="32"
+buffer-sz="64"
+optype="auth-then-cipher"
+cipher-algo="aes-ctr"
+cipher-op="encrypt"
+cipher-key-sz="16"
+cipher-iv-sz="16"
+auth-algo="aes-cmac"
+auth-op="generate"
+auth-key-sz="16"
+auth-iv-sz="0"
+digest-sz="4"
+csv-friendly="True"
+
+[test_perf_mrvl_zuc_eea3_snow3g_uia2_cipher_then_auth]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="1000000"
+burst-sz="32"
+buffer-sz="64"
+optype="auth-then-cipher"
+cipher-algo="zuc-eea3"
+cipher-op="encrypt"
+cipher-key-sz="16"
+cipher-iv-sz="16"
+auth-algo="snow3g-uia2"
+auth-op="generate"
+auth-key-sz="16"
+auth-iv-sz="16"
+digest-sz="4"
+csv-friendly="True"
+
+[test_perf_mrvl_zuc_eea3_zuc_eia3_cipher_then_auth]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="1000000"
+burst-sz="32"
+buffer-sz="64"
+optype="auth-then-cipher"
+cipher-algo="zuc-eea3"
+cipher-op="encrypt"
+cipher-key-sz="16"
+cipher-iv-sz="16"
+auth-algo="zuc-eia3"
+auth-op="generate"
+auth-key-sz="16"
+auth-iv-sz="16"
+digest-sz="4"
+csv-friendly="True"
+
+[test_perf_mrvl_zuc_eea3_aes_cmac_cipher_then_auth]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="1000000"
+burst-sz="32"
+buffer-sz="64"
+optype="auth-then-cipher"
+cipher-algo="zuc-eea3"
+cipher-op="encrypt"
+cipher-key-sz="16"
+cipher-iv-sz="16"
+auth-algo="aes-cmac"
+auth-op="generate"
+auth-key-sz="16"
+auth-iv-sz="0"
+digest-sz="4"
+csv-friendly="True"
+
+[test_perf_mrvl_snow3g_uea2_snow3g_uia2_cipher_then_auth]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="1000000"
+burst-sz="32"
+buffer-sz="64"
+optype="auth-then-cipher"
+cipher-algo="snow3g-uea2"
+cipher-op="encrypt"
+cipher-key-sz="16"
+cipher-iv-sz="16"
+auth-algo="snow3g-uia2"
+auth-op="generate"
+auth-key-sz="16"
+auth-iv-sz="16"
+digest-sz="4"
+csv-friendly="True"
+
+[test_perf_mrvl_snow3g_uea2_zuc_eia3_cipher_then_auth]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="1000000"
+burst-sz="32"
+buffer-sz="64"
+optype="auth-then-cipher"
+cipher-algo="snow3g-uea2"
+cipher-op="encrypt"
+cipher-key-sz="16"
+cipher-iv-sz="16"
+auth-algo="zuc-eia3"
+auth-op="generate"
+auth-key-sz="16"
+auth-iv-sz="16"
+digest-sz="4"
+csv-friendly="True"
+
+[test_perf_mrvl_snow3g_uea2_aes_cmac_cipher_then_auth]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="1000000"
+burst-sz="32"
+buffer-sz="64"
+optype="auth-then-cipher"
+cipher-algo="snow3g-uea2"
+cipher-op="encrypt"
+cipher-key-sz="16"
+cipher-iv-sz="16"
+auth-algo="aes-cmac"
+auth-op="generate"
+auth-key-sz="16"
+auth-iv-sz="0"
+digest-sz="4"
+csv-friendly="True"
+
+[test_perf_mrvl_md5_hmac_auth_only]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="auth-only"
+auth-algo="md5-hmac"
+auth-op="generate"
+auth-key-sz="64"
+aead-aad-sz="0"
+digest-sz="16"
+csv-friendly="True"
+
+[test_perf_mrvl_aes_ctr_cipher_only]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="cipher-only"
+cipher-algo="aes-ctr"
+cipher-op="encrypt"
+cipher-key-sz="16"
+cipher-iv-sz="16"
+csv-friendly="True"
+
+[test_perf_mrvl_3des_cbc_cipher_only]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="cipher-only"
+cipher-algo="3des-cbc"
+cipher-op="encrypt"
+cipher-key-sz="24"
+cipher-iv-sz="16"
+csv-friendly="True"
+
+[test_perf_mrvl_sha2_224_hmac_auth_only]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="auth-only"
+auth-algo="sha2-224-hmac"
+auth-op="generate"
+auth-key-sz="64"
+aead-aad-sz="0"
+digest-sz="28"
+csv-friendly="True"
+
+[test_perf_mrvl_sha2_256_hmac_auth_only]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="auth-only"
+auth-algo="sha2-256-hmac"
+auth-op="generate"
+auth-key-sz="64"
+aead-aad-sz="0"
+digest-sz="32"
+csv-friendly="True"
+
+[test_perf_mrvl_sha2_384_hmac_auth_only]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="auth-only"
+auth-algo="sha2-384-hmac"
+auth-op="generate"
+auth-key-sz="64"
+aead-aad-sz="0"
+digest-sz="48"
+csv-friendly="True"
+
+[test_perf_mrvl_sha2_512_hmac_auth_only]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="auth-only"
+auth-algo="sha2-512-hmac"
+auth-op="generate"
+auth-key-sz="64"
+aead-aad-sz="0"
+digest-sz="64"
+csv-friendly="True"
+
+[test_perf_mrvl_aes_xts__cipher_only]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="cipher-only"
+cipher-algo="aes-xts"
+cipher-op="encrypt"
+cipher-key-sz="64"
+cipher-iv-sz="16"
+csv-friendly="True"
+
+[test_perf_mrvl_des_cbc_cipher_only]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="cipher-only"
+cipher-algo="des-cbc"
+cipher-op="encrypt"
+cipher-key-sz="8"
+cipher-iv-sz="8"
+csv-friendly="True"
+
+[test_perf_mrvl_md5_auth_only]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="auth-only"
+auth-algo="md5"
+auth-op="generate"
+auth-key-sz="0"
+aead-aad-sz="0"
+digest-sz="16"
+csv-friendly="True"
+
+[test_perf_mrvl_sha512_auth_only]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="auth-only"
+auth-algo="sha2-512"
+auth-op="generate"
+auth-key-sz="0"
+aead-aad-sz="0"
+digest-sz="64"
+csv-friendly="True"
+
+[test_perf_mrvl_sha384_auth_only]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="auth-only"
+auth-algo="sha2-384"
+auth-op="generate"
+auth-key-sz="0"
+aead-aad-sz="0"
+digest-sz="48"
+csv-friendly="True"
+
+[test_perf_mrvl_sha256_auth_only]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="auth-only"
+auth-algo="sha2-256"
+auth-op="generate"
+auth-key-sz="0"
+aead-aad-sz="0"
+digest-sz="32"
+csv-friendly="True"
+
+[test_perf_mrvl_sha224_auth_only]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="auth-only"
+auth-algo="sha2-224"
+auth-op="generate"
+auth-key-sz="0"
+aead-aad-sz="0"
+digest-sz="28"
+csv-friendly="True"
+
+[test_perf_mrvl_snow3g_uea2_snow3g_uia2]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="auth-then-cipher"
+cipher-algo="snow3g-uea2"
+cipher-op="encrypt"
+cipher-key-sz="16"
+cipher-iv-sz="16"
+auth-algo="snow3g-uia2"
+auth-op="generate"
+auth-key-sz="16"
+auth-iv-sz="16"
+digest-sz="4"
+csv-friendly="True"
+
+[test_perf_mrvl_sha1_hmac_armv8]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="cipher-then-auth"
+cipher-algo="aes-cbc"
+cipher-op="encrypt"
+cipher-key-sz="16"
+cipher-iv-sz="16"
+auth-algo="sha1-hmac"
+auth-op="generate"
+auth-key-sz="64"
+digest-sz="20"
+csv-friendly="True"
+
+[test_perf_mrvl_sha2_hmac_armv8]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="cipher-then-auth"
+cipher-algo="aes-cbc"
+cipher-op="encrypt"
+cipher-key-sz="16"
+cipher-iv-sz="16"
+auth-algo="sha2-256-hmac"
+auth-op="generate"
+auth-key-sz="64"
+digest-sz="20"
+csv-friendly="True"
+
+[test_mrvl_buffer]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+buffer-sz="[64,128,256,512,1024,1280,2048,4096,8192,16384]"
+optype="cipher-then-auth"
+cipher-algo="aes-cbc"
+cipher-op="encrypt"
+cipher-key-sz="32"
+cipher-iv-sz="16"
+auth-algo="sha1-hmac"
+auth-op="generate"
+auth-key-sz="64"
+digest-sz="20"
+csv-friendly="True"
+
+[test_mrvl_chacha20_poly1305_aead]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="aead"
+aead-algo="chacha20-poly1305"
+aead-op="encrypt"
+aead-key-sz="32"
+aead-iv-sz="12"
+aead-aad-sz="32"
+digest-sz="16"
+csv-friendly="True"
+
+[test_mrvl_crypto_sha2_hmac_armv8]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="cipher-then-auth"
+cipher-algo="aes-cbc"
+cipher-op="encrypt"
+cipher-key-sz="16"
+cipher-iv-sz="16"
+auth-algo="sha2-256-hmac"
+auth-op="generate"
+auth-key-sz="64"
+digest-sz="20"
+csv-friendly="True"
+
+[test_mrvl_devtype]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="cipher-only"
+cipher-algo="aes-cbc"
+cipher-op="encrypt"
+cipher-key-sz="16"
+cipher-iv-sz="16"
+csv-friendly="True"
+
+[test_mrvl_crypto_sha1_hmac_armv8]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="cipher-then-auth"
+cipher-algo="aes-cbc"
+cipher-op="encrypt"
+cipher-key-sz="16"
+cipher-iv-sz="16"
+auth-algo="sha1-hmac"
+auth-op="generate"
+auth-key-sz="64"
+digest-sz="20"
+csv-friendly="True"
+
+[test_mrvl_ptest]
+silent="True"
+ptest=["throughput","latency","pmd-cyclecount"]
+pool-sz="16384"
+total-ops="10000000"
+burst-sz="32"
+optype="cipher-then-auth"
+cipher-algo="aes-cbc"
+cipher-op="encrypt"
+cipher-key-sz="32"
+cipher-iv-sz="16"
+auth-algo="sha1-hmac"
+auth-op="generate"
+auth-key-sz="64"
+digest-sz="20"
+csv-friendly="True"
+
+[test_mrvl_burst]
+silent="True"
+ptest="throughput"
+pool-sz="16384"
+total-ops="10000000"
+burst-sz=["16", "24", "32"]
+optype="cipher-then-auth"
+cipher-algo="aes-cbc"
+cipher-op="encrypt"
+cipher-key-sz="32"
+cipher-iv-sz="16"
+auth-algo="sha1-hmac"
+auth-op="generate"
+auth-key-sz="64"
+digest-sz="20"
+csv-friendly="True"
diff --git a/framework/crb.py b/framework/crb.py
index 9e3b0a58..2fe9d147 100644
--- a/framework/crb.py
+++ b/framework/crb.py
@@ -374,15 +374,15 @@  class Crb(object):
                 pass
 
         for i in range(len(match)):
-            # check if device is cavium and check its linkspeed, append only if it is 10G
+            # for cavium supported link speed specfied under linkspeeds
             if "177d:" in match[i][1]:
-                linkspeed = "10000"
+                linkspeeds = ["40000", "10000", "25000", "50000", "100000"]
                 nic_linkspeed = self.send_expect(
                     "cat /sys/bus/pci/devices/%s/net/*/speed" % match[i][0],
                     "# ",
                     alt_session=True,
                 )
-                if nic_linkspeed.split()[0] == linkspeed:
+                if nic_linkspeed.split()[0] in  linkspeeds:
                     self.pci_devices_info.append((match[i][0], match[i][1]))
             else:
                 self.pci_devices_info.append((match[i][0], match[i][1]))
diff --git a/framework/settings.py b/framework/settings.py
index 1a561dda..b2c371f8 100644
--- a/framework/settings.py
+++ b/framework/settings.py
@@ -166,8 +166,8 @@  DRIVERS = {
     "cavium_a034": "thunder-nicvf",
     "cavium_0011": "thunder-nicvf",
     "IXGBE_10G-X550EM_X_SFP": "ixgbe",
-    "cavium_a063": "octeontx2-nicpf",
-    "cavium_a064": "octeontx2-nicvf",
+    "cavium_a063": "rvu_nicpf",
+    "cavium_a064": "rvu_nicvf",
     "ICE_100G-E810C_QSFP": "ice",
     "ICE_25G-E810C_SFP": "ice",
     "ICE_25G-E823C_QSFP": "ice",
diff --git a/framework/ssh_pexpect.py b/framework/ssh_pexpect.py
index 2132c066..a96087d2 100644
--- a/framework/ssh_pexpect.py
+++ b/framework/ssh_pexpect.py
@@ -186,7 +186,7 @@  class SSHPexpect:
 
     def get_output_all(self):
         output = self.session.before
-        output.replace("[PEXPECT]", "")
+        output = output.replace("[PEXPECT]#", "")
         return output
 
     def close(self, force=False):
diff --git a/nics/net_device.py b/nics/net_device.py
index 0f9c1af4..dd0d0e92 100644
--- a/nics/net_device.py
+++ b/nics/net_device.py
@@ -43,6 +43,7 @@  class NetDevice(object):
         if not isinstance(crb, Crb):
             raise Exception("  Please input the instance of Crb!!!")
         self.crb = crb
+        self.__send_expect = self.crb.send_expect
         self.domain_id = domain_id
         self.bus_id = bus_id
         self.devfun_id = devfun_id
@@ -727,6 +728,8 @@  class NetDevice(object):
             "/sys/bus/pci/devices/%s:%s:%s" % (domain_id, bus_id, devfun_id),
             vf_reg_file,
         )
+        self.__send_expect("echo 0 > %s" %
+                           (vf_reg_path), "# ")
         self.__send_expect("echo %d > %s" % (int(vf_num), vf_reg_path), "# ")
 
     def generate_sriov_vfs_linux_igb_uio(self, domain_id, bus_id, devfun_id, vf_num):
diff --git a/tests/TestSuite_crypto_perf_cryptodev_perf.py b/tests/TestSuite_crypto_perf_cryptodev_perf.py
index a3f48eee..b435cefa 100644
--- a/tests/TestSuite_crypto_perf_cryptodev_perf.py
+++ b/tests/TestSuite_crypto_perf_cryptodev_perf.py
@@ -59,14 +59,74 @@  class TestCryptoPerfCryptodevPerf(TestCase):
                 "# ",
                 5,
             )
-
-        cc.bind_qat_device(self, "vfio-pci")
+        if self.nic == "cavium_a063":
+            count = len(self.get_suite_cfg().get('l').split(",")) - 1
+            cc.bind_qat_device(self, "vfio-pci", generate_vfs=True, vf_count=count)
+        else:
+            cc.bind_qat_device(self, "vfio-pci")
         src_files = ["dep/test_aes_cbc.data", "dep/test_aes_gcm.data"]
         self.dut_file_dir = "/tmp"
         for file in src_files:
             self.dut.session.copy_file_to(file, self.dut_file_dir)
 
     def tear_down_all(self):
+        if self.nic == "cavium_a063":
+            cc.bind_qat_device(self, "vfio-pci", generate_vfs=True,
+                               vf_count=2)
+            import pandas as pd
+            tuples = [('TestCase',), ('performance', 'failed_enq'),
+                      ('performance', 'failed_deq'),
+                      ('performance', 'throughput_mops'),
+                      ('performance', 'cycle_buf'),
+                      ('performance', 'throughput', 'value'),
+                      ('performance', 'throughput', 'delta'),
+                      ('parameters', 'core_num/thread_num'),
+                      ('parameters', 'frame_size'),
+                      ('parameters', 'burst_size'),
+                      ('parameters', 'total_ops'), ('status',)]
+            mindex = pd.MultiIndex.from_tuples(tuples)
+            index = 0
+            df = pd.DataFrame([], columns=mindex)
+            for tname, data in self._perf_result.items():
+                for dt in data:
+                    for key, value in dt.items():
+                        df.loc[index, 'TestCase'] = tname
+                        if key == 'status':
+                            df.loc[index, (key)] = value
+                            continue
+                        for item in value:
+                            if item['name'] == 'throughput':
+                                df.loc[index, (key, item['name'], 'value')] = item['value']
+                                df.loc[index, (key, item['name'], 'delta')] = item['delta']
+                            else:
+                                df.loc[index, (key, item['name'])] = item['value']
+                    index += 1
+            perf_xl = self.logger.log_path + "/" + "perf_cryptodev_result.xls"
+            writer = pd.ExcelWriter(perf_xl, engine='xlsxwriter')
+            wb  = writer.book
+            df.to_excel(writer, sheet_name='Sheet1')
+            ws = writer.sheets["Sheet1"]
+            merge_format = wb.add_format({"bold": 1,"border": 1,"align": "center",
+                                          "valign": "vcenter"})
+            ws.merge_range('B1:B3', "TestCase", merge_format)
+            ws.merge_range('C2:C3', "failed_enq", merge_format)
+            ws.merge_range('D2:D3', "failed_deq", merge_format)
+            ws.merge_range('E2:E3', "throughput_mops", merge_format)
+            ws.merge_range('F2:F3', "throughput_mops", merge_format)
+            ws.merge_range('I2:I3', "core_num/thread_num", merge_format)
+            ws.merge_range('J2:J3', "frame_size", merge_format)
+            ws.merge_range('K2:K3', "burst_size", merge_format)
+            ws.merge_range('L2:L3', "total_ops", merge_format)
+            ws.merge_range('M1:M3', "status", merge_format)
+
+            format1 = wb.add_format({"bg_color": "#C6EFCE", "font_color": "#006100", "border": 1})
+            format2 = wb.add_format({"bg_color": "#FFC7CE", "font_color": "#9C0006"})
+            ws.conditional_format("B1:M3", {"type": "cell", "criteria": ">=", "value": 0 , 'format': format2})
+            condition = 'B5:M%s'%(5+len(df)-1)
+            ws.conditional_format(condition, {'type': 'no_blanks', 'format':format1})
+            ws.autofit()
+
+            wb.close()
         if self._perf_result:
             with open(
                 self.logger.log_path + "/" + "perf_cryptodev_result.json", "a"
@@ -175,12 +235,197 @@  class TestCryptoPerfCryptodevPerf(TestCase):
     def test_scheduler_zuc(self):
         self._run_crypto_perf_throughput()
 
+    # Marvell CN10K Testcases
+    def _crypto_func_cn10k(self):
+        """Functional tests Helper
+        Returns:
+           None
+        """
+        self._run_crypto_func()
+
+    def _crypto_perf_cn10k(self):
+        """Perf tests Helper function
+        Returns:
+           None
+        """
+        self._run_crypto_perf_throughput(expected="# ",
+                                         trim_whitespace=False)
+    def test_perf_mrvl_aes_cbc(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_aes_gcm_encrypt(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_aes_cbc_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_aes_sha2_256_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_zuc_eea3_cipher_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_zuc_eia3_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_aes_gmac_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_null_cipher_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_snow3g_uea2_cipher_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_kasumi_f8_cipher_only(self):
+        self._crypto_perf_cn10k() #F
+
+    def test_perf_mrvl_snow3g_uia2_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_kasumi_f9_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha1_hmac_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha1_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_aes_ctr_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_aes_ctr_snow_3g_uia2_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_aes_cmac_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_3des_cbc_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha1_hmac_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha2_224_hmac_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha2_256_hmac_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha2_384_hmac_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_aes_ctr_snow_3g_uia2_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_aes_ctr_zuc_eia3_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_aes_ctr_aes_cmac_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_zuc_eea3_snow3g_uia2_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_zuc_eea3_zuc_eia3_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_zuc_eea3_aes_cmac_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_snow3g_uea2_snow3g_uia2_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_snow3g_uea2_zuc_eia3_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_snow3g_uea2_aes_cmac_cipher_then_auth(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_md5_hmac_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_aes_ctr_cipher_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_3des_cbc_cipher_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha2_224_hmac_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha2_256_hmac_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha2_384_hmac_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha2_512_hmac_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_aes_xts__cipher_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_des_cbc_cipher_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_md5_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha512_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha384_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha256_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha224_auth_only(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_snow3g_uea2_snow3g_uia2(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha1_hmac_armv8(self):
+        self._crypto_perf_cn10k()
+
+    def test_perf_mrvl_sha2_hmac_armv8(self):
+        self._crypto_perf_cn10k()
+
+    def test_mrvl_ptest(self):
+        self._crypto_func_cn10k()
+
+    def test_mrvl_burst(self):
+        self._crypto_func_cn10k()
+
+    def test_mrvl_devtype(self):
+        self._crypto_func_cn10k()
+
+    def test_mrvl_crypto_sha1_hmac_armv8(self):
+        self._crypto_func_cn10k()
+
+    def test_mrvl_chacha20_poly1305_aead(self):
+        self._crypto_func_cn10k()
+
+    def test_mrvl_crypto_sha2_hmac_armv8(self):
+        self._crypto_func_cn10k()
+
+    def test_mrvl_buffer(self):
+        self._crypto_func_cn10k()
+
     # Private functions
     def _run_crypto_func(self):
+        """ Runs Crypto functional test.
+        Returns:
+            None
+        """
+
         if cc.is_test_skip(self):
             return
-
-        cores = ",".join(self.dut.get_core_list("1S/2C/1T"))
+        core_list = self.get_suite_cfg().get('core_list',"1S/2C/1T")
+        cores = ",".join(self.dut.get_core_list(core_list))
         config = {"l": cores}
         devices = self._get_crypto_device(1)
         if not devices:
@@ -206,12 +451,22 @@  class TestCryptoPerfCryptodevPerf(TestCase):
 
         out = self.dut.send_command(
             "cat %s/%s.txt" % (self.dut_file_dir, self.running_case), 30
-        )
-
+        ).strip()
         self.verify("Error" not in out, "Test function failed")
         self.verify("failed" not in out, "Test function failed")
-
-    def _run_crypto_perf(self):
+        assert (out != ""), "No output"
+
+    def _run_crypto_perf(self, **kwargs):
+        """ Runs Crypto Performance test.
+        Args:
+          **kwargs:
+              expected: send_expect match prompt
+              trim_whitespace: trime whitespace from command output
+        Returns:
+            None
+        """
+        expected = kwargs.get("expected", "#")
+        trim_whitespace = kwargs.get("trim_whitespace", True)
         if cc.is_test_skip(self):
             return "skip"
 
@@ -222,13 +477,14 @@  class TestCryptoPerfCryptodevPerf(TestCase):
             return "skip"
 
         eal_opt_str = cc.get_eal_opt_str(self, devices)
-        crypto_perf_opt_str = self._get_crypto_perf_opt_str()
+        crypto_perf_opt_str = self._get_crypto_perf_opt_str(**kwargs)
 
         cmd_str = cc.get_dpdk_app_cmd_str(
             self._app_path, eal_opt_str, crypto_perf_opt_str
         )
         try:
-            out = self.dut.send_expect(cmd_str, "#", 600)
+            out = self.dut.send_expect(cmd_str, expected, 600,
+                                       trim_whitespace=trim_whitespace)
         except Exception as ex:
             self.logger.error(ex)
             raise ex
@@ -237,25 +493,44 @@  class TestCryptoPerfCryptodevPerf(TestCase):
 
         return results
 
-    def _get_crypto_perf_opt_str(self, override_crypto_perf_opts={}):
+    def _get_crypto_perf_opt_str(self, **kwargs):
+        """get crypto perf app option string
+        args:
+            **kwargs:
+                override_crypto_perf_opts- Suite/Case specfic perf
+                app config to overwrite defaults
+        Returns:
+            crypto perf option string
+        """
+        override_crypto_perf_opts = kwargs.get("override_crypto_perf_opts", {})
         return cc.get_opt_str(
             self, self._default_crypto_perf_opts, override_crypto_perf_opts
         )
 
     def _parse_output(self, output):
         try:
+            dtype = self.get_case_cfg().get('devtype')
+            match_str = r"    lcore id|#lcore id"
             lines = output.split("\r\n")
             line_nb = len(lines)
             self.logger.debug("Total output lines: " + str(line_nb))
 
             for line_index in range(line_nb):
-                if lines[line_index].startswith("    lcore id"):
+                if re.match(match_str, lines[line_index]):
                     self.logger.debug("data output line from: " + str(line_index))
                     break
-            data_line = line_index - 1
+            if dtype == 'crypto_cn10k':
+                data_line = line_index + 2
+            else:
+                data_line = line_index - 1
+
+            if len(lines[data_line].split(","))>1:
+                pattern = re.compile(r',')
+            else:
+                pattern = re.compile(r'\s+')
 
             results = []
-            pattern = re.compile(r"\s+")
+
             for line in lines[data_line:-1]:
                 print(line)
                 result = {}
@@ -334,9 +609,15 @@  class TestCryptoPerfCryptodevPerf(TestCase):
             cpu_info[key] = value.strip()
         core, thread = 0, 0
         lcores = self.get_case_cfg()["l"].split(",")
+        if 'Core(s) per cluster' in out:
+            cl_soc1 = 'Core(s) per cluster'
+            cl_soc2 = 'Cluster(s)'
+        else:
+            cl_soc1 = 'Core(s) per socket'
+            cl_soc2 = 'Socket(s)'
         for lcore in lcores[1:]:
-            if int(lcore.strip()) < int(cpu_info["Core(s) per socket"]) * int(
-                cpu_info["Socket(s)"]
+            if int(lcore.strip()) < int(cpu_info[cl_soc1]) * int(
+                cpu_info[cl_soc2]
             ):
                 core += 1
                 thread += 1
@@ -362,6 +643,11 @@  class TestCryptoPerfCryptodevPerf(TestCase):
             dev = "crypto_snow3g"
         elif self.get_case_cfg()["devtype"] == "crypto_zuc":
             dev = "crypto_zuc"
+        elif self.get_case_cfg()["devtype"] == "crypto_cn10k":
+            dev = "crypto_cn10k"
+            vf = cc.get_qat_devices(self, cpm_num=1, num=num)
+            device["a"] = ' -a '.join(vf)
+            device["vdev"] = None
         elif self.get_case_cfg()["devtype"] == "crypto_scheduler":
             dev = "crypto_scheduler"
             w = cc.get_qat_devices(self, cpm_num=3, num=num * 3)
@@ -388,8 +674,8 @@  class TestCryptoPerfCryptodevPerf(TestCase):
 
         return device
 
-    def _run_crypto_perf_throughput(self):
-        results = self._run_crypto_perf()
+    def _run_crypto_perf_throughput(self, **kwargs):
+        results = self._run_crypto_perf(**kwargs)
         if results == "skip":
             return
         self.verify(results, "test results is none, Test Failed")
diff --git a/tests/cryptodev_common.py b/tests/cryptodev_common.py
index b550b468..91067b05 100644
--- a/tests/cryptodev_common.py
+++ b/tests/cryptodev_common.py
@@ -8,7 +8,9 @@  from nics.net_device import GetNicObj
 conf = SuiteConf("cryptodev_sample")
 
 
-def bind_qat_device(test_case, driver="igb_uio"):
+def bind_qat_device(test_case, driver="igb_uio", generate_vfs=False,
+                    vf_count=2, drvr_type='generic'):
+
     if driver == "vfio-pci":
         test_case.dut.send_expect("modprobe vfio", "#", 10)
         test_case.dut.send_expect("modprobe vfio-pci", "#", 10)
@@ -38,6 +40,8 @@  def bind_qat_device(test_case, driver="igb_uio"):
         devfun_id = addr_array[2]
         pf_port = GetNicObj(test_case.dut, domain_id, bus_id, devfun_id)
 
+        if generate_vfs:
+            pf_port.generate_sriov_vfs_linux(domain_id, bus_id, devfun_id, vf_count, drvr_type)
         sriov_vfs_pci = pf_port.get_sriov_vfs_pci()
         if not sriov_vfs_pci:
             raise Exception("can not get vf pci")
@@ -51,6 +55,11 @@  def bind_qat_device(test_case, driver="igb_uio"):
 
     test_case.dev = dev
 
+def bind_mrvl_devices(test_case, driver='vfio-pci', **kwargs):
+    pci_list = kwargs.get("pci_list").split()
+    for pci in pci_list:
+        test_case.dut.bind_eventdev_port(driver, pci)
+    return pci_list
 
 def get_qat_devices(test_case, cpm_num=None, num=1):
     if not cpm_num: