From patchwork Wed Dec 20 13:26:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nayak, Nishikanta" X-Patchwork-Id: 135394 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 012614374A; Wed, 20 Dec 2023 14:26:50 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E1F5740685; Wed, 20 Dec 2023 14:26:49 +0100 (CET) Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.126]) by mails.dpdk.org (Postfix) with ESMTP id 0FA8B4021F for ; Wed, 20 Dec 2023 14:26:47 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703078808; x=1734614808; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=QzLTLRsexr7Dnm+VZ2J+azEQlNbGhVXo/9zf0/VCz2s=; b=dVh/hcZpy0fl4owGTIJDxm3Ww3/HEwdrDTlYhoKAg3NNd+LdIXV1gmbm 5iTcGthaVbDkoH+eBD3hXKRjPVRTHV6tFysZUfYooFHgsrD2SWRmCLlA1 2TVyQIFbHdSsrhqUkyZhdMHct8GhLrsfr/loofELlXKxi6ZkWuLtiaI1o xdQKlGjFHyVZ3SO1pfzmI24LBGLHZCjE7XoYTEuM+GCpDTTEAs6njiHEB 8upryNNIygcuptOyLn5N07BJyPQH4wyWQ1YHSkKIqHFfrNJ0ui2ywybFf cKXdTRQTFAtkJS8PYPYbMxX7kDEz2Kj+NGwliGG598Gep54oS60sIPnRs A==; X-IronPort-AV: E=McAfee;i="6600,9927,10929"; a="380801520" X-IronPort-AV: E=Sophos;i="6.04,291,1695711600"; d="scan'208";a="380801520" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Dec 2023 05:26:46 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10929"; a="842276242" X-IronPort-AV: E=Sophos;i="6.04,291,1695711600"; d="scan'208";a="842276242" Received: from silpixa00400355.ir.intel.com (HELO silpixa00400355.ger.corp.intel.com) ([10.237.222.80]) by fmsmga008.fm.intel.com with ESMTP; 20 Dec 2023 05:26:43 -0800 From: Nishikant Nayak To: dev@dpdk.org Cc: kai.ji@intel.com, ciara.power@intel.com, arkadiuszx.kusztal@intel.com, Nishikant Nayak , Thomas Monjalon , Anatoly Burakov Subject: [PATCH 1/4] common/qat: add files specific to GEN5 Date: Wed, 20 Dec 2023 13:26:13 +0000 Message-Id: <20231220132616.318983-1-nishikanta.nayak@intel.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Adding GEN5 files for handling GEN5 specific operaions. These files are inherited from the existing files/APIs which has some changes specific GEN5 requirements Also updated the mailmap file. Signed-off-by: Nishikant Nayak --- .mailmap | 1 + drivers/common/qat/dev/qat_dev_gen5.c | 336 ++++++++++++++++++ .../adf_transport_access_macros_gen5.h | 51 +++ .../adf_transport_access_macros_gen5vf.h | 48 +++ drivers/crypto/qat/dev/qat_crypto_pmd_gen5.c | 336 ++++++++++++++++++ 5 files changed, 772 insertions(+) create mode 100644 drivers/common/qat/dev/qat_dev_gen5.c create mode 100644 drivers/common/qat/qat_adf/adf_transport_access_macros_gen5.h create mode 100644 drivers/common/qat/qat_adf/adf_transport_access_macros_gen5vf.h create mode 100644 drivers/crypto/qat/dev/qat_crypto_pmd_gen5.c diff --git a/.mailmap b/.mailmap index ab0742a382..ef8e0b79e5 100644 --- a/.mailmap +++ b/.mailmap @@ -1027,6 +1027,7 @@ Ning Li Nipun Gupta Nir Efrati Nirmoy Das +Nishikant Nayak Nithin Dabilpuram Nitin Saxena Nitzan Weller diff --git a/drivers/common/qat/dev/qat_dev_gen5.c b/drivers/common/qat/dev/qat_dev_gen5.c new file mode 100644 index 0000000000..dc2bcd5650 --- /dev/null +++ b/drivers/common/qat/dev/qat_dev_gen5.c @@ -0,0 +1,336 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2023 Intel Corporation + */ + +#include +#include + +#include "qat_device.h" +#include "qat_qp.h" +#include "adf_transport_access_macros_gen5vf.h" +#include "adf_pf2vf_msg.h" +#include "qat_pf2vf.h" + +#include +#include +#include +#include + +#define BITS_PER_LONG (sizeof(unsigned long) * 8) +#define BITS_PER_ULONG (sizeof(unsigned long) * 8) + +#define VFIO_PCI_LCE_DEVICE_CFG_REGION_INDEX VFIO_PCI_NUM_REGIONS +#define VFIO_PCI_LCE_CY_CFG_REGION_INDEX (VFIO_PCI_NUM_REGIONS + 2) +#define VFIO_PCI_LCE_RING_CFG_REGION_INDEX (VFIO_PCI_NUM_REGIONS + 4) + +#define min_t(type, x, y) ({ \ + type __min1 = (x); \ + type __min2 = (y); \ + __min1 < __min2 ? __min1 : __min2; }) + +/** + * struct lce_vfio_dev_cap - LCE device capabilities + * + * Device level capabilities and service level capabilities + */ +struct lce_vfio_dev_cap { + uint16_t device_num; + uint16_t device_type; + +#define LCE_DEVICE_CAP_DYNAMIC_BANK BIT(31) + uint32_t capability_mask; + uint32_t extended_capabilities; + uint16_t max_banks; + uint16_t max_rings_per_bank; + uint16_t arb_mask; + +#define SERV_TYPE_DC BIT(0) +#define SERV_TYPE_SYM BIT(1) +#define SERV_TYPE_ASYM BIT(2) +#define SERV_TYPE_DMA BIT(3) + uint16_t services; + uint16_t pkg_id; + uint16_t node_id; + +#define LCE_DEVICE_NAME_SIZE 64 + __u8 device_name[LCE_DEVICE_NAME_SIZE]; +}; + +#define LCE_DEVICE_MAX_BANKS 2080 +#define LCE_DEVICE_BITMAP_SIZE \ + __KERNEL_DIV_ROUND_UP(LCE_DEVICE_MAX_BANKS, BITS_PER_LONG) + +/* struct lce_vfio_dev_cy_cap - CY capabilities of LCE device */ +struct lce_vfio_dev_cy_cap { + uint32_t nr_banks; + unsigned long bitmap[LCE_DEVICE_BITMAP_SIZE]; +}; + +#define LCE_QAT_NID_LOCAL 0x7 +#define LCE_QAT_FUNC_LOCAL 0x3ff +#define LCE_QAT_RID_LOCAL 0xf +#define LCE_QAT_PASID_LOCAL 0xfffff + +struct lce_qat_domain { + uint32_t nid :3; + uint32_t fid :7; + uint32_t ftype :2; + uint32_t vfid :13; + uint32_t rid :4; + uint32_t vld :1; + uint32_t desc_over :1; + uint32_t pasid_vld :1; + uint32_t pasid :20; +}; + +struct lce_qat_buf_domain { + uint32_t bank_id: 20; +#define LCE_REQ_BUFFER_DOMAIN 1UL +#define LCE_RES_BUFFER_DOMAIN 2UL +#define LCE_SRC_BUFFER_DOMAIN 4UL +#define LCE_DST_BUFFER_DOMAIN 8UL + uint32_t type: 4; + uint32_t resv: 8; + struct lce_qat_domain dom; +}; + +/* QAT GEN 5 specific macros */ +#define QAT_GEN5_BUNDLE_NUM LCE_DEVICE_MAX_BANKS +#define QAT_GEN5_QPS_PER_BUNDLE_NUM 1 + +struct qat_dev_gen5_extra { + struct qat_qp_hw_data + qp_gen5_data[QAT_GEN5_BUNDLE_NUM][QAT_GEN5_QPS_PER_BUNDLE_NUM]; +}; + +static struct qat_pf2vf_dev qat_pf2vf_gen5 = { + .pf2vf_offset = ADF_4XXXIOV_PF2VM_OFFSET, + .vf2pf_offset = ADF_4XXXIOV_VM2PF_OFFSET, + .pf2vf_type_shift = ADF_PFVF_2X_MSGTYPE_SHIFT, + .pf2vf_type_mask = ADF_PFVF_2X_MSGTYPE_MASK, + .pf2vf_data_shift = ADF_PFVF_2X_MSGDATA_SHIFT, + .pf2vf_data_mask = ADF_PFVF_2X_MSGDATA_MASK, +}; + +static int +qat_select_valid_queue_gen5(struct qat_pci_device *qat_dev, int qp_id, + enum qat_service_type service_type) +{ + int i = 0, valid_qps = 0; + struct qat_dev_gen5_extra *dev_extra = qat_dev->dev_private; + + for (; i < QAT_GEN5_BUNDLE_NUM; i++) { + if (dev_extra->qp_gen5_data[i][0].service_type == + service_type) { + if (valid_qps == qp_id) + return i; + ++valid_qps; + } + } + return -1; +} + +static const struct qat_qp_hw_data * +qat_qp_get_hw_data_gen5(struct qat_pci_device *qat_dev, + enum qat_service_type service_type, uint16_t qp_id) +{ + struct qat_dev_gen5_extra *dev_extra = qat_dev->dev_private; + int ring_pair = qat_select_valid_queue_gen5(qat_dev, qp_id, + service_type); + + if (ring_pair < 0) + return NULL; + + return &dev_extra->qp_gen5_data[ring_pair][0]; +} + +static int +qat_qp_rings_per_service_gen5(struct qat_pci_device *qat_dev, + enum qat_service_type service) +{ + int i = 0, count = 0, max_ops_per_srv = 0; + struct qat_dev_gen5_extra *dev_extra = qat_dev->dev_private; + + max_ops_per_srv = QAT_GEN5_BUNDLE_NUM; + for (i = 0, count = 0; i < max_ops_per_srv; i++) + if (dev_extra->qp_gen5_data[i][0].service_type == service) + count++; + return count; +} + +static int qat_dev_read_config(struct qat_pci_device *qat_dev) +{ + struct qat_dev_gen5_extra *dev_extra = qat_dev->dev_private; + struct qat_qp_hw_data *hw_data; + + /** Enable only crypto ring: RP-0 */ + hw_data = &dev_extra->qp_gen5_data[0][0]; + memset(hw_data, 0, sizeof(*hw_data)); + + hw_data->service_type = QAT_SERVICE_SYMMETRIC; + hw_data->tx_msg_size = 128; + hw_data->rx_msg_size = 32; + + hw_data->tx_ring_num = 0; + hw_data->rx_ring_num = 1; + + hw_data->hw_bundle_num = 0; + + return 0; +} + + +static int qat_dev_read_config_gen5(struct qat_pci_device *qat_dev) +{ + return qat_dev_read_config(qat_dev); +} + +static void qat_qp_build_ring_base_gen5(void *io_addr, struct qat_queue *queue) +{ + uint64_t queue_base; + + queue_base = BUILD_RING_BASE_ADDR_GEN5(queue->base_phys_addr, + queue->queue_size); + WRITE_CSR_RING_BASE_GEN5VF(io_addr, queue->hw_bundle_number, + queue->hw_queue_number, queue_base); +} + +static void +qat_qp_adf_arb_enable_gen5(const struct qat_queue *txq, + void *base_addr, rte_spinlock_t *lock) +{ + uint32_t arb_csr_offset = 0, value; + + rte_spinlock_lock(lock); + arb_csr_offset = ADF_ARB_RINGSRVARBEN_OFFSET + + (ADF_RING_BUNDLE_SIZE_GEN5 * + txq->hw_bundle_number); + value = ADF_CSR_RD(base_addr + ADF_RING_CSR_ADDR_OFFSET_GEN5VF, + arb_csr_offset); + value |= 0x01; + ADF_CSR_WR(base_addr, arb_csr_offset, value); + rte_spinlock_unlock(lock); +} + +static void +qat_qp_adf_arb_disable_gen5(const struct qat_queue *txq, + void *base_addr, rte_spinlock_t *lock) +{ + uint32_t arb_csr_offset = 0, value; + + rte_spinlock_lock(lock); + arb_csr_offset = ADF_ARB_RINGSRVARBEN_OFFSET + (ADF_RING_BUNDLE_SIZE_GEN5 * + txq->hw_bundle_number); + value = ADF_CSR_RD(base_addr + ADF_RING_CSR_ADDR_OFFSET_GEN5VF, + arb_csr_offset); + value &= ~(0x01); + ADF_CSR_WR(base_addr, arb_csr_offset, value); + rte_spinlock_unlock(lock); +} + +static void +qat_qp_adf_configure_queues_gen5(struct qat_qp *qp) +{ + uint32_t q_tx_config, q_resp_config; + struct qat_queue *q_tx = &qp->tx_q, *q_rx = &qp->rx_q; + + /* q_tx/rx->queue_size is initialized as per bundle config register */ + q_tx_config = BUILD_RING_CONFIG(q_tx->queue_size); + + q_resp_config = BUILD_RESP_RING_CONFIG(q_rx->queue_size, + ADF_RING_NEAR_WATERMARK_512, + ADF_RING_NEAR_WATERMARK_0); + + WRITE_CSR_RING_CONFIG_GEN5VF(qp->mmap_bar_addr, q_tx->hw_bundle_number, + q_tx->hw_queue_number, q_tx_config); + WRITE_CSR_RING_CONFIG_GEN5VF(qp->mmap_bar_addr, q_rx->hw_bundle_number, + q_rx->hw_queue_number, q_resp_config); +} + +static void +qat_qp_csr_write_tail_gen5(struct qat_qp *qp, struct qat_queue *q) +{ + WRITE_CSR_RING_TAIL_GEN5VF(qp->mmap_bar_addr, q->hw_bundle_number, + q->hw_queue_number, q->tail); +} + +static void +qat_qp_csr_write_head_gen5(struct qat_qp *qp, struct qat_queue *q, + uint32_t new_head) +{ + WRITE_CSR_RING_HEAD_GEN5VF(qp->mmap_bar_addr, q->hw_bundle_number, + q->hw_queue_number, new_head); +} + +static void +qat_qp_csr_setup_gen5(struct qat_pci_device *qat_dev, void *io_addr, + struct qat_qp *qp) +{ + qat_qp_build_ring_base_gen5(io_addr, &qp->tx_q); + qat_qp_build_ring_base_gen5(io_addr, &qp->rx_q); + qat_qp_adf_configure_queues_gen5(qp); + qat_qp_adf_arb_enable_gen5(&qp->tx_q, qp->mmap_bar_addr, + &qat_dev->arb_csr_lock); +} + +static struct qat_qp_hw_spec_funcs qat_qp_hw_spec_gen5 = { + .qat_qp_rings_per_service = qat_qp_rings_per_service_gen5, + .qat_qp_build_ring_base = qat_qp_build_ring_base_gen5, + .qat_qp_adf_arb_enable = qat_qp_adf_arb_enable_gen5, + .qat_qp_adf_arb_disable = qat_qp_adf_arb_disable_gen5, + .qat_qp_adf_configure_queues = qat_qp_adf_configure_queues_gen5, + .qat_qp_csr_write_tail = qat_qp_csr_write_tail_gen5, + .qat_qp_csr_write_head = qat_qp_csr_write_head_gen5, + .qat_qp_csr_setup = qat_qp_csr_setup_gen5, + .qat_qp_get_hw_data = qat_qp_get_hw_data_gen5, +}; + +static int +qat_reset_ring_pairs_gen5(struct qat_pci_device *qat_pci_dev __rte_unused) +{ + return 0; +} + +static const struct rte_mem_resource* +qat_dev_get_transport_bar_gen5(struct rte_pci_device *pci_dev) +{ + return &pci_dev->mem_resource[0]; +} + +static int +qat_dev_get_misc_bar_gen5(struct rte_mem_resource **mem_resource, + struct rte_pci_device *pci_dev) +{ + *mem_resource = &pci_dev->mem_resource[2]; + return 0; +} + +static int +qat_dev_get_extra_size_gen5(void) +{ + return sizeof(struct qat_dev_gen5_extra); +} + +static int +qat_dev_get_slice_map_gen5(uint32_t *map __rte_unused, + const struct rte_pci_device *pci_dev __rte_unused) +{ + return 0; +} + +static struct qat_dev_hw_spec_funcs qat_dev_hw_spec_gen5 = { + .qat_dev_reset_ring_pairs = qat_reset_ring_pairs_gen5, + .qat_dev_get_transport_bar = qat_dev_get_transport_bar_gen5, + .qat_dev_get_misc_bar = qat_dev_get_misc_bar_gen5, + .qat_dev_read_config = qat_dev_read_config_gen5, + .qat_dev_get_extra_size = qat_dev_get_extra_size_gen5, + .qat_dev_get_slice_map = qat_dev_get_slice_map_gen5, +}; + +RTE_INIT(qat_dev_gen_5_init) +{ + qat_qp_hw_spec[QAT_GEN5] = &qat_qp_hw_spec_gen5; + qat_dev_hw_spec[QAT_GEN5] = &qat_dev_hw_spec_gen5; + qat_gen_config[QAT_GEN5].dev_gen = QAT_GEN5; + qat_gen_config[QAT_GEN5].pf2vf_dev = &qat_pf2vf_gen5; +} diff --git a/drivers/common/qat/qat_adf/adf_transport_access_macros_gen5.h b/drivers/common/qat/qat_adf/adf_transport_access_macros_gen5.h new file mode 100644 index 0000000000..29ce6b8e60 --- /dev/null +++ b/drivers/common/qat/qat_adf/adf_transport_access_macros_gen5.h @@ -0,0 +1,51 @@ +/* SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0) + * Copyright(c) 2023 Intel Corporation + */ + +#ifndef ADF_TRANSPORT_ACCESS_MACROS_GEN5_H +#define ADF_TRANSPORT_ACCESS_MACROS_GEN5_H + +#include "adf_transport_access_macros.h" + +#define ADF_RINGS_PER_INT_SRCSEL_GEN4 2 +#define ADF_BANK_INT_SRC_SEL_MASK_GEN4 0x44UL +#define ADF_BANK_INT_FLAG_CLEAR_MASK_GEN4 0x3 +#define ADF_RING_BUNDLE_SIZE_GEN5 0x2000 +#define ADF_RING_CSR_RING_CONFIG_GEN5 0x1000 +#define ADF_RING_CSR_RING_LBASE_GEN5 0x1040 +#define ADF_RING_CSR_RING_UBASE_GEN5 0x1080 + +#define BUILD_RING_BASE_ADDR_GEN5(addr, size) \ + ((((addr) >> 6) & (0xFFFFFFFFFFFFFFFFULL << (size))) << 6) + +#define WRITE_CSR_RING_BASE_GEN5(csr_base_addr, bank, ring, value) \ +do { \ + uint32_t l_base = 0, u_base = 0; \ + l_base = (uint32_t)(value & 0xFFFFFFFF); \ + u_base = (uint32_t)((value & 0xFFFFFFFF00000000ULL) >> 32); \ + ADF_CSR_WR(csr_base_addr, \ + (ADF_RING_BUNDLE_SIZE_GEN5 * bank) + \ + ADF_RING_CSR_RING_LBASE_GEN5 + (ring << 2), \ + l_base); \ + ADF_CSR_WR(csr_base_addr, \ + (ADF_RING_BUNDLE_SIZE_GEN5 * bank) + \ + ADF_RING_CSR_RING_UBASE_GEN5 + (ring << 2), \ + u_base); \ +} while (0) + +#define WRITE_CSR_RING_CONFIG_GEN5(csr_base_addr, bank, ring, value) \ + ADF_CSR_WR(csr_base_addr, \ + (ADF_RING_BUNDLE_SIZE_GEN5 * bank) + \ + ADF_RING_CSR_RING_CONFIG_GEN5 + (ring << 2), value) + +#define WRITE_CSR_RING_TAIL_GEN5(csr_base_addr, bank, ring, value) \ + ADF_CSR_WR((u8 *)(csr_base_addr), \ + (ADF_RING_BUNDLE_SIZE_GEN5 * (bank)) + \ + ADF_RING_CSR_RING_TAIL + ((ring) << 2), value) + +#define WRITE_CSR_RING_HEAD_GEN5(csr_base_addr, bank, ring, value) \ + ADF_CSR_WR((u8 *)(csr_base_addr), \ + (ADF_RING_BUNDLE_SIZE_GEN5 * (bank)) + \ + ADF_RING_CSR_RING_HEAD + ((ring) << 2), value) + +#endif diff --git a/drivers/common/qat/qat_adf/adf_transport_access_macros_gen5vf.h b/drivers/common/qat/qat_adf/adf_transport_access_macros_gen5vf.h new file mode 100644 index 0000000000..5d2c6706a6 --- /dev/null +++ b/drivers/common/qat/qat_adf/adf_transport_access_macros_gen5vf.h @@ -0,0 +1,48 @@ +/* SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0) + * Copyright(c) 2023 Intel Corporation + */ + +#ifndef ADF_TRANSPORT_ACCESS_MACROS_GEN5VF_H +#define ADF_TRANSPORT_ACCESS_MACROS_GEN5VF_H + +#include "adf_transport_access_macros.h" +#include "adf_transport_access_macros_gen5.h" + +#define ADF_RING_CSR_ADDR_OFFSET_GEN5VF 0x0 + +#define WRITE_CSR_RING_BASE_GEN5VF(csr_base_addr, bank, ring, value) \ +do { \ + uint32_t l_base = 0, u_base = 0; \ + l_base = (uint32_t)(value & 0xFFFFFFFF); \ + u_base = (uint32_t)((value & 0xFFFFFFFF00000000ULL) >> 32); \ + ADF_CSR_WR(csr_base_addr + ADF_RING_CSR_ADDR_OFFSET_GEN5VF, \ + (ADF_RING_BUNDLE_SIZE_GEN5 * bank) + \ + ADF_RING_CSR_RING_LBASE_GEN5 + (ring << 2), \ + l_base); \ + ADF_CSR_WR(csr_base_addr + ADF_RING_CSR_ADDR_OFFSET_GEN5VF, \ + (ADF_RING_BUNDLE_SIZE_GEN5 * bank) + \ + ADF_RING_CSR_RING_UBASE_GEN5 + (ring << 2), \ + u_base); \ +} while (0) + +#define WRITE_CSR_RING_CONFIG_GEN5VF(csr_base_addr, bank, ring, value) \ + ADF_CSR_WR(csr_base_addr + ADF_RING_CSR_ADDR_OFFSET_GEN5VF, \ + (ADF_RING_BUNDLE_SIZE_GEN5 * bank) + \ + ADF_RING_CSR_RING_CONFIG_GEN5 + (ring << 2), value) + +#define WRITE_CSR_RING_TAIL_GEN5VF(csr_base_addr, bank, ring, value) \ + ADF_CSR_WR((csr_base_addr) + ADF_RING_CSR_ADDR_OFFSET_GEN5VF, \ + (ADF_RING_BUNDLE_SIZE_GEN5 * (bank)) + \ + ADF_RING_CSR_RING_TAIL + ((ring) << 2), (value)) + +#define WRITE_CSR_RING_HEAD_GEN5VF(csr_base_addr, bank, ring, value) \ + ADF_CSR_WR((csr_base_addr) + ADF_RING_CSR_ADDR_OFFSET_GEN5VF, \ + (ADF_RING_BUNDLE_SIZE_GEN5 * (bank)) + \ + ADF_RING_CSR_RING_HEAD + ((ring) << 2), (value)) + +#define WRITE_CSR_RING_SRV_ARB_EN_GEN5VF(csr_base_addr, bank, value) \ + ADF_CSR_WR((csr_base_addr) + ADF_RING_CSR_ADDR_OFFSET_GEN5VF, \ + (ADF_RING_BUNDLE_SIZE_GEN5 * (bank)) + \ + ADF_RING_CSR_RING_SRV_ARB_EN, (value)) + +#endif diff --git a/drivers/crypto/qat/dev/qat_crypto_pmd_gen5.c b/drivers/crypto/qat/dev/qat_crypto_pmd_gen5.c new file mode 100644 index 0000000000..1f1242c5c0 --- /dev/null +++ b/drivers/crypto/qat/dev/qat_crypto_pmd_gen5.c @@ -0,0 +1,336 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2023 Intel Corporation + */ + +#include +#include +#include "qat_sym_session.h" +#include "qat_sym.h" +#include "qat_asym.h" +#include "qat_crypto.h" +#include "qat_crypto_pmd_gens.h" + +static struct rte_cryptodev_capabilities qat_sym_crypto_caps_gen5[] = { + QAT_SYM_AEAD_CAP(AES_GCM, + CAP_SET(block_size, 16), + CAP_RNG(key_size, 32, 32, 1), CAP_RNG(digest_size, 16, 16, 1), + CAP_RNG(aad_size, 0, 240, 1), CAP_RNG(iv_size, 12, 12, 1)), + RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST() +}; + +static int +qat_sgl_add_buffer_gen5(void *list_in, uint64_t addr, uint32_t len) +{ + struct qat_sgl *list = (struct qat_sgl *)list_in; + uint32_t nr; + + nr = list->num_bufs; + + if (nr >= QAT_SYM_SGL_MAX_NUMBER) { + QAT_DP_LOG(ERR, "Adding %d entry failed, no empty SGL buffer", nr); + return -EINVAL; + } + + list->buffers[nr].len = len; + list->buffers[nr].resrvd = 0; + list->buffers[nr].addr = addr; + + list->num_bufs++; +#if RTE_LOG_DP_LEVEL >= RTE_LOG_DEBUG + QAT_DP_LOG(INFO, "SGL with %d buffers:", list->num_bufs); + QAT_DP_LOG(INFO, "QAT SGL buf %d, len = %d, iova = 0x%012"PRIx64, + nr, list->buffers[nr].len, list->buffers[nr].addr); +#endif + return 0; +} + +static int +qat_sgl_fill_array_with_mbuf(struct rte_mbuf *buf, int64_t offset, + void *list_in, uint32_t data_len) +{ + struct qat_sgl *list = (struct qat_sgl *)list_in; + uint32_t nr, buf_len; + int res = -EINVAL; +#if RTE_LOG_DP_LEVEL >= RTE_LOG_DEBUG + uint32_t start_idx; + start_idx = list->num_bufs; +#endif + + /* Append to the existing list */ + nr = list->num_bufs; + + for (buf_len = 0; buf && nr < QAT_SYM_SGL_MAX_NUMBER; buf = buf->next) { + if (offset >= rte_pktmbuf_data_len(buf)) { + offset -= rte_pktmbuf_data_len(buf); + /* Jump to next mbuf */ + continue; + } + + list->buffers[nr].len = rte_pktmbuf_data_len(buf) - offset; + list->buffers[nr].resrvd = 0; + list->buffers[nr].addr = rte_pktmbuf_iova_offset(buf, offset); + + offset = 0; + buf_len += list->buffers[nr].len; + + if (buf_len >= data_len) { + list->buffers[nr].len -= buf_len - data_len; + res = 0; + break; + } + ++nr; + } + + if (unlikely(res != 0)) { + if (nr == QAT_SYM_SGL_MAX_NUMBER) + QAT_DP_LOG(ERR, "Exceeded max segments in QAT SGL (%u)", + QAT_SYM_SGL_MAX_NUMBER); + else + QAT_DP_LOG(ERR, "Mbuf chain is too short"); + } else { + + list->num_bufs = ++nr; +#if RTE_LOG_DP_LEVEL >= RTE_LOG_DEBUG + QAT_DP_LOG(INFO, "SGL with %d buffers:", list->num_bufs); + for (nr = start_idx; nr < list->num_bufs; nr++) { + QAT_DP_LOG(INFO, "QAT SGL buf %d, len = %d, iova = 0x%012"PRIx64, + nr, list->buffers[nr].len, + list->buffers[nr].addr); + } +#endif + } + + return res; +} + +static int +qat_sym_build_op_aead_gen5(void *in_op, struct qat_sym_session *ctx, + uint8_t *out_msg, void *op_cookie) +{ + struct qat_sym_op_cookie *cookie = op_cookie; + struct rte_crypto_op *op = in_op; + uint64_t digest_phys_addr, aad_phys_addr; + uint16_t iv_len, aad_len, digest_len, key_len; + uint32_t cipher_ofs, iv_offset, cipher_len; + register struct icp_qat_fw_la_bulk_req *qat_req; + struct icp_qat_fw_la_cipher_30_req_params *cipher_param; + enum icp_qat_hw_cipher_dir dir; + bool is_digest_adjacent = false; + + if (ctx->qat_cmd != ICP_QAT_FW_LA_CMD_CIPHER || + ctx->qat_cipher_alg != ICP_QAT_HW_CIPHER_ALGO_AES256 || + ctx->qat_mode != ICP_QAT_HW_CIPHER_AEAD_MODE) { + + QAT_DP_LOG(ERR, "Not supported (cmd: %d, alg: %d, mode: %d). " + "GEN5 PMD only supports AES-256 AEAD mode", + ctx->qat_cmd, ctx->qat_cipher_alg, ctx->qat_mode); + return -EINVAL; + } + + qat_req = (struct icp_qat_fw_la_bulk_req *)out_msg; + rte_mov128((uint8_t *)qat_req, (const uint8_t *)&(ctx->fw_req)); + qat_req->comn_mid.opaque_data = (uint64_t)(uintptr_t)op; + cipher_param = (void *)&qat_req->serv_specif_rqpars; + + dir = ctx->qat_dir; + + aad_phys_addr = op->sym->aead.aad.phys_addr; + aad_len = ctx->aad_len; + + iv_offset = ctx->cipher_iv.offset; + iv_len = ctx->cipher_iv.length; + + cipher_ofs = op->sym->aead.data.offset; + cipher_len = op->sym->aead.data.length; + + digest_phys_addr = op->sym->aead.digest.phys_addr; + digest_len = ctx->digest_length; + + /* Upto 16B IV can be directly embedded in descriptor. + * But GCM supports only 12B IV + */ + if (iv_len != GCM_IV_LENGTH) { + QAT_DP_LOG(ERR, "iv_len: %d not supported. Must be 12B.", + iv_len); + return -EINVAL; + } + + rte_memcpy(cipher_param->u.cipher_IV_array, + rte_crypto_op_ctod_offset(op, uint8_t*, iv_offset), + iv_len); + + /* Always SGL */ + RTE_ASSERT((qat_req->comn_hdr.comn_req_flags & + ICP_QAT_FW_SYM_COMM_ADDR_SGL) == 1); + /* Always inplace */ + RTE_ASSERT(op->sym->m_dst == NULL); + + /* Key buffer address is already programmed by reusing the + * content-descriptor buffer + */ + key_len = ctx->auth_key_length; + + cipher_param->spc_aad_sz = aad_len; + cipher_param->cipher_length = key_len; + cipher_param->spc_auth_res_sz = digest_len; + + /* Knowing digest is contiguous to cipher-text helps optimizing SGL */ + if (rte_pktmbuf_iova_offset(op->sym->m_src, cipher_ofs + cipher_len) + == digest_phys_addr) + is_digest_adjacent = true; + + /* SRC-SGL: 3 entries: + * a) AAD + * b) cipher + * c) digest (only for decrypt and buffer is_NOT_adjacent) + * + */ + cookie->qat_sgl_src.num_bufs = 0; + if (aad_len) + qat_sgl_add_buffer_gen5(&cookie->qat_sgl_src, aad_phys_addr, + aad_len); + + if (is_digest_adjacent && dir == ICP_QAT_HW_CIPHER_DECRYPT) { + qat_sgl_fill_array_with_mbuf(op->sym->m_src, cipher_ofs, + &cookie->qat_sgl_src, + cipher_len + digest_len); + } else { + qat_sgl_fill_array_with_mbuf(op->sym->m_src, cipher_ofs, + &cookie->qat_sgl_src, + cipher_len); + + /* Digest buffer in decrypt job */ + if (dir == ICP_QAT_HW_CIPHER_DECRYPT) + qat_sgl_add_buffer_gen5(&cookie->qat_sgl_src, + digest_phys_addr, digest_len); + } + + /* (in-place) DST-SGL: 2 entries: + * a) cipher + * b) digest (only for encrypt and buffer is_NOT_adjacent) + */ + cookie->qat_sgl_dst.num_bufs = 0; + + if (is_digest_adjacent && dir == ICP_QAT_HW_CIPHER_ENCRYPT) { + qat_sgl_fill_array_with_mbuf(op->sym->m_src, cipher_ofs, + &cookie->qat_sgl_dst, + cipher_len + digest_len); + } else { + qat_sgl_fill_array_with_mbuf(op->sym->m_src, cipher_ofs, + &cookie->qat_sgl_dst, + cipher_len); + + /* Digest buffer in Encrypt job */ + if (dir == ICP_QAT_HW_CIPHER_ENCRYPT) + qat_sgl_add_buffer_gen5(&cookie->qat_sgl_dst, + digest_phys_addr, digest_len); + } + + /* Length values in 128B descriptor */ + qat_req->comn_mid.src_length = cipher_len; + qat_req->comn_mid.dst_length = cipher_len; + + if (dir == ICP_QAT_HW_CIPHER_ENCRYPT) /* Digest buffer in Encrypt job */ + qat_req->comn_mid.dst_length += GCM_256_DIGEST_LEN; + + /* src & dst SGL addresses in 128B descriptor */ + qat_req->comn_mid.src_data_addr = cookie->qat_sgl_src_phys_addr; + qat_req->comn_mid.dest_data_addr = cookie->qat_sgl_dst_phys_addr; + +#if RTE_LOG_DP_LEVEL >= RTE_LOG_DEBUG + QAT_DP_HEXDUMP_LOG(DEBUG, "qat_req:", qat_req, + sizeof(struct icp_qat_fw_la_bulk_req)); + QAT_DP_HEXDUMP_LOG(DEBUG, "src_data:", + rte_pktmbuf_mtod(op->sym->m_src, uint8_t*), + rte_pktmbuf_data_len(op->sym->m_src)); + QAT_DP_HEXDUMP_LOG(DEBUG, "digest:", op->sym->aead.digest.data, + digest_len); + QAT_DP_HEXDUMP_LOG(DEBUG, "aad:", op->sym->aead.aad.data, aad_len); +#endif + return 0; +} + +static int +qat_sym_crypto_set_session_gen5(void *cdev __rte_unused, void *session) +{ + struct qat_sym_session *ctx = session; + qat_sym_build_request_t build_request = NULL; + enum rte_proc_type_t proc_type = rte_eal_process_type(); + + if (proc_type == RTE_PROC_AUTO || proc_type == RTE_PROC_INVALID) + return -EINVAL; + + /* build request for aead */ + if (ctx->qat_cipher_alg == ICP_QAT_HW_CIPHER_ALGO_AES256 && + ctx->qat_hash_alg == ICP_QAT_HW_AUTH_ALGO_GALOIS_128) { + build_request = qat_sym_build_op_aead_gen5; + if (build_request) + ctx->build_request[proc_type] = build_request; + else + return -EINVAL; + } + return 0; +} + + +static int +qat_sym_crypto_cap_get_gen5(struct qat_cryptodev_private *internals, + const char *capa_memz_name, + const uint16_t __rte_unused slice_map) +{ + const uint32_t size = sizeof(qat_sym_crypto_caps_gen5); + uint32_t i; + + internals->capa_mz = rte_memzone_lookup(capa_memz_name); + if (internals->capa_mz == NULL) { + internals->capa_mz = rte_memzone_reserve(capa_memz_name, + size, rte_socket_id(), 0); + if (internals->capa_mz == NULL) { + QAT_LOG(DEBUG, + "Error allocating memzone for capabilities"); + return -1; + } + } + + struct rte_cryptodev_capabilities *addr = + (struct rte_cryptodev_capabilities *) + internals->capa_mz->addr; + const struct rte_cryptodev_capabilities *capabilities = + qat_sym_crypto_caps_gen5; + const uint32_t capa_num = + size / sizeof(struct rte_cryptodev_capabilities); + uint32_t curr_capa = 0; + + for (i = 0; i < capa_num; i++) { + memcpy(addr + curr_capa, capabilities + i, + sizeof(struct rte_cryptodev_capabilities)); + curr_capa++; + } + internals->qat_dev_capabilities = internals->capa_mz->addr; + + return 0; +} + +RTE_INIT(qat_sym_crypto_gen5_init) +{ + qat_sym_gen_dev_ops[QAT_GEN5].cryptodev_ops = &qat_sym_crypto_ops_gen1; + qat_sym_gen_dev_ops[QAT_GEN5].get_capabilities = + qat_sym_crypto_cap_get_gen5; + qat_sym_gen_dev_ops[QAT_GEN5].set_session = + qat_sym_crypto_set_session_gen5; + qat_sym_gen_dev_ops[QAT_GEN5].set_raw_dp_ctx = NULL; + qat_sym_gen_dev_ops[QAT_GEN5].get_feature_flags = + qat_sym_crypto_feature_flags_get_gen1; +#ifdef RTE_LIB_SECURITY + qat_sym_gen_dev_ops[QAT_GEN5].create_security_ctx = + qat_sym_create_security_gen1; +#endif +} + +RTE_INIT(qat_asym_crypto_gen5_init) +{ + qat_asym_gen_dev_ops[QAT_GEN5].cryptodev_ops = NULL; + qat_asym_gen_dev_ops[QAT_GEN5].get_capabilities = NULL; + qat_asym_gen_dev_ops[QAT_GEN5].get_feature_flags = NULL; + qat_asym_gen_dev_ops[QAT_GEN5].set_session = NULL; +} From patchwork Wed Dec 20 13:26:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nayak, Nishikanta" X-Patchwork-Id: 135395 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1389C4374A; Wed, 20 Dec 2023 14:26:58 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8627E410EE; Wed, 20 Dec 2023 14:26:52 +0100 (CET) Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.126]) by mails.dpdk.org (Postfix) with ESMTP id 4BF644021F for ; Wed, 20 Dec 2023 14:26:49 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703078809; x=1734614809; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=vXNVeSUb+slx32luvo6A51kf9HytgOdY/+lTWOoJHII=; b=S7SBgFIPXI2MBycqMOBM5df45u70FFwFNkt21i2dtNqlhqYBUpn36hce 6xVRdmb11XAwcdSsstSJpYiL0GTixXSCgpmhRqnjXYDfZBgrIg1OZIVjh DdNhm6KZ0dvJrQfbfNrWVuJaz8Xam0blYOJaBkpaWUcAt0h29xm8XLilA a3dfrFiVoEMNyBodO2pWGlIxn9Eyjvc3DY5hD8cUM7jnca2AlCv7AgnEZ rHgAPwLJnfZcfwMdTahaPmRS9uhrfaV1u4622/2AeVJzaq2Z16om33QxU 7UopjGGG907eHwJvMjGDVT7A0IPFEqKeAiPqZqL2irtcgo0FN+ginA6xX Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10929"; a="380801525" X-IronPort-AV: E=Sophos;i="6.04,291,1695711600"; d="scan'208";a="380801525" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Dec 2023 05:26:48 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10929"; a="842276266" X-IronPort-AV: E=Sophos;i="6.04,291,1695711600"; d="scan'208";a="842276266" Received: from silpixa00400355.ir.intel.com (HELO silpixa00400355.ger.corp.intel.com) ([10.237.222.80]) by fmsmga008.fm.intel.com with ESMTP; 20 Dec 2023 05:26:47 -0800 From: Nishikant Nayak To: dev@dpdk.org Cc: kai.ji@intel.com, ciara.power@intel.com, arkadiuszx.kusztal@intel.com, Nishikant Nayak Subject: [PATCH 2/4] common/qat: update common driver to support GEN5 Date: Wed, 20 Dec 2023 13:26:14 +0000 Message-Id: <20231220132616.318983-2-nishikanta.nayak@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231220132616.318983-1-nishikanta.nayak@intel.com> References: <20231220132616.318983-1-nishikanta.nayak@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Adding GEN5 specific macros which is required for updating the support for GEN5 features. Also this patch adds other macros which is being used by GEN5 Specific APIs. Signed-off-by: Nishikant Nayak --- drivers/common/qat/meson.build | 2 + .../qat/qat_adf/adf_transport_access_macros.h | 1 + drivers/common/qat/qat_adf/icp_qat_fw.h | 27 ++++++++++ drivers/common/qat/qat_adf/icp_qat_fw_la.h | 51 +++++++++++++++++++ drivers/common/qat/qat_common.h | 1 + drivers/common/qat/qat_device.c | 9 ++++ 6 files changed, 91 insertions(+) diff --git a/drivers/common/qat/meson.build b/drivers/common/qat/meson.build index 5c36fbb270..35389e5aba 100644 --- a/drivers/common/qat/meson.build +++ b/drivers/common/qat/meson.build @@ -82,6 +82,7 @@ sources += files( 'dev/qat_dev_gen2.c', 'dev/qat_dev_gen3.c', 'dev/qat_dev_gen4.c', + 'dev/qat_dev_gen5.c', ) includes += include_directories( 'qat_adf', @@ -108,6 +109,7 @@ if qat_crypto 'dev/qat_crypto_pmd_gen2.c', 'dev/qat_crypto_pmd_gen3.c', 'dev/qat_crypto_pmd_gen4.c', + 'dev/qat_crypto_pmd_gen5.c', ] sources += files(join_paths(qat_crypto_relpath, f)) endforeach diff --git a/drivers/common/qat/qat_adf/adf_transport_access_macros.h b/drivers/common/qat/qat_adf/adf_transport_access_macros.h index 12a7258c60..19bd812419 100644 --- a/drivers/common/qat/qat_adf/adf_transport_access_macros.h +++ b/drivers/common/qat/qat_adf/adf_transport_access_macros.h @@ -47,6 +47,7 @@ #define ADF_RING_SIZE_512 0x03 #define ADF_RING_SIZE_4K 0x06 #define ADF_RING_SIZE_16K 0x08 +#define ADF_RING_SIZE_64K 0x0A #define ADF_RING_SIZE_4M 0x10 #define ADF_MIN_RING_SIZE ADF_RING_SIZE_128 #define ADF_MAX_RING_SIZE ADF_RING_SIZE_4M diff --git a/drivers/common/qat/qat_adf/icp_qat_fw.h b/drivers/common/qat/qat_adf/icp_qat_fw.h index 3aa17ae041..b06b7ec989 100644 --- a/drivers/common/qat/qat_adf/icp_qat_fw.h +++ b/drivers/common/qat/qat_adf/icp_qat_fw.h @@ -123,6 +123,11 @@ struct icp_qat_fw_comn_resp { #define ICP_QAT_FW_COMN_NULL_VERSION_FLAG_BITPOS 0 #define ICP_QAT_FW_COMN_NULL_VERSION_FLAG_MASK 0x1 +/* GEN5 specific Common Header fields */ +#define ICP_QAT_FW_COMN_DESC_LAYOUT_BITPOS 5 +#define ICP_QAT_FW_COMN_DESC_LAYOUT_MASK 0x3 +#define ICP_QAT_FW_COMN_GEN5_DESC_LAYOUT 3 + #define ICP_QAT_FW_COMN_OV_SRV_TYPE_GET(icp_qat_fw_comn_req_hdr_t) \ icp_qat_fw_comn_req_hdr_t.service_type @@ -168,6 +173,12 @@ struct icp_qat_fw_comn_resp { (((valid) & ICP_QAT_FW_COMN_VALID_FLAG_MASK) << \ ICP_QAT_FW_COMN_VALID_FLAG_BITPOS) +#define ICP_QAT_FW_COMN_HDR_FLAGS_BUILD_GEN5(valid, desc_layout) \ + ((((valid) & ICP_QAT_FW_COMN_VALID_FLAG_MASK) << \ + ICP_QAT_FW_COMN_VALID_FLAG_BITPOS) | \ + (((desc_layout) & ICP_QAT_FW_COMN_DESC_LAYOUT_MASK) << \ + ICP_QAT_FW_COMN_DESC_LAYOUT_BITPOS)) + #define QAT_COMN_PTR_TYPE_BITPOS 0 #define QAT_COMN_PTR_TYPE_MASK 0x1 #define QAT_COMN_CD_FLD_TYPE_BITPOS 1 @@ -180,10 +191,20 @@ struct icp_qat_fw_comn_resp { #define QAT_COMN_EXT_FLAGS_MASK 0x1 #define QAT_COMN_EXT_FLAGS_USED 0x1 +/* GEN5 specific Common Request Flags fields */ +#define QAT_COMN_KEYBUF_USAGE_BITPOS 1 +#define QAT_COMN_KEYBUF_USAGE_MASK 0x1 +#define QAT_COMN_KEY_BUFFER_USED 1 + #define ICP_QAT_FW_COMN_FLAGS_BUILD(cdt, ptr) \ ((((cdt) & QAT_COMN_CD_FLD_TYPE_MASK) << QAT_COMN_CD_FLD_TYPE_BITPOS) \ | (((ptr) & QAT_COMN_PTR_TYPE_MASK) << QAT_COMN_PTR_TYPE_BITPOS)) +#define ICP_QAT_FW_COMN_FLAGS_BUILD_GEN5(ptr, keybuf) \ + ((((ptr) & QAT_COMN_PTR_TYPE_MASK) << QAT_COMN_PTR_TYPE_BITPOS) | \ + (((keybuf) & QAT_COMN_PTR_TYPE_MASK) << \ + QAT_COMN_KEYBUF_USAGE_BITPOS)) + #define ICP_QAT_FW_COMN_PTR_TYPE_GET(flags) \ QAT_FIELD_GET(flags, QAT_COMN_PTR_TYPE_BITPOS, QAT_COMN_PTR_TYPE_MASK) @@ -249,6 +270,8 @@ struct icp_qat_fw_comn_resp { #define QAT_COMN_RESP_CMP_END_OF_LAST_BLK_MASK 0x1 #define QAT_COMN_RESP_UNSUPPORTED_REQUEST_BITPOS 2 #define QAT_COMN_RESP_UNSUPPORTED_REQUEST_MASK 0x1 +#define QAT_COMN_RESP_INVALID_PARAM_BITPOS 1 +#define QAT_COMN_RESP_INVALID_PARAM_MASK 0x1 #define QAT_COMN_RESP_XLT_WA_APPLIED_BITPOS 0 #define QAT_COMN_RESP_XLT_WA_APPLIED_MASK 0x1 @@ -280,6 +303,10 @@ struct icp_qat_fw_comn_resp { QAT_FIELD_GET(status, QAT_COMN_RESP_UNSUPPORTED_REQUEST_BITPOS, \ QAT_COMN_RESP_UNSUPPORTED_REQUEST_MASK) +#define ICP_QAT_FW_COMN_RESP_INVALID_PARAM_STAT_GET(status) \ + QAT_FIELD_GET(status, QAT_COMN_RESP_INVALID_PARAM_BITPOS, \ + QAT_COMN_RESP_INVALID_PARAM_MASK) + #define ICP_QAT_FW_COMN_STATUS_FLAG_OK 0 #define ICP_QAT_FW_COMN_STATUS_FLAG_ERROR 1 #define ICP_QAT_FW_COMN_STATUS_CMP_END_OF_LAST_BLK_FLAG_CLR 0 diff --git a/drivers/common/qat/qat_adf/icp_qat_fw_la.h b/drivers/common/qat/qat_adf/icp_qat_fw_la.h index 70f0effa62..f61241d12a 100644 --- a/drivers/common/qat/qat_adf/icp_qat_fw_la.h +++ b/drivers/common/qat/qat_adf/icp_qat_fw_la.h @@ -22,11 +22,18 @@ enum icp_qat_fw_la_cmd_id { ICP_QAT_FW_LA_CMD_DELIMITER = 18 }; +/* In GEN5 Command ID 4 corresponds to AEAD */ +#define ICP_QAT_FW_LA_CMD_AEAD 4 + #define ICP_QAT_FW_LA_ICV_VER_STATUS_PASS ICP_QAT_FW_COMN_STATUS_FLAG_OK #define ICP_QAT_FW_LA_ICV_VER_STATUS_FAIL ICP_QAT_FW_COMN_STATUS_FLAG_ERROR #define ICP_QAT_FW_LA_TRNG_STATUS_PASS ICP_QAT_FW_COMN_STATUS_FLAG_OK #define ICP_QAT_FW_LA_TRNG_STATUS_FAIL ICP_QAT_FW_COMN_STATUS_FLAG_ERROR +/* GEN5 Hash, HMAC and GCM Verification Status */ +#define ICP_QAT_FW_LA_VER_STATUS_FAIL ICP_QAT_FW_COMN_STATUS_FLAG_OK + + struct icp_qat_fw_la_bulk_req { struct icp_qat_fw_comn_req_hdr comn_hdr; struct icp_qat_fw_comn_req_hdr_cd_pars cd_pars; @@ -81,6 +88,18 @@ struct icp_qat_fw_la_bulk_req { #define ICP_QAT_FW_LA_PARTIAL_END 2 #define QAT_LA_PARTIAL_BITPOS 0 #define QAT_LA_PARTIAL_MASK 0x3 + +/* GEN5 specific Crypto Flags fields */ +#define ICP_QAT_FW_SYM_AEAD_ALGO_BITPOS 6 +#define ICP_QAT_FW_SYM_AEAD_ALGO_MASK 0x3 +#define ICP_QAT_FW_SYM_IV_SIZE_BITPOS 9 +#define ICP_QAT_FW_SYM_IV_SIZE_MASK 0x3 +#define ICP_QAT_FW_SYM_IV_IN_DESC_BITPOS 11 +#define ICP_QAT_FW_SYM_IV_IN_DESC_MASK 0x1 +#define ICP_QAT_FW_SYM_IV_IN_DESC_VALID 1 +#define ICP_QAT_FW_SYM_DIRECTION_BITPOS 15 +#define ICP_QAT_FW_SYM_DIRECTION_MASK 0x1 + #define ICP_QAT_FW_LA_FLAGS_BUILD(zuc_proto, gcm_iv_len, auth_rslt, proto, \ cmp_auth, ret_auth, update_state, \ ciph_iv, ciphcfg, partial) \ @@ -188,6 +207,23 @@ struct icp_qat_fw_la_bulk_req { QAT_FIELD_SET(flags, val, QAT_LA_PARTIAL_BITPOS, \ QAT_LA_PARTIAL_MASK) +/* GEN5 specific Crypto Flags operations */ +#define ICP_QAT_FW_SYM_AEAD_ALGO_SET(flags, val) \ + QAT_FIELD_SET(flags, val, ICP_QAT_FW_SYM_AEAD_ALGO_BITPOS, \ + ICP_QAT_FW_SYM_AEAD_ALGO_MASK) + +#define ICP_QAT_FW_SYM_IV_SIZE_SET(flags, val) \ + QAT_FIELD_SET(flags, val, ICP_QAT_FW_SYM_IV_SIZE_BITPOS, \ + ICP_QAT_FW_SYM_IV_SIZE_MASK) + +#define ICP_QAT_FW_SYM_IV_IN_DESC_FLAG_SET(flags, val) \ + QAT_FIELD_SET(flags, val, ICP_QAT_FW_SYM_IV_IN_DESC_BITPOS, \ + ICP_QAT_FW_SYM_IV_IN_DESC_MASK) + +#define ICP_QAT_FW_SYM_DIR_FLAG_SET(flags, val) \ + QAT_FIELD_SET(flags, val, ICP_QAT_FW_SYM_DIRECTION_BITPOS, \ + ICP_QAT_FW_SYM_DIRECTION_MASK) + #define QAT_FW_LA_MODE2 1 #define QAT_FW_LA_NO_MODE2 0 #define QAT_FW_LA_MODE2_MASK 0x1 @@ -410,4 +446,19 @@ struct icp_qat_fw_la_cipher_20_req_params { uint8_t spc_auth_res_sz; }; +struct icp_qat_fw_la_cipher_30_req_params { + uint32_t spc_aad_sz; + uint8_t cipher_length; + uint8_t reserved[2]; + uint8_t spc_auth_res_sz; + union { + uint32_t cipher_IV_array[ICP_QAT_FW_NUM_LONGWORDS_4]; + struct { + uint64_t cipher_IV_ptr; + uint64_t resrvd1; + } s; + + } u; +}; + #endif diff --git a/drivers/common/qat/qat_common.h b/drivers/common/qat/qat_common.h index 9411a79301..dc48a2e1ee 100644 --- a/drivers/common/qat/qat_common.h +++ b/drivers/common/qat/qat_common.h @@ -21,6 +21,7 @@ enum qat_device_gen { QAT_GEN2, QAT_GEN3, QAT_GEN4, + QAT_GEN5, QAT_N_GENS }; diff --git a/drivers/common/qat/qat_device.c b/drivers/common/qat/qat_device.c index f55dc3c6f0..d4f5391d12 100644 --- a/drivers/common/qat/qat_device.c +++ b/drivers/common/qat/qat_device.c @@ -62,6 +62,12 @@ static const struct rte_pci_id pci_id_qat_map[] = { { RTE_PCI_DEVICE(0x8086, 0x4945), }, + { + RTE_PCI_DEVICE(0x8086, 0x1454), /* GEN5: AVFs */ + }, + { + RTE_PCI_DEVICE(0x8086, 0x1456), /* GEN5: CPF */ + }, {.device_id = 0}, }; @@ -199,6 +205,9 @@ pick_gen(const struct rte_pci_device *pci_dev) case 0x4943: case 0x4945: return QAT_GEN4; + case 0x1454: /* QAT30: AVF */ + case 0x1456: /* QAT30: CPF-mdev */ + return QAT_GEN5; default: QAT_LOG(ERR, "Invalid dev_id, can't determine generation"); return QAT_N_GENS; From patchwork Wed Dec 20 13:26:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nayak, Nishikanta" X-Patchwork-Id: 135396 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2C0434374A; Wed, 20 Dec 2023 14:27:04 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B20A142E59; Wed, 20 Dec 2023 14:26:53 +0100 (CET) Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.126]) by mails.dpdk.org (Postfix) with ESMTP id 0013A4067C for ; Wed, 20 Dec 2023 14:26:51 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703078812; x=1734614812; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=03YcFYIUsrJaKolvBy58JUwOhAPZF98hANTXspdM27I=; b=SWNK4t2dxOKmmbdlrJQil6VHZJsgnboSg49AiaXrtl6trMrmAhJR0qW8 6TYMZCM4JiZg8YYbx1DL3pVWGOG/MZK5aZ0V6IxeeLjoVY0ArmPI4y1sx 7stYTnTOssLH0XISrjAQggRI+it7MvOJUK5QUwmTgQ3x7/QJbCZUFkSdN j+JAxCap4Dpvh4TgBn1WoE6SUAWug0nP05EuV5MeGPh7+ArN7p1Vm1MBY hRRacFEAJTamYYw+W3mvYwcPBErcfozRi07enQbwPQCesdqbvkVO1TY/I UYQ2w40zOBIqYtKm1NQo591oKdfqfxzm8AU09z6OoeG88W4Kay1VFKrdY w==; X-IronPort-AV: E=McAfee;i="6600,9927,10929"; a="380801533" X-IronPort-AV: E=Sophos;i="6.04,291,1695711600"; d="scan'208";a="380801533" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Dec 2023 05:26:51 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10929"; a="842276293" X-IronPort-AV: E=Sophos;i="6.04,291,1695711600"; d="scan'208";a="842276293" Received: from silpixa00400355.ir.intel.com (HELO silpixa00400355.ger.corp.intel.com) ([10.237.222.80]) by fmsmga008.fm.intel.com with ESMTP; 20 Dec 2023 05:26:49 -0800 From: Nishikant Nayak To: dev@dpdk.org Cc: kai.ji@intel.com, ciara.power@intel.com, arkadiuszx.kusztal@intel.com, Nishikant Nayak , Akhil Goyal , Fan Zhang Subject: [PATCH 3/4] crypto/qat: update headers for GEN5 support Date: Wed, 20 Dec 2023 13:26:15 +0000 Message-Id: <20231220132616.318983-3-nishikanta.nayak@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231220132616.318983-1-nishikanta.nayak@intel.com> References: <20231220132616.318983-1-nishikanta.nayak@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch handles the changes required for updating the common header fields specific to GEN5, Also added/updated of the response processing APIs based on GEN5 requirement. Signed-off-by: Nishikant Nayak --- drivers/crypto/qat/qat_sym.c | 10 ++++- drivers/crypto/qat/qat_sym.h | 60 +++++++++++++++++++++++++++- drivers/crypto/qat/qat_sym_session.c | 52 ++++++++++++++++++++++++ drivers/crypto/qat/qat_sym_session.h | 5 ++- lib/cryptodev/rte_crypto_sym.h | 3 ++ 5 files changed, 126 insertions(+), 4 deletions(-) diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c index 6e03bde841..8fbb8831ab 100644 --- a/drivers/crypto/qat/qat_sym.c +++ b/drivers/crypto/qat/qat_sym.c @@ -179,8 +179,14 @@ uint16_t qat_sym_dequeue_burst(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops) { - return qat_dequeue_op_burst(qp, (void **)ops, - qat_sym_process_response, nb_ops); + struct qat_qp *tmp_qp = (struct qat_qp *)qp; + + if (tmp_qp->qat_dev_gen == QAT_GEN5) + return qat_dequeue_op_burst(qp, (void **)ops, + qat_sym_process_response_gen5, nb_ops); + else + return qat_dequeue_op_burst(qp, (void **)ops, + qat_sym_process_response, nb_ops); } int diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h index 71e9d5f34b..7db21fc341 100644 --- a/drivers/crypto/qat/qat_sym.h +++ b/drivers/crypto/qat/qat_sym.h @@ -95,6 +95,12 @@ /* Maximum data length for single pass GMAC: 2^14-1 */ #define QAT_AES_GMAC_SPC_MAX_SIZE 16383 +/* Digest length for GCM Algo is 16 bytes */ +#define GCM_256_DIGEST_LEN 16 + +/* IV length for GCM algo is 12 bytes */ +#define GCM_IV_LENGTH 12 + struct qat_sym_session; struct qat_sym_sgl { @@ -383,6 +389,52 @@ qat_sym_process_response(void **op, uint8_t *resp, void *op_cookie, return 1; } +static __rte_always_inline int +qat_sym_process_response_gen5(void **op, uint8_t *resp, + void *op_cookie __rte_unused, + uint64_t *dequeue_err_count __rte_unused) +{ + struct icp_qat_fw_comn_resp *resp_msg = + (struct icp_qat_fw_comn_resp *)resp; + struct rte_crypto_op *rx_op = (struct rte_crypto_op *)(uintptr_t) + (resp_msg->opaque_data); + struct qat_sym_session *sess; + +#if RTE_LOG_DP_LEVEL >= RTE_LOG_DEBUG + QAT_DP_HEXDUMP_LOG(DEBUG, "qat_response:", (uint8_t *)resp_msg, + sizeof(struct icp_qat_fw_comn_resp)); +#endif + + sess = CRYPTODEV_GET_SYM_SESS_PRIV(rx_op->sym->session); + + rx_op->status = RTE_CRYPTO_OP_STATUS_SUCCESS; + + if (ICP_QAT_FW_COMN_STATUS_FLAG_OK != + ICP_QAT_FW_COMN_RESP_UNSUPPORTED_REQUEST_STAT_GET( + resp_msg->comn_hdr.comn_status)) + rx_op->status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED; + + else if (ICP_QAT_FW_COMN_STATUS_FLAG_OK != + ICP_QAT_FW_COMN_RESP_INVALID_PARAM_STAT_GET( + resp_msg->comn_hdr.comn_status)) + rx_op->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS; + + if (sess->qat_dir == ICP_QAT_HW_CIPHER_DECRYPT) { + if (ICP_QAT_FW_LA_VER_STATUS_FAIL == + ICP_QAT_FW_COMN_RESP_CRYPTO_STAT_GET( + resp_msg->comn_hdr.comn_status)) + rx_op->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED; + } + + *op = (void *)rx_op; + + /* + * return 1 as dequeue op only move on to the next op + * if one was ready to return to API + */ + return 1; +} + int qat_sym_configure_dp_ctx(struct rte_cryptodev *dev, uint16_t qp_id, struct rte_crypto_raw_dp_ctx *raw_dp_ctx, @@ -448,7 +500,13 @@ qat_sym_preprocess_requests(void **ops __rte_unused, static inline void qat_sym_process_response(void **op __rte_unused, uint8_t *resp __rte_unused, - void *op_cookie __rte_unused) + void *op_cookie __rte_unused, uint64_t *dequeue_err_count __rte_unused) +{ +} + +static inline void +qat_sym_process_response_gen5(void **op __rte_unused, uint8_t *resp __rte_unused, + void *op_cookie __rte_unused, uint64_t *dequeue_err_count __rte_unused) { } diff --git a/drivers/crypto/qat/qat_sym_session.c b/drivers/crypto/qat/qat_sym_session.c index 9f4f6c3d93..c97d6509b8 100644 --- a/drivers/crypto/qat/qat_sym_session.c +++ b/drivers/crypto/qat/qat_sym_session.c @@ -136,6 +136,9 @@ qat_sym_cd_auth_set(struct qat_sym_session *cdesc, static void qat_sym_session_init_common_hdr(struct qat_sym_session *session); +static void +qat_sym_session_init_gen5_hdr(struct qat_sym_session *session); + /* Req/cd init functions */ static void @@ -738,6 +741,12 @@ qat_sym_session_set_parameters(struct rte_cryptodev *dev, session->qat_cmd); return -ENOTSUP; } + + if (qat_dev_gen == QAT_GEN5) { + qat_sym_session_init_gen5_hdr(session); + return 0; + } + qat_sym_session_finalize(session); return qat_sym_gen_dev_ops[qat_dev_gen].set_session((void *)dev, @@ -1082,6 +1091,12 @@ qat_sym_session_configure_aead(struct rte_cryptodev *dev, if (qat_sym_cd_cipher_set(session, aead_xform->key.data, aead_xform->key.length)) return -EINVAL; + + if (qat_dev_gen == QAT_GEN5) { + session->auth_key_length = aead_xform->key.length; + memcpy(session->key_array, aead_xform->key.data, + aead_xform->key.length); + } } else if ((aead_xform->op == RTE_CRYPTO_AEAD_OP_ENCRYPT && aead_xform->algo == RTE_CRYPTO_AEAD_AES_GCM) || (aead_xform->op == RTE_CRYPTO_AEAD_OP_DECRYPT && @@ -1970,6 +1985,43 @@ qat_sym_session_init_common_hdr(struct qat_sym_session *session) ICP_QAT_FW_LA_NO_DIGEST_IN_BUFFER); } +static void +qat_sym_session_init_gen5_hdr(struct qat_sym_session *session) +{ + struct icp_qat_fw_la_bulk_req *req_tmpl = &session->fw_req; + struct icp_qat_fw_comn_req_hdr *header = &req_tmpl->comn_hdr; + + /* + * GEN5 specifies separate command id for AEAD operations but Cryptodev + * API processes AEAD operations as Single pass Crypto operations. + * Hence even for GEN5, Session Algo Command ID is CIPHER. + * Note, however Session Algo Mode is AEAD. + */ + header->service_cmd_id = ICP_QAT_FW_LA_CMD_AEAD; + header->service_type = ICP_QAT_FW_COMN_REQ_CPM_FW_LA; + header->hdr_flags = + ICP_QAT_FW_COMN_HDR_FLAGS_BUILD_GEN5(ICP_QAT_FW_COMN_REQ_FLAG_SET, + ICP_QAT_FW_COMN_GEN5_DESC_LAYOUT); + header->comn_req_flags = + ICP_QAT_FW_COMN_FLAGS_BUILD_GEN5(QAT_COMN_PTR_TYPE_SGL, + QAT_COMN_KEY_BUFFER_USED); + + ICP_QAT_FW_SYM_AEAD_ALGO_SET(header->serv_specif_flags, + RTE_CRYPTO_AEAD_AES_GCM_GEN5); + ICP_QAT_FW_SYM_IV_SIZE_SET(header->serv_specif_flags, + ICP_QAT_FW_LA_GCM_IV_LEN_12_OCTETS); + ICP_QAT_FW_SYM_IV_IN_DESC_FLAG_SET(header->serv_specif_flags, + ICP_QAT_FW_SYM_IV_IN_DESC_VALID); + + if (session->qat_dir == ICP_QAT_HW_CIPHER_DECRYPT) { + ICP_QAT_FW_SYM_DIR_FLAG_SET(header->serv_specif_flags, + ICP_QAT_HW_CIPHER_DECRYPT); + } else { + ICP_QAT_FW_SYM_DIR_FLAG_SET(header->serv_specif_flags, + ICP_QAT_HW_CIPHER_ENCRYPT); + } +} + int qat_sym_cd_cipher_set(struct qat_sym_session *cdesc, const uint8_t *cipherkey, uint32_t cipherkeylen) diff --git a/drivers/crypto/qat/qat_sym_session.h b/drivers/crypto/qat/qat_sym_session.h index 9209e2e8df..821c53dfbb 100644 --- a/drivers/crypto/qat/qat_sym_session.h +++ b/drivers/crypto/qat/qat_sym_session.h @@ -111,7 +111,10 @@ struct qat_sym_session { enum icp_qat_hw_auth_op auth_op; enum icp_qat_hw_auth_mode auth_mode; void *bpi_ctx; - struct qat_sym_cd cd; + union { + struct qat_sym_cd cd; + uint8_t key_array[32]; + }; uint8_t prefix_state[QAT_PREFIX_TBL_SIZE] __rte_cache_aligned; uint8_t *cd_cur_ptr; phys_addr_t cd_paddr; diff --git a/lib/cryptodev/rte_crypto_sym.h b/lib/cryptodev/rte_crypto_sym.h index 53b18b9412..e545b1ba76 100644 --- a/lib/cryptodev/rte_crypto_sym.h +++ b/lib/cryptodev/rte_crypto_sym.h @@ -492,6 +492,9 @@ enum rte_crypto_aead_operation { /**< Verify digest and decrypt */ }; +/* In GEN5 AEAD AES GCM Algorithm has ID 0 */ +#define RTE_CRYPTO_AEAD_AES_GCM_GEN5 0 + /** Authentication operation name strings */ extern const char * rte_crypto_aead_operation_strings[]; From patchwork Wed Dec 20 13:26:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nayak, Nishikanta" X-Patchwork-Id: 135397 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 00F424374A; Wed, 20 Dec 2023 14:27:11 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D2B7040E01; Wed, 20 Dec 2023 14:26:56 +0100 (CET) Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.126]) by mails.dpdk.org (Postfix) with ESMTP id 5BB9A40A8A for ; Wed, 20 Dec 2023 14:26:55 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703078815; x=1734614815; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=WIZL40qoYrTmeUCTKwRxgI9sQI3SSPi393kCtYyD5Ao=; b=eQxECUtG7G+qUw4FVANNFiWb7pbJYHxGPdoEFa3xpw9h3YrDoiZS/Zxp LSk3OHAY9oDZD7wgkFG8yL0xVzjkXgo1+shO6ApwjJnBYYErLmryggf/P smfRvQ2TeR9KNaywbgLTJZCLIZvkKazf8mtwy8cZdVyOE32Hu0xpIgdB8 6CLFa0s0iL7K1NSZAddFSQzH3jKwb5Gn7feuaE4dxLC4+gyOD+rvvTHC4 YfXeMXTUjKc69m+yrSIy7hpL7b84DshPBjUElC1sgJQGAWzXNj5Uhp0op Af8DFMkbIYR6j0ocqLekCWTtUgzqAyxckwECsIUtZezi4zjdN4y/zkWwJ Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10929"; a="380801541" X-IronPort-AV: E=Sophos;i="6.04,291,1695711600"; d="scan'208";a="380801541" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Dec 2023 05:26:54 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10929"; a="842276314" X-IronPort-AV: E=Sophos;i="6.04,291,1695711600"; d="scan'208";a="842276314" Received: from silpixa00400355.ir.intel.com (HELO silpixa00400355.ger.corp.intel.com) ([10.237.222.80]) by fmsmga008.fm.intel.com with ESMTP; 20 Dec 2023 05:26:51 -0800 From: Nishikant Nayak To: dev@dpdk.org Cc: kai.ji@intel.com, ciara.power@intel.com, arkadiuszx.kusztal@intel.com, Nishikant Nayak , Akhil Goyal , Fan Zhang Subject: [PATCH 4/4] test/cryptodev: add tests for GCM with AAD Date: Wed, 20 Dec 2023 13:26:16 +0000 Message-Id: <20231220132616.318983-4-nishikanta.nayak@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231220132616.318983-1-nishikanta.nayak@intel.com> References: <20231220132616.318983-1-nishikanta.nayak@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Adding one new unit test code for validating the features added as part of GCM with 64 byte AAD. The new test case adds one new test for GCM algo for both encrypt and decrypt operations. Signed-off-by: Nishikant Nayak --- app/test/test_cryptodev.c | 48 +++++++++++++--- app/test/test_cryptodev_aead_test_vectors.h | 62 +++++++++++++++++++++ 2 files changed, 103 insertions(+), 7 deletions(-) diff --git a/app/test/test_cryptodev.c b/app/test/test_cryptodev.c index 58561ededf..73581db8cd 100644 --- a/app/test/test_cryptodev.c +++ b/app/test/test_cryptodev.c @@ -12156,6 +12156,18 @@ test_AES_GCM_auth_decryption_test_case_256_7(void) return test_authenticated_decryption(&gcm_test_case_256_7); } +static int +test_AES_GCM_auth_decryption_test_case_256_8(void) +{ + return test_authenticated_decryption(&gcm_test_case_256_8); +} + +static int +test_AES_GCM_auth_encryption_test_case_256_8(void) +{ + return test_authenticated_encryption(&gcm_test_case_256_8); +} + static int test_AES_GCM_auth_decryption_test_case_aad_1(void) { @@ -12275,10 +12287,16 @@ test_authenticated_encryption_oop(const struct aead_test_data *tdata) /* Verify the capabilities */ struct rte_cryptodev_sym_capability_idx cap_idx; + const struct rte_cryptodev_symmetric_capability *capability; cap_idx.type = RTE_CRYPTO_SYM_XFORM_AEAD; cap_idx.algo.aead = tdata->algo; - if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0], - &cap_idx) == NULL) + capability = rte_cryptodev_sym_capability_get(ts_params->valid_devs[0], + &cap_idx); + if (capability == NULL) + return TEST_SKIPPED; + if (rte_cryptodev_sym_capability_check_aead( + capability, tdata->key.len, tdata->auth_tag.len, + tdata->aad.len, tdata->iv.len)) return TEST_SKIPPED; rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info); @@ -12381,16 +12399,22 @@ test_authenticated_decryption_oop(const struct aead_test_data *tdata) /* Verify the capabilities */ struct rte_cryptodev_sym_capability_idx cap_idx; + const struct rte_cryptodev_symmetric_capability *capability; cap_idx.type = RTE_CRYPTO_SYM_XFORM_AEAD; cap_idx.algo.aead = tdata->algo; - if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0], - &cap_idx) == NULL) - return TEST_SKIPPED; + capability = rte_cryptodev_sym_capability_get(ts_params->valid_devs[0], + &cap_idx); /* not supported with CPU crypto and raw data-path APIs*/ if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO || global_api_test_type == CRYPTODEV_RAW_API_TEST) return TEST_SKIPPED; + if (capability == NULL) + return TEST_SKIPPED; + if (rte_cryptodev_sym_capability_check_aead( + capability, tdata->key.len, tdata->auth_tag.len, + tdata->aad.len, tdata->iv.len)) + return TEST_SKIPPED; if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) && (!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) { @@ -15411,10 +15435,16 @@ test_authenticated_encryption_SGL(const struct aead_test_data *tdata, /* Verify the capabilities */ struct rte_cryptodev_sym_capability_idx cap_idx; + const struct rte_cryptodev_symmetric_capability *capability; cap_idx.type = RTE_CRYPTO_SYM_XFORM_AEAD; cap_idx.algo.aead = tdata->algo; - if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0], - &cap_idx) == NULL) + capability = rte_cryptodev_sym_capability_get(ts_params->valid_devs[0], + &cap_idx); + if (capability == NULL) + return TEST_SKIPPED; + if (rte_cryptodev_sym_capability_check_aead( + capability, tdata->key.len, tdata->auth_tag.len, + tdata->aad.len, tdata->iv.len)) return TEST_SKIPPED; /* @@ -16853,6 +16883,8 @@ static struct unit_test_suite cryptodev_aes_gcm_auth_testsuite = { test_AES_GCM_auth_encryption_test_case_256_6), TEST_CASE_ST(ut_setup, ut_teardown, test_AES_GCM_auth_encryption_test_case_256_7), + TEST_CASE_ST(ut_setup, ut_teardown, + test_AES_GCM_auth_encryption_test_case_256_8), /** AES GCM Authenticated Decryption 256 bits key */ TEST_CASE_ST(ut_setup, ut_teardown, @@ -16869,6 +16901,8 @@ static struct unit_test_suite cryptodev_aes_gcm_auth_testsuite = { test_AES_GCM_auth_decryption_test_case_256_6), TEST_CASE_ST(ut_setup, ut_teardown, test_AES_GCM_auth_decryption_test_case_256_7), + TEST_CASE_ST(ut_setup, ut_teardown, + test_AES_GCM_auth_decryption_test_case_256_8), /** AES GCM Authenticated Encryption big aad size */ TEST_CASE_ST(ut_setup, ut_teardown, diff --git a/app/test/test_cryptodev_aead_test_vectors.h b/app/test/test_cryptodev_aead_test_vectors.h index 07292620a4..eadf206e4d 100644 --- a/app/test/test_cryptodev_aead_test_vectors.h +++ b/app/test/test_cryptodev_aead_test_vectors.h @@ -17,6 +17,16 @@ static uint8_t gcm_aad_text[MAX_AAD_LENGTH] = { 0x00, 0xf1, 0xe2, 0xd3, 0xc4, 0xb5, 0xa6, 0x97, 0x88, 0x79, 0x6a, 0x5b, 0x4c, 0x3d, 0x2e, 0x1f }; +static uint8_t gcm_aad_64B_text[MAX_AAD_LENGTH] = { + 0xED, 0x3E, 0xA8, 0x1F, 0x74, 0xE5, 0xD1, 0x96, + 0xA4, 0xD5, 0x4B, 0x26, 0xBB, 0x20, 0x61, 0x7B, + 0x3B, 0x9C, 0x2A, 0x69, 0x90, 0xEF, 0xD7, 0x9A, + 0x94, 0xC2, 0xF5, 0x86, 0xBD, 0x00, 0xF6, 0xEA, + 0x0B, 0x14, 0x24, 0xF2, 0x08, 0x67, 0x42, 0x3A, + 0xB5, 0xB8, 0x32, 0x97, 0xB5, 0x99, 0x69, 0x75, + 0x60, 0x00, 0x8F, 0xF7, 0x6F, 0x16, 0x52, 0x66, + 0xF1, 0xA9, 0x38, 0xFD, 0xB0, 0x61, 0x60, 0xB5 }; + static uint8_t ccm_aad_test_1[8] = { 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07 }; @@ -1736,6 +1746,58 @@ static const struct aead_test_data gcm_test_case_256_7 = { } }; +static const struct aead_test_data gcm_test_case_256_8 = { + .algo = RTE_CRYPTO_AEAD_AES_GCM, + .key = { + .data = { + 0xD8, 0xFD, 0x8F, 0x5A, 0x13, 0x7B, 0x05, 0x2C, + 0xA4, 0x64, 0x7A, 0xDD, 0x1E, 0x9A, 0x68, 0x33, + 0x04, 0x70, 0xE8, 0x1E, 0x42, 0x84, 0x64, 0xD2, + 0x23, 0xA1, 0x6A, 0x0A, 0x05, 0x7B, 0x90, 0xDE}, + .len = 32 + }, + .iv = { + .data = { + 0x8D, 0xDF, 0xB8, 0x7F, 0xD0, 0x79, 0x77, 0x55, + 0xD5, 0x48, 0x03, 0x05}, + .len = 12 + }, + .aad = { + .data = gcm_aad_64B_text, + .len = 64 + }, + .plaintext = { + .data = { + 0x4D, 0xBC, 0x2C, 0x7F, 0x25, 0x1F, 0x07, 0x25, + 0x54, 0x8C, 0x43, 0xDB, 0xD8, 0x06, 0x9F, 0xBF, + 0xCA, 0x60, 0xF4, 0xEF, 0x13, 0x87, 0xE8, 0x2F, + 0x4D, 0x9D, 0x1D, 0x87, 0x9F, 0x91, 0x79, 0x7E, + 0x3E, 0x98, 0xA3, 0x63, 0xC6, 0xFE, 0xDB, 0x35, + 0x96, 0x59, 0xB2, 0x0C, 0x80, 0x96, 0x70, 0x07, + 0x87, 0x42, 0xAB, 0x4F, 0x31, 0x73, 0xC4, 0xF9, + 0xB0, 0x1E, 0xF1, 0xBC, 0x7D, 0x45, 0xE5, 0xF3}, + .len = 64 + }, + .ciphertext = { + .data = { + 0x21, 0xFA, 0x59, 0x4F, 0x1F, 0x6B, 0x19, 0xC2, + 0x68, 0xBC, 0x05, 0x93, 0x4E, 0x48, 0x6C, 0x5B, + 0x0B, 0x7A, 0x43, 0xB7, 0x60, 0x8E, 0x00, 0xC4, + 0xAB, 0x14, 0x6B, 0xCC, 0xA1, 0x27, 0x6A, 0xDE, + 0x8E, 0xB6, 0x98, 0xBB, 0x4F, 0xD0, 0x6F, 0x30, + 0x0F, 0x04, 0xA8, 0x5B, 0xDC, 0xD8, 0xE8, 0x8A, + 0x73, 0xD9, 0xB8, 0x60, 0x7C, 0xE4, 0x32, 0x4C, + 0x3A, 0x0B, 0xC2, 0x82, 0xDA, 0x88, 0x17, 0x69}, + .len = 64 + }, + .auth_tag = { + .data = { + 0x3B, 0x80, 0x83, 0x72, 0xE5, 0x1B, 0x94, 0x15, + 0x75, 0xC8, 0x62, 0xBC, 0xA1, 0x66, 0x91, 0x45}, + .len = 16 + } +}; + /** variable AAD AES-GCM-128 Test Vectors */ static const struct aead_test_data gcm_test_case_aad_1 = { .algo = RTE_CRYPTO_AEAD_AES_GCM,