From patchwork Sun Dec 8 11:54:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 63626 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B5B59A04F1; Sun, 8 Dec 2019 12:55:43 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 5BEB01BC25; Sun, 8 Dec 2019 12:55:39 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 616B21B9B5 for ; Sun, 8 Dec 2019 12:55:38 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB8Bp9Fm006219; Sun, 8 Dec 2019 03:55:37 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=ZzY3S/QTotqInCHyWcBWdcTD7JEH2pT7NBq75ffP+8Q=; b=LjB0EPAANMg+08TdTrE28EN7xH4AadmF3rRzlGSKUov2M7PMKzZAOxpN12u/DZeFGDV/ WyVWV98ciQ9Ajug4G3LQvZq20THVtL6eY0Fm8hHj6AZ1rHyZISNQFSO5hBUkWuuXs4aw 2ZyCv9EfvVS5T+TZ/2gqOH7y7xrKyzorWQg104d9GJBtZIu7XVtxnnZ41lVIeylnniIZ /GeMAl14IlYLbCkhwZ9wIuTJRJLgLnfasW9P6CUGSaGLfCAMZ4QZW7PK25CJ5/Eox+Jj 6BA1YKh0zXP7/GLUMA5rr9IFaXBIS/3eI//VEi1rS17O0HU6RZcgHRzw6T+lUPBy8Qkg KQ== Received: from sc-exch01.marvell.com ([199.233.58.181]) by mx0a-0016f401.pphosted.com with ESMTP id 2wrbawjj1k-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 08 Dec 2019 03:55:37 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Sun, 8 Dec 2019 03:55:35 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Sun, 8 Dec 2019 03:55:35 -0800 Received: from ajoseph83.caveonetworks.com.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id 9C3623F703F; Sun, 8 Dec 2019 03:55:30 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Declan Doherty , Thomas Monjalon CC: Anoob Joseph , Jerin Jacob , Narayana Prasad , Kiran Kumar K , Nithin Dabilpuram , "Pavan Nikhilesh" , Ankur Dwivedi , Archana Muniganti , Tejasree Kondoj , Vamsi Attunuru , "Lukasz Bartosik" , , Tejasree Kondoj Date: Sun, 8 Dec 2019 17:24:40 +0530 Message-ID: <1575806094-28391-2-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1575806094-28391-1-git-send-email-anoobj@marvell.com> References: <1575806094-28391-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-08_03:2019-12-05,2019-12-08 signatures=0 Subject: [dpdk-dev] [PATCH 01/15] common/octeontx2: add CPT LF mbox for inline inbound X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Adding the new mbox introduced to configure CPT LF to be used for inline inbound. Signed-off-by: Anoob Joseph Signed-off-by: Tejasree Kondoj Acked-by: Jerin Jacob --- drivers/common/octeontx2/otx2_mbox.h | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/common/octeontx2/otx2_mbox.h b/drivers/common/octeontx2/otx2_mbox.h index e0e4e2f..70452d1 100644 --- a/drivers/common/octeontx2/otx2_mbox.h +++ b/drivers/common/octeontx2/otx2_mbox.h @@ -193,6 +193,8 @@ M(CPT_SET_CRYPTO_GRP, 0xA03, cpt_set_crypto_grp, \ msg_rsp) \ M(CPT_INLINE_IPSEC_CFG, 0xA04, cpt_inline_ipsec_cfg, \ cpt_inline_ipsec_cfg_msg, msg_rsp) \ +M(CPT_RX_INLINE_LF_CFG, 0xBFE, cpt_rx_inline_lf_cfg, \ + cpt_rx_inline_lf_cfg_msg, msg_rsp) \ /* NPC mbox IDs (range 0x6000 - 0x7FFF) */ \ M(NPC_MCAM_ALLOC_ENTRY, 0x6000, npc_mcam_alloc_entry, \ npc_mcam_alloc_entry_req, \ @@ -1202,6 +1204,11 @@ struct cpt_inline_ipsec_cfg_msg { uint16_t __otx2_io nix_pf_func; /* Outbound path NIX_PF_FUNC */ }; +struct cpt_rx_inline_lf_cfg_msg { + struct mbox_msghdr hdr; + uint16_t __otx2_io sso_pf_func; +}; + /* NPC mbox message structs */ #define NPC_MCAM_ENTRY_INVALID 0xFFFF From patchwork Sun Dec 8 11:54:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 63627 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7844FA04F1; Sun, 8 Dec 2019 12:55:52 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 01EE61BE83; Sun, 8 Dec 2019 12:55:46 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 8CB841BE83 for ; Sun, 8 Dec 2019 12:55:44 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB8BpXxH006265; Sun, 8 Dec 2019 03:55:43 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=8ub6eZ/SYyu2YScDsBBFEG4ZESKG1hp9f1Q6PDcURvM=; b=O8ENEJKqdq3dzteIc9AwVnOQFgadQ3AUFS5Dl4cv9ej72CwoTO/I0ZjPfZeal6fVR1v8 iUH0Wcgbul5UHiUpOShjiz4/hYaTXfue2i5W/P6N2YpHPj8B+7KQLU8gf16se/nxoRXV 92Q58hRGfGDarYSS+6hi09E+b8RB0jN5ymxwOD0KWgmY+Ux+/jgZoyHiaMwaZx5RYhzu f9M0+imAkyqPbFpcX7QfUKVmoAeOnStHYMdBciBMQDQ4w3butmA1/XuJQnxNwu86dG/j P5xvgvLXd2bsWwhnvAAHia4MxTp6/8KCcCmQl9n//mk8/1ObBgo5t7AHDI/581ldKM7n PA== Received: from sc-exch03.marvell.com ([199.233.58.183]) by mx0a-0016f401.pphosted.com with ESMTP id 2wrbawjj1v-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 08 Dec 2019 03:55:43 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Sun, 8 Dec 2019 03:55:42 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Sun, 8 Dec 2019 03:55:42 -0800 Received: from ajoseph83.caveonetworks.com.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id 8E2533F703F; Sun, 8 Dec 2019 03:55:37 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Declan Doherty , Thomas Monjalon CC: Vamsi Attunuru , Jerin Jacob , Narayana Prasad , Kiran Kumar K , Nithin Dabilpuram , "Pavan Nikhilesh" , Ankur Dwivedi , Anoob Joseph , Archana Muniganti , Tejasree Kondoj , Lukasz Bartosik , Date: Sun, 8 Dec 2019 17:24:41 +0530 Message-ID: <1575806094-28391-3-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1575806094-28391-1-git-send-email-anoobj@marvell.com> References: <1575806094-28391-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-08_03:2019-12-05,2019-12-08 signatures=0 Subject: [dpdk-dev] [PATCH 02/15] common/octeontx2: add routine to check if rte_eth_dev belongs to otx2 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Vamsi Attunuru This routine returns true if given rte_eth_dev belongs to octeontx2 Signed-off-by: Anoob Joseph Signed-off-by: Tejasree Kondoj Signed-off-by: Vamsi Attunuru --- drivers/common/octeontx2/otx2_common.c | 20 ++++++++++++++++++++ drivers/common/octeontx2/otx2_common.h | 2 ++ .../octeontx2/rte_common_octeontx2_version.map | 1 + 3 files changed, 23 insertions(+) diff --git a/drivers/common/octeontx2/otx2_common.c b/drivers/common/octeontx2/otx2_common.c index 7e45366..116db0f 100644 --- a/drivers/common/octeontx2/otx2_common.c +++ b/drivers/common/octeontx2/otx2_common.c @@ -3,6 +3,7 @@ */ #include +#include #include #include @@ -23,6 +24,25 @@ otx2_npa_set_defaults(struct otx2_idev_cfg *idev) /** * @internal + * Check if rte_eth_dev is otx2_eth_dev + */ +uint8_t +otx2_is_ethdev(struct rte_eth_dev *eth_dev) +{ + struct rte_pci_device *pci_dev; + + pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev); + + if (pci_dev->id.device_id == PCI_DEVID_OCTEONTX2_RVU_PF || + pci_dev->id.device_id == PCI_DEVID_OCTEONTX2_RVU_VF || + pci_dev->id.device_id == PCI_DEVID_OCTEONTX2_RVU_AF_VF) + return 1; + + return 0; +} + +/** + * @internal * Get intra device config structure. */ struct otx2_idev_cfg * diff --git a/drivers/common/octeontx2/otx2_common.h b/drivers/common/octeontx2/otx2_common.h index f62c45d..d32e59a 100644 --- a/drivers/common/octeontx2/otx2_common.h +++ b/drivers/common/octeontx2/otx2_common.h @@ -8,6 +8,7 @@ #include #include #include +#include #include #include #include @@ -70,6 +71,7 @@ struct otx2_idev_cfg *otx2_intra_dev_get_cfg(void); void otx2_sso_pf_func_set(uint16_t sso_pf_func); uint16_t otx2_sso_pf_func_get(void); uint16_t otx2_npa_pf_func_get(void); +uint8_t otx2_is_ethdev(struct rte_eth_dev *eth_dev); struct otx2_npa_lf *otx2_npa_lf_obj_get(void); void otx2_npa_set_defaults(struct otx2_idev_cfg *idev); int otx2_npa_lf_active(void *dev); diff --git a/drivers/common/octeontx2/rte_common_octeontx2_version.map b/drivers/common/octeontx2/rte_common_octeontx2_version.map index adad21a..dac2283 100644 --- a/drivers/common/octeontx2/rte_common_octeontx2_version.map +++ b/drivers/common/octeontx2/rte_common_octeontx2_version.map @@ -6,6 +6,7 @@ DPDK_20.0 { otx2_dev_priv_init; otx2_disable_irqs; otx2_intra_dev_get_cfg; + otx2_is_ethdev; otx2_logtype_base; otx2_logtype_dpi; otx2_logtype_mbox; From patchwork Sun Dec 8 11:54:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 63628 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 566FEA04F1; Sun, 8 Dec 2019 12:56:03 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C70AA1BECF; Sun, 8 Dec 2019 12:55:53 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 723FD1BF6D for ; Sun, 8 Dec 2019 12:55:52 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB8Bqi8h006520; Sun, 8 Dec 2019 03:55:51 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=hy3ud0K3hF2W+ZjEUviFOPRwmPPunlzJuelA+FxWlfE=; b=dBgGRGvfQiJSFL7/H1rBFKLt7bJ4EARjDgXxdjMh6K3C2yGxgPX5diYa6tVXTq2CFTXJ nSSx/U1v5DKtPe07jt7Lj/JnZADRU6yusb0dvl2qsIL/7KZJ+Vk+cPK4NAJ85HxrIb4X IS3cE8KpeZyyEsj41RuahZm4Q+xI0zsA89A456NqWwrSH+hJJ9yNxKXwkQW4m77LQMLz WziO7ONNm6cIWyRj4CifsmrQvzWP80rDAEYnljmHAI95TI2EXDwaTqnjeyw2UeKT7H2i kt8dpK3bbKX4n+QaDacAeMHKuNLU+/CMJX4PT21FDKiyq1CDaC3Qodp2wGSSMBlB+jsR Ug== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0b-0016f401.pphosted.com with ESMTP id 2wrcfptc07-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 08 Dec 2019 03:55:51 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Sun, 8 Dec 2019 03:55:48 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Sun, 8 Dec 2019 03:55:48 -0800 Received: from ajoseph83.caveonetworks.com.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id E5E663F703F; Sun, 8 Dec 2019 03:55:43 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Declan Doherty , Thomas Monjalon CC: Tejasree Kondoj , Jerin Jacob , Narayana Prasad , Kiran Kumar K , Nithin Dabilpuram , "Pavan Nikhilesh" , Ankur Dwivedi , Anoob Joseph , Archana Muniganti , Vamsi Attunuru , Lukasz Bartosik , Date: Sun, 8 Dec 2019 17:24:42 +0530 Message-ID: <1575806094-28391-4-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1575806094-28391-1-git-send-email-anoobj@marvell.com> References: <1575806094-28391-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-08_03:2019-12-05,2019-12-08 signatures=0 Subject: [dpdk-dev] [PATCH 03/15] crypto/octeontx2: configure for inline IPsec X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Tejasree Kondoj For enabling outbound inline IPsec, a CPT queue needs to be tied to a NIX PF_FUNC. Distribute CPT queues fairly among all availble otx2 eth ports. For inbound, one CPT LF will be assigned and initialized by kernel. Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Archana Muniganti Signed-off-by: Tejasree Kondoj Signed-off-by: Vamsi Attunuru --- drivers/crypto/octeontx2/Makefile | 3 +- drivers/crypto/octeontx2/meson.build | 2 + drivers/crypto/octeontx2/otx2_cryptodev_mbox.c | 53 ++++++++++++++++++++++++++ drivers/crypto/octeontx2/otx2_cryptodev_mbox.h | 7 ++++ drivers/crypto/octeontx2/otx2_cryptodev_ops.c | 38 ++++++++++++++++++ 5 files changed, 102 insertions(+), 1 deletion(-) diff --git a/drivers/crypto/octeontx2/Makefile b/drivers/crypto/octeontx2/Makefile index f7d6c37..3ba67ed 100644 --- a/drivers/crypto/octeontx2/Makefile +++ b/drivers/crypto/octeontx2/Makefile @@ -10,7 +10,7 @@ LIB = librte_pmd_octeontx2_crypto.a # build flags CFLAGS += $(WERROR_FLAGS) -LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring +LDLIBS += -lrte_eal -lrte_ethdev -lrte_mbuf -lrte_mempool -lrte_ring LDLIBS += -lrte_cryptodev LDLIBS += -lrte_pci -lrte_bus_pci LDLIBS += -lrte_common_cpt -lrte_common_octeontx2 @@ -21,6 +21,7 @@ CFLAGS += -O3 CFLAGS += -I$(RTE_SDK)/drivers/common/cpt CFLAGS += -I$(RTE_SDK)/drivers/common/octeontx2 CFLAGS += -I$(RTE_SDK)/drivers/mempool/octeontx2 +CFLAGS += -I$(RTE_SDK)/drivers/net/octeontx2 CFLAGS += -DALLOW_EXPERIMENTAL_API ifneq ($(CONFIG_RTE_ARCH_64),y) diff --git a/drivers/crypto/octeontx2/meson.build b/drivers/crypto/octeontx2/meson.build index b6e5b73..67deca3 100644 --- a/drivers/crypto/octeontx2/meson.build +++ b/drivers/crypto/octeontx2/meson.build @@ -8,6 +8,7 @@ endif deps += ['bus_pci'] deps += ['common_cpt'] deps += ['common_octeontx2'] +deps += ['ethdev'] name = 'octeontx2_crypto' allow_experimental_apis = true @@ -32,3 +33,4 @@ endforeach includes += include_directories('../../common/cpt') includes += include_directories('../../common/octeontx2') includes += include_directories('../../mempool/octeontx2') +includes += include_directories('../../net/octeontx2') diff --git a/drivers/crypto/octeontx2/otx2_cryptodev_mbox.c b/drivers/crypto/octeontx2/otx2_cryptodev_mbox.c index b54e407..3183210 100644 --- a/drivers/crypto/octeontx2/otx2_cryptodev_mbox.c +++ b/drivers/crypto/octeontx2/otx2_cryptodev_mbox.c @@ -2,10 +2,13 @@ * Copyright (C) 2019 Marvell International Ltd. */ #include +#include #include "otx2_cryptodev.h" +#include "otx2_cryptodev_hw_access.h" #include "otx2_cryptodev_mbox.h" #include "otx2_dev.h" +#include "otx2_ethdev.h" #include "otx2_mbox.h" #include "cpt_pmd_logs.h" @@ -173,3 +176,53 @@ otx2_cpt_af_reg_write(const struct rte_cryptodev *dev, uint64_t reg, return otx2_cpt_send_mbox_msg(vf); } + +int +otx2_cpt_inline_init(const struct rte_cryptodev *dev) +{ + struct otx2_cpt_vf *vf = dev->data->dev_private; + struct otx2_mbox *mbox = vf->otx2_dev.mbox; + struct cpt_rx_inline_lf_cfg_msg *msg; + int ret; + + msg = otx2_mbox_alloc_msg_cpt_rx_inline_lf_cfg(mbox); + msg->sso_pf_func = otx2_sso_pf_func_get(); + + otx2_mbox_msg_send(mbox, 0); + ret = otx2_mbox_process(mbox); + if (ret < 0) + return -EIO; + + return 0; +} + +int +otx2_cpt_qp_ethdev_bind(const struct rte_cryptodev *dev, struct otx2_cpt_qp *qp, + uint16_t port_id) +{ + struct rte_eth_dev *eth_dev = &rte_eth_devices[port_id]; + struct otx2_cpt_vf *vf = dev->data->dev_private; + struct otx2_mbox *mbox = vf->otx2_dev.mbox; + struct cpt_inline_ipsec_cfg_msg *msg; + struct otx2_eth_dev *otx2_eth_dev; + int ret; + + if (!otx2_is_ethdev(&rte_eth_devices[port_id])) + return -EINVAL; + + otx2_eth_dev = otx2_eth_pmd_priv(eth_dev); + + msg = otx2_mbox_alloc_msg_cpt_inline_ipsec_cfg(mbox); + msg->dir = CPT_INLINE_OUTBOUND; + msg->enable = 1; + msg->slot = qp->id; + + msg->nix_pf_func = otx2_eth_dev->pf_func; + + otx2_mbox_msg_send(mbox, 0); + ret = otx2_mbox_process(mbox); + if (ret < 0) + return -EIO; + + return 0; +} diff --git a/drivers/crypto/octeontx2/otx2_cryptodev_mbox.h b/drivers/crypto/octeontx2/otx2_cryptodev_mbox.h index a298718..ae66b08 100644 --- a/drivers/crypto/octeontx2/otx2_cryptodev_mbox.h +++ b/drivers/crypto/octeontx2/otx2_cryptodev_mbox.h @@ -7,6 +7,8 @@ #include +#include "otx2_cryptodev_hw_access.h" + int otx2_cpt_available_queues_get(const struct rte_cryptodev *dev, uint16_t *nb_queues); @@ -22,4 +24,9 @@ int otx2_cpt_af_reg_read(const struct rte_cryptodev *dev, uint64_t reg, int otx2_cpt_af_reg_write(const struct rte_cryptodev *dev, uint64_t reg, uint64_t val); +int otx2_cpt_qp_ethdev_bind(const struct rte_cryptodev *dev, + struct otx2_cpt_qp *qp, uint16_t port_id); + +int otx2_cpt_inline_init(const struct rte_cryptodev *dev); + #endif /* _OTX2_CRYPTODEV_MBOX_H_ */ diff --git a/drivers/crypto/octeontx2/otx2_cryptodev_ops.c b/drivers/crypto/octeontx2/otx2_cryptodev_ops.c index 65101b0..a1213ca 100644 --- a/drivers/crypto/octeontx2/otx2_cryptodev_ops.c +++ b/drivers/crypto/octeontx2/otx2_cryptodev_ops.c @@ -6,6 +6,7 @@ #include #include +#include #include "otx2_cryptodev.h" #include "otx2_cryptodev_capabilities.h" @@ -125,6 +126,29 @@ otx2_cpt_metabuf_mempool_destroy(struct otx2_cpt_qp *qp) meta_info->sg_mlen = 0; } +static int +otx2_cpt_qp_inline_cfg(const struct rte_cryptodev *dev, struct otx2_cpt_qp *qp) +{ + static rte_atomic16_t port_offset = RTE_ATOMIC16_INIT(-1); + uint16_t port_id, nb_ethport = rte_eth_dev_count_avail(); + int i, ret; + + for (i = 0; i < nb_ethport; i++) { + port_id = rte_atomic16_add_return(&port_offset, 1) % nb_ethport; + if (otx2_is_ethdev(&rte_eth_devices[port_id])) + break; + } + + if (i >= nb_ethport) + return 0; + + ret = otx2_cpt_qp_ethdev_bind(dev, qp, port_id); + if (ret) + return ret; + + return 0; +} + static struct otx2_cpt_qp * otx2_cpt_qp_create(const struct rte_cryptodev *dev, uint16_t qp_id, uint8_t group) @@ -218,6 +242,12 @@ otx2_cpt_qp_create(const struct rte_cryptodev *dev, uint16_t qp_id, otx2_cpt_iq_disable(qp); + ret = otx2_cpt_qp_inline_cfg(dev, qp); + if (ret) { + CPT_LOG_ERR("Could not configure queue for inline IPsec"); + goto mempool_destroy; + } + ret = otx2_cpt_iq_enable(dev, qp, group, OTX2_CPT_QUEUE_HI_PRIO, size_div40); if (ret) { @@ -857,12 +887,20 @@ otx2_cpt_dev_config(struct rte_cryptodev *dev, goto queues_detach; } + ret = otx2_cpt_inline_init(dev); + if (ret) { + CPT_LOG_ERR("Could not enable inline IPsec"); + goto intr_unregister; + } + dev->enqueue_burst = otx2_cpt_enqueue_burst; dev->dequeue_burst = otx2_cpt_dequeue_burst; rte_mb(); return 0; +intr_unregister: + otx2_cpt_err_intr_unregister(dev); queues_detach: otx2_cpt_queues_detach(dev); return ret; From patchwork Sun Dec 8 11:54:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 63629 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 46785A04F1; Sun, 8 Dec 2019 12:56:12 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 80B3A1BF73; Sun, 8 Dec 2019 12:55:58 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 537401BF71 for ; Sun, 8 Dec 2019 12:55:57 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB8Bqnm8007482; Sun, 8 Dec 2019 03:55:56 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=pWvV4tQaASanfhRiAxAIE4UE/F99fGfl7d7CLmlxIuc=; b=mQznd/AugaoYusYid88ds26jnLox5ROjqVNDoQngG0zBKyFKxxiwj8WVCIgqeHCypNl+ 3U+np1paxLA9mwU1LROUYYcWTvAGwpMoLwCDWgG0fVKLBeB9RKHUaOIxbaiyt1vAIwDP MUka221FVEVxTCQ2lbeaRjNsPoRFNMSnCqgUqoevRbMwv5u25Jj6vG0Wc8nQM4VQYUnW e8gXuCS+FvUlZf/wm7MyLa4uZkeihnnrqr1gvZ4tqQ9ysEkMcdLHGIgZbjvJrc4EqqK5 1t+OsvHKX8ORChjpdamx6ApJxTqyKX6bjvwW4/Jy69xmrbPQcNtelr/hvs16ih8B1327 3w== Received: from sc-exch03.marvell.com ([199.233.58.183]) by mx0a-0016f401.pphosted.com with ESMTP id 2wrbawjj28-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 08 Dec 2019 03:55:56 -0800 Received: from SC-EXCH03.marvell.com (10.93.176.83) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Sun, 8 Dec 2019 03:55:55 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Sun, 8 Dec 2019 03:55:55 -0800 Received: from ajoseph83.caveonetworks.com.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id 51C023F703F; Sun, 8 Dec 2019 03:55:50 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Declan Doherty , Thomas Monjalon CC: Anoob Joseph , Jerin Jacob , Narayana Prasad , Kiran Kumar K , Nithin Dabilpuram , "Pavan Nikhilesh" , Ankur Dwivedi , Archana Muniganti , Tejasree Kondoj , Vamsi Attunuru , "Lukasz Bartosik" , Date: Sun, 8 Dec 2019 17:24:43 +0530 Message-ID: <1575806094-28391-5-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1575806094-28391-1-git-send-email-anoobj@marvell.com> References: <1575806094-28391-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-08_03:2019-12-05,2019-12-08 signatures=0 Subject: [dpdk-dev] [PATCH 04/15] crypto/octeontx2: create eth security ctx X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Adding security ctx to the eth device. Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Archana Muniganti Signed-off-by: Tejasree Kondoj Signed-off-by: Vamsi Attunuru --- drivers/common/octeontx2/otx2_common.c | 2 ++ drivers/common/octeontx2/otx2_common.h | 10 +++++++ .../octeontx2/rte_common_octeontx2_version.map | 2 ++ drivers/crypto/octeontx2/Makefile | 3 +- drivers/crypto/octeontx2/meson.build | 4 ++- drivers/crypto/octeontx2/otx2_cryptodev.c | 4 +++ drivers/crypto/octeontx2/otx2_security.c | 35 ++++++++++++++++++++++ drivers/crypto/octeontx2/otx2_security.h | 14 +++++++++ drivers/net/octeontx2/otx2_ethdev.c | 18 ++++++++++- 9 files changed, 89 insertions(+), 3 deletions(-) create mode 100644 drivers/crypto/octeontx2/otx2_security.c create mode 100644 drivers/crypto/octeontx2/otx2_security.h diff --git a/drivers/common/octeontx2/otx2_common.c b/drivers/common/octeontx2/otx2_common.c index 116db0f..764f6cd 100644 --- a/drivers/common/octeontx2/otx2_common.c +++ b/drivers/common/octeontx2/otx2_common.c @@ -11,6 +11,8 @@ #include "otx2_dev.h" #include "otx2_mbox.h" +struct otx2_sec_eth_crypto_idev_ops otx2_sec_idev_ops; + /** * @internal * Set default NPA configuration. diff --git a/drivers/common/octeontx2/otx2_common.h b/drivers/common/octeontx2/otx2_common.h index d32e59a..a1cb434 100644 --- a/drivers/common/octeontx2/otx2_common.h +++ b/drivers/common/octeontx2/otx2_common.h @@ -77,6 +77,16 @@ void otx2_npa_set_defaults(struct otx2_idev_cfg *idev); int otx2_npa_lf_active(void *dev); int otx2_npa_lf_obj_ref(void); +typedef int (*otx2_sec_eth_ctx_create_t)(struct rte_eth_dev *eth_dev); +typedef void (*otx2_sec_eth_ctx_destroy_t)(struct rte_eth_dev *eth_dev); + +struct otx2_sec_eth_crypto_idev_ops { + otx2_sec_eth_ctx_create_t ctx_create; + otx2_sec_eth_ctx_destroy_t ctx_destroy; +}; + +extern struct otx2_sec_eth_crypto_idev_ops otx2_sec_idev_ops; + /* Log */ extern int otx2_logtype_base; extern int otx2_logtype_mbox; diff --git a/drivers/common/octeontx2/rte_common_octeontx2_version.map b/drivers/common/octeontx2/rte_common_octeontx2_version.map index dac2283..d1dcb52 100644 --- a/drivers/common/octeontx2/rte_common_octeontx2_version.map +++ b/drivers/common/octeontx2/rte_common_octeontx2_version.map @@ -32,5 +32,7 @@ DPDK_20.0 { otx2_sso_pf_func_set; otx2_unregister_irq; + otx2_sec_idev_ops; + local: *; }; diff --git a/drivers/crypto/octeontx2/Makefile b/drivers/crypto/octeontx2/Makefile index 3ba67ed..d2e9b9f 100644 --- a/drivers/crypto/octeontx2/Makefile +++ b/drivers/crypto/octeontx2/Makefile @@ -11,7 +11,7 @@ LIB = librte_pmd_octeontx2_crypto.a CFLAGS += $(WERROR_FLAGS) LDLIBS += -lrte_eal -lrte_ethdev -lrte_mbuf -lrte_mempool -lrte_ring -LDLIBS += -lrte_cryptodev +LDLIBS += -lrte_cryptodev -lrte_security LDLIBS += -lrte_pci -lrte_bus_pci LDLIBS += -lrte_common_cpt -lrte_common_octeontx2 @@ -38,6 +38,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX2_CRYPTO) += otx2_cryptodev_capabilities.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX2_CRYPTO) += otx2_cryptodev_hw_access.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX2_CRYPTO) += otx2_cryptodev_mbox.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX2_CRYPTO) += otx2_cryptodev_ops.c +SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX2_CRYPTO) += otx2_security.c # export include files SYMLINK-y-include += diff --git a/drivers/crypto/octeontx2/meson.build b/drivers/crypto/octeontx2/meson.build index 67deca3..f7b2937 100644 --- a/drivers/crypto/octeontx2/meson.build +++ b/drivers/crypto/octeontx2/meson.build @@ -9,6 +9,7 @@ deps += ['bus_pci'] deps += ['common_cpt'] deps += ['common_octeontx2'] deps += ['ethdev'] +deps += ['security'] name = 'octeontx2_crypto' allow_experimental_apis = true @@ -16,7 +17,8 @@ sources = files('otx2_cryptodev.c', 'otx2_cryptodev_capabilities.c', 'otx2_cryptodev_hw_access.c', 'otx2_cryptodev_mbox.c', - 'otx2_cryptodev_ops.c') + 'otx2_cryptodev_ops.c', + 'otx2_security.c') extra_flags = [] # This integrated controller runs only on a arm64 machine, remove 32bit warnings diff --git a/drivers/crypto/octeontx2/otx2_cryptodev.c b/drivers/crypto/octeontx2/otx2_cryptodev.c index 7fd216b..86c1188 100644 --- a/drivers/crypto/octeontx2/otx2_cryptodev.c +++ b/drivers/crypto/octeontx2/otx2_cryptodev.c @@ -17,6 +17,7 @@ #include "otx2_cryptodev_mbox.h" #include "otx2_cryptodev_ops.h" #include "otx2_dev.h" +#include "otx2_security.h" /* CPT common headers */ #include "cpt_common.h" @@ -154,4 +155,7 @@ RTE_INIT(otx2_cpt_init_log) otx2_cpt_logtype = rte_log_register("pmd.crypto.octeontx2"); if (otx2_cpt_logtype >= 0) rte_log_set_level(otx2_cpt_logtype, RTE_LOG_NOTICE); + + otx2_sec_idev_ops.ctx_create = otx2_sec_eth_ctx_create; + otx2_sec_idev_ops.ctx_destroy = otx2_sec_eth_ctx_destroy; } diff --git a/drivers/crypto/octeontx2/otx2_security.c b/drivers/crypto/octeontx2/otx2_security.c new file mode 100644 index 0000000..14394d2 --- /dev/null +++ b/drivers/crypto/octeontx2/otx2_security.c @@ -0,0 +1,35 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (C) 2019 Marvell International Ltd. + */ + +#include +#include +#include + +#include "otx2_security.h" + +int +otx2_sec_eth_ctx_create(struct rte_eth_dev *eth_dev) +{ + struct rte_security_ctx *ctx; + + ctx = rte_malloc("otx2_sec_eth_ctx", + sizeof(struct rte_security_ctx), 0); + if (ctx == NULL) + return -ENOMEM; + + /* Populate ctx */ + + ctx->device = eth_dev; + ctx->sess_cnt = 0; + + eth_dev->security_ctx = ctx; + + return 0; +} + +void +otx2_sec_eth_ctx_destroy(struct rte_eth_dev *eth_dev) +{ + rte_free(eth_dev->security_ctx); +} diff --git a/drivers/crypto/octeontx2/otx2_security.h b/drivers/crypto/octeontx2/otx2_security.h new file mode 100644 index 0000000..6ebf73f --- /dev/null +++ b/drivers/crypto/octeontx2/otx2_security.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (C) 2019 Marvell International Ltd. + */ + +#ifndef __OTX2_SECURITY_H__ +#define __OTX2_SECURITY_H__ + +#include + +int otx2_sec_eth_ctx_create(struct rte_eth_dev *eth_dev); + +void otx2_sec_eth_ctx_destroy(struct rte_eth_dev *eth_dev); + +#endif /* __OTX2_SECURITY_H__ */ diff --git a/drivers/net/octeontx2/otx2_ethdev.c b/drivers/net/octeontx2/otx2_ethdev.c index ed32927..3e19ac2 100644 --- a/drivers/net/octeontx2/otx2_ethdev.c +++ b/drivers/net/octeontx2/otx2_ethdev.c @@ -2236,10 +2236,19 @@ otx2_eth_dev_init(struct rte_eth_dev *eth_dev) dev->hwcap |= OTX2_FIXUP_F_LIMIT_CQ_FULL; } + /* Create security ctx */ + if (otx2_sec_idev_ops.ctx_create != NULL) { + rc = otx2_sec_idev_ops.ctx_create(eth_dev); + if (rc) + goto free_mac_addrs; + dev->tx_offload_capa |= DEV_TX_OFFLOAD_SECURITY; + dev->rx_offload_capa |= DEV_RX_OFFLOAD_SECURITY; + } + /* Initialize rte-flow */ rc = otx2_flow_init(dev); if (rc) - goto free_mac_addrs; + goto sec_ctx_destroy; otx2_nix_mc_filter_init(dev); @@ -2250,6 +2259,9 @@ otx2_eth_dev_init(struct rte_eth_dev *eth_dev) dev->rx_offload_capa, dev->tx_offload_capa); return 0; +sec_ctx_destroy: + if (otx2_sec_idev_ops.ctx_destroy != NULL) + otx2_sec_idev_ops.ctx_destroy(eth_dev); free_mac_addrs: rte_free(eth_dev->data->mac_addrs); unregister_irq: @@ -2333,6 +2345,10 @@ otx2_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) if (rc) otx2_err("Failed to cleanup npa lf, rc=%d", rc); + /* Destroy security ctx */ + if (otx2_sec_idev_ops.ctx_destroy != NULL) + otx2_sec_idev_ops.ctx_destroy(eth_dev); + rte_free(eth_dev->data->mac_addrs); eth_dev->data->mac_addrs = NULL; dev->drv_inited = false; From patchwork Sun Dec 8 11:54:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 63630 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 72A16A04F1; Sun, 8 Dec 2019 12:56:22 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 673001BF7A; Sun, 8 Dec 2019 12:56:06 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id BB5241BF7A for ; Sun, 8 Dec 2019 12:56:04 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB8BpS6R005483; Sun, 8 Dec 2019 03:56:04 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=BsoV6WhQEqthDLoDf11sK2LWiyS3W7DBSV6aQGE6yls=; b=B+O2ZfAiNOqpXMPZMUDWjrbuenrktEabeAxhBJrSi/GhoNGjZ7ZRV9DOrxLKCkBesRto GUP0gLj2maM4gcymHuLB/f7OnOO7jSjVnQnJCQt0bni7S9nID0rQ2J1I4ccbaaYxuRHN P4+iHvzo2AZRxRNjsfC2RMnI4UO8LLkqMuNrlIrD4ZASPOBU3t7s8dTYpNm52j1AwHiH U4yUnKHP9EFvCN8qcBUhHJ7G8dr6WAr8arkXcyuU+tZw6YKo2JgfJ+pNXR6i2KHgB/QI xmCYiq7CKJb85IkPnq+GRZ5YnmqEndWHK89+3rHpkDfrlYgUGQeJ6RIKMHRhn16Hz7so 8w== Received: from sc-exch01.marvell.com ([199.233.58.181]) by mx0b-0016f401.pphosted.com with ESMTP id 2wrcfptc19-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 08 Dec 2019 03:56:03 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Sun, 8 Dec 2019 03:56:01 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Sun, 8 Dec 2019 03:56:01 -0800 Received: from ajoseph83.caveonetworks.com.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id BB2AB3F7043; Sun, 8 Dec 2019 03:55:56 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Declan Doherty , Thomas Monjalon CC: Tejasree Kondoj , Jerin Jacob , Narayana Prasad , Kiran Kumar K , Nithin Dabilpuram , "Pavan Nikhilesh" , Ankur Dwivedi , Anoob Joseph , Archana Muniganti , Vamsi Attunuru , Lukasz Bartosik , Date: Sun, 8 Dec 2019 17:24:44 +0530 Message-ID: <1575806094-28391-6-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1575806094-28391-1-git-send-email-anoobj@marvell.com> References: <1575806094-28391-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-08_03:2019-12-05,2019-12-08 signatures=0 Subject: [dpdk-dev] [PATCH 05/15] crypto/octeontx2: add security in eth dev configure X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Tejasree Kondoj Adding security in eth device configure. Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Archana Muniganti Signed-off-by: Tejasree Kondoj Signed-off-by: Vamsi Attunuru --- doc/guides/nics/octeontx2.rst | 20 +++++ drivers/common/octeontx2/otx2_common.h | 4 + drivers/crypto/octeontx2/Makefile | 2 +- drivers/crypto/octeontx2/meson.build | 1 + drivers/crypto/octeontx2/otx2_cryptodev.c | 2 + drivers/crypto/octeontx2/otx2_ipsec_fp.h | 55 +++++++++++++ drivers/crypto/octeontx2/otx2_security.c | 122 ++++++++++++++++++++++++++++ drivers/crypto/octeontx2/otx2_security.h | 4 + drivers/net/octeontx2/otx2_ethdev.c | 22 ++++- drivers/net/octeontx2/otx2_ethdev.h | 2 + drivers/net/octeontx2/otx2_ethdev_devargs.c | 19 +++++ 11 files changed, 251 insertions(+), 2 deletions(-) create mode 100644 drivers/crypto/octeontx2/otx2_ipsec_fp.h diff --git a/doc/guides/nics/octeontx2.rst b/doc/guides/nics/octeontx2.rst index db62a45..fd4e455 100644 --- a/doc/guides/nics/octeontx2.rst +++ b/doc/guides/nics/octeontx2.rst @@ -38,6 +38,7 @@ Features of the OCTEON TX2 Ethdev PMD are: - IEEE1588 timestamping - HW offloaded `ethdev Rx queue` to `eventdev event queue` packet injection - Support Rx interrupt +- Inline IPsec processing support Prerequisites ------------- @@ -178,6 +179,17 @@ Runtime Config Options traffic on this port should be higig2 traffic only. Supported switch header types are "higig2" and "dsa". +- ``Max SPI for inbound inline IPsec`` (default ``1``) + + Max SPI supported for inbound inline IPsec processing can be specified by + ``ipsec_in_max_spi`` ``devargs`` parameter. + + For example:: + -w 0002:02:00.0,ipsec_in_max_spi=128 + + With the above configuration, application can enable inline IPsec processing + on 128 SAs (SPI 0-127). + .. note:: Above devarg parameters are configurable per device, user needs to pass the @@ -211,6 +223,14 @@ SDP interface support ~~~~~~~~~~~~~~~~~~~~~ OCTEON TX2 SDP interface support is limited to PF device, No VF support. +Inline Protocol Processing +~~~~~~~~~~~~~~~~~~~~~~~~~~ +``net_octeontx2`` pmd doesn't support the following features for packets to be +inline protocol processed. +- TSO offload +- VLAN/QinQ offload +- Fragmentation + Debugging Options ----------------- diff --git a/drivers/common/octeontx2/otx2_common.h b/drivers/common/octeontx2/otx2_common.h index a1cb434..9705a8d 100644 --- a/drivers/common/octeontx2/otx2_common.h +++ b/drivers/common/octeontx2/otx2_common.h @@ -79,10 +79,14 @@ int otx2_npa_lf_obj_ref(void); typedef int (*otx2_sec_eth_ctx_create_t)(struct rte_eth_dev *eth_dev); typedef void (*otx2_sec_eth_ctx_destroy_t)(struct rte_eth_dev *eth_dev); +typedef int (*otx2_sec_eth_init_t)(struct rte_eth_dev *eth_dev); +typedef void (*otx2_sec_eth_fini_t)(struct rte_eth_dev *eth_dev); struct otx2_sec_eth_crypto_idev_ops { otx2_sec_eth_ctx_create_t ctx_create; otx2_sec_eth_ctx_destroy_t ctx_destroy; + otx2_sec_eth_init_t init; + otx2_sec_eth_fini_t fini; }; extern struct otx2_sec_eth_crypto_idev_ops otx2_sec_idev_ops; diff --git a/drivers/crypto/octeontx2/Makefile b/drivers/crypto/octeontx2/Makefile index d2e9b9f..5966ddc 100644 --- a/drivers/crypto/octeontx2/Makefile +++ b/drivers/crypto/octeontx2/Makefile @@ -11,7 +11,7 @@ LIB = librte_pmd_octeontx2_crypto.a CFLAGS += $(WERROR_FLAGS) LDLIBS += -lrte_eal -lrte_ethdev -lrte_mbuf -lrte_mempool -lrte_ring -LDLIBS += -lrte_cryptodev -lrte_security +LDLIBS += -lrte_cryptodev -lrte_security -lrte_eventdev LDLIBS += -lrte_pci -lrte_bus_pci LDLIBS += -lrte_common_cpt -lrte_common_octeontx2 diff --git a/drivers/crypto/octeontx2/meson.build b/drivers/crypto/octeontx2/meson.build index f7b2937..f0f5043 100644 --- a/drivers/crypto/octeontx2/meson.build +++ b/drivers/crypto/octeontx2/meson.build @@ -9,6 +9,7 @@ deps += ['bus_pci'] deps += ['common_cpt'] deps += ['common_octeontx2'] deps += ['ethdev'] +deps += ['eventdev'] deps += ['security'] name = 'octeontx2_crypto' diff --git a/drivers/crypto/octeontx2/otx2_cryptodev.c b/drivers/crypto/octeontx2/otx2_cryptodev.c index 86c1188..34feb82 100644 --- a/drivers/crypto/octeontx2/otx2_cryptodev.c +++ b/drivers/crypto/octeontx2/otx2_cryptodev.c @@ -158,4 +158,6 @@ RTE_INIT(otx2_cpt_init_log) otx2_sec_idev_ops.ctx_create = otx2_sec_eth_ctx_create; otx2_sec_idev_ops.ctx_destroy = otx2_sec_eth_ctx_destroy; + otx2_sec_idev_ops.init = otx2_sec_eth_init; + otx2_sec_idev_ops.fini = otx2_sec_eth_fini; } diff --git a/drivers/crypto/octeontx2/otx2_ipsec_fp.h b/drivers/crypto/octeontx2/otx2_ipsec_fp.h new file mode 100644 index 0000000..520a3fe --- /dev/null +++ b/drivers/crypto/octeontx2/otx2_ipsec_fp.h @@ -0,0 +1,55 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2019 Marvell International Ltd. + */ + +#ifndef __OTX2_IPSEC_FP_H__ +#define __OTX2_IPSEC_FP_H__ + +struct otx2_ipsec_fp_sa_ctl { + rte_be32_t spi : 32; + uint64_t exp_proto_inter_frag : 8; + uint64_t rsvd_42_40 : 3; + uint64_t esn_en : 1; + uint64_t rsvd_45_44 : 2; + uint64_t encap_type : 2; + uint64_t enc_type : 3; + uint64_t rsvd_48 : 1; + uint64_t auth_type : 4; + uint64_t valid : 1; + uint64_t direction : 1; + uint64_t outer_ip_ver : 1; + uint64_t inner_ip_ver : 1; + uint64_t ipsec_mode : 1; + uint64_t ipsec_proto : 1; + uint64_t aes_key_len : 2; +}; + +struct otx2_ipsec_fp_in_sa { + /* w0 */ + struct otx2_ipsec_fp_sa_ctl ctl; + + /* w1 */ + uint8_t nonce[4]; /* Only for AES-GCM */ + uint32_t unused; + + /* w2 */ + uint32_t esn_low; + uint32_t esn_hi; + + /* w3-w6 */ + uint8_t cipher_key[32]; + + /* w7-w12 */ + uint8_t hmac_key[48]; + + RTE_STD_C11 + union { + void *userdata; + uint64_t udata64; + }; + + uint64_t reserved1; + uint64_t reserved2; +}; + +#endif /* __OTX2_IPSEC_FP_H__ */ diff --git a/drivers/crypto/octeontx2/otx2_security.c b/drivers/crypto/octeontx2/otx2_security.c index 14394d2..9cd4683 100644 --- a/drivers/crypto/octeontx2/otx2_security.c +++ b/drivers/crypto/octeontx2/otx2_security.c @@ -3,11 +3,36 @@ */ #include +#include #include +#include #include +#include "otx2_ethdev.h" +#include "otx2_ipsec_fp.h" #include "otx2_security.h" +#define SEC_ETH_MAX_PKT_LEN 1450 + +struct sec_eth_tag_const { + RTE_STD_C11 + union { + struct { + uint32_t rsvd_11_0 : 12; + uint32_t port : 8; + uint32_t event_type : 4; + uint32_t rsvd_31_24 : 8; + }; + uint32_t u32; + }; +}; + +static inline void +in_sa_mz_name_get(char *name, int size, uint16_t port) +{ + snprintf(name, size, "otx2_ipsec_in_sadb_%u", port); +} + int otx2_sec_eth_ctx_create(struct rte_eth_dev *eth_dev) { @@ -33,3 +58,100 @@ otx2_sec_eth_ctx_destroy(struct rte_eth_dev *eth_dev) { rte_free(eth_dev->security_ctx); } + +static int +sec_eth_ipsec_cfg(struct rte_eth_dev *eth_dev, uint8_t tt) +{ + struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev); + uint16_t port = eth_dev->data->port_id; + struct nix_inline_ipsec_lf_cfg *req; + struct otx2_mbox *mbox = dev->mbox; + struct sec_eth_tag_const tag_const; + char name[RTE_MEMZONE_NAMESIZE]; + const struct rte_memzone *mz; + + in_sa_mz_name_get(name, RTE_MEMZONE_NAMESIZE, port); + mz = rte_memzone_lookup(name); + if (mz == NULL) + return -EINVAL; + + req = otx2_mbox_alloc_msg_nix_inline_ipsec_lf_cfg(mbox); + req->enable = 1; + req->sa_base_addr = mz->iova; + + req->ipsec_cfg0.tt = tt; + + tag_const.u32 = 0; + tag_const.event_type = RTE_EVENT_TYPE_ETHDEV; + tag_const.port = port; + req->ipsec_cfg0.tag_const = tag_const.u32; + + req->ipsec_cfg0.sa_pow2_size = + rte_log2_u32(sizeof(struct otx2_ipsec_fp_in_sa)); + req->ipsec_cfg0.lenm1_max = SEC_ETH_MAX_PKT_LEN - 1; + + req->ipsec_cfg1.sa_idx_w = rte_log2_u32(dev->ipsec_in_max_spi); + req->ipsec_cfg1.sa_idx_max = dev->ipsec_in_max_spi - 1; + + return otx2_mbox_process(mbox); +} + +int +otx2_sec_eth_init(struct rte_eth_dev *eth_dev) +{ + const size_t sa_width = sizeof(struct otx2_ipsec_fp_in_sa); + struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev); + uint16_t port = eth_dev->data->port_id; + char name[RTE_MEMZONE_NAMESIZE]; + const struct rte_memzone *mz; + int mz_sz, ret; + uint16_t nb_sa; + + RTE_BUILD_BUG_ON(sa_width < 32 || sa_width > 512 || + !RTE_IS_POWER_OF_2(sa_width)); + + if (!(dev->tx_offloads & DEV_TX_OFFLOAD_SECURITY) && + !(dev->rx_offloads & DEV_RX_OFFLOAD_SECURITY)) + return 0; + + nb_sa = dev->ipsec_in_max_spi; + mz_sz = nb_sa * sa_width; + in_sa_mz_name_get(name, RTE_MEMZONE_NAMESIZE, port); + mz = rte_memzone_reserve_aligned(name, mz_sz, rte_socket_id(), + RTE_MEMZONE_IOVA_CONTIG, OTX2_ALIGN); + + if (mz == NULL) { + otx2_err("Could not allocate inbound SA DB"); + return -ENOMEM; + } + + memset(mz->addr, 0, mz_sz); + + ret = sec_eth_ipsec_cfg(eth_dev, SSO_TT_ORDERED); + if (ret < 0) { + otx2_err("Could not configure inline IPsec"); + goto sec_fini; + } + + return 0; + +sec_fini: + otx2_err("Could not configure device for security"); + otx2_sec_eth_fini(eth_dev); + return ret; +} + +void +otx2_sec_eth_fini(struct rte_eth_dev *eth_dev) +{ + struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev); + uint16_t port = eth_dev->data->port_id; + char name[RTE_MEMZONE_NAMESIZE]; + + if (!(dev->tx_offloads & DEV_TX_OFFLOAD_SECURITY) && + !(dev->rx_offloads & DEV_RX_OFFLOAD_SECURITY)) + return; + + in_sa_mz_name_get(name, RTE_MEMZONE_NAMESIZE, port); + rte_memzone_free(rte_memzone_lookup(name)); +} diff --git a/drivers/crypto/octeontx2/otx2_security.h b/drivers/crypto/octeontx2/otx2_security.h index 6ebf73f..4704781 100644 --- a/drivers/crypto/octeontx2/otx2_security.h +++ b/drivers/crypto/octeontx2/otx2_security.h @@ -11,4 +11,8 @@ int otx2_sec_eth_ctx_create(struct rte_eth_dev *eth_dev); void otx2_sec_eth_ctx_destroy(struct rte_eth_dev *eth_dev); +int otx2_sec_eth_init(struct rte_eth_dev *eth_dev); + +void otx2_sec_eth_fini(struct rte_eth_dev *eth_dev); + #endif /* __OTX2_SECURITY_H__ */ diff --git a/drivers/net/octeontx2/otx2_ethdev.c b/drivers/net/octeontx2/otx2_ethdev.c index 3e19ac2..131e883 100644 --- a/drivers/net/octeontx2/otx2_ethdev.c +++ b/drivers/net/octeontx2/otx2_ethdev.c @@ -337,6 +337,10 @@ nix_cq_rq_init(struct rte_eth_dev *eth_dev, struct otx2_eth_dev *dev, aq->op = NIX_AQ_INSTOP_INIT; aq->rq.sso_ena = 0; + + if (rxq->offloads & DEV_RX_OFFLOAD_SECURITY) + aq->rq.ipsech_ena = 1; + aq->rq.cq = qid; /* RQ to CQ 1:1 mapped */ aq->rq.spb_ena = 0; aq->rq.lpb_aura = npa_lf_aura_handle_to_aura(mp->pool_id); @@ -1610,6 +1614,8 @@ otx2_nix_configure(struct rte_eth_dev *eth_dev) /* Free the resources allocated from the previous configure */ if (dev->configured == 1) { + if (otx2_sec_idev_ops.fini != NULL) + otx2_sec_idev_ops.fini(eth_dev); otx2_nix_rxchan_bpid_cfg(eth_dev, false); otx2_nix_vlan_fini(eth_dev); otx2_nix_mc_addr_list_uninstall(eth_dev); @@ -1714,10 +1720,17 @@ otx2_nix_configure(struct rte_eth_dev *eth_dev) goto cq_fini; } + /* Enable security */ + if (otx2_sec_idev_ops.init != NULL) { + rc = otx2_sec_idev_ops.init(eth_dev); + if (rc) + goto cq_fini; + } + rc = otx2_nix_mc_addr_list_install(eth_dev); if (rc < 0) { otx2_err("Failed to install mc address list rc=%d", rc); - goto cq_fini; + goto sec_fini; } /* @@ -1753,6 +1766,9 @@ otx2_nix_configure(struct rte_eth_dev *eth_dev) uninstall_mc_list: otx2_nix_mc_addr_list_uninstall(eth_dev); +sec_fini: + if (otx2_sec_idev_ops.fini != NULL) + otx2_sec_idev_ops.fini(eth_dev); cq_fini: oxt2_nix_unregister_cq_irqs(eth_dev); q_irq_fini: @@ -2345,6 +2361,10 @@ otx2_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) if (rc) otx2_err("Failed to cleanup npa lf, rc=%d", rc); + /* Disable security */ + if (otx2_sec_idev_ops.fini != NULL) + otx2_sec_idev_ops.fini(eth_dev); + /* Destroy security ctx */ if (otx2_sec_idev_ops.ctx_destroy != NULL) otx2_sec_idev_ops.ctx_destroy(eth_dev); diff --git a/drivers/net/octeontx2/otx2_ethdev.h b/drivers/net/octeontx2/otx2_ethdev.h index 987e760..41fef6e 100644 --- a/drivers/net/octeontx2/otx2_ethdev.h +++ b/drivers/net/octeontx2/otx2_ethdev.h @@ -324,6 +324,8 @@ struct otx2_eth_dev { bool mc_tbl_set; struct otx2_nix_mc_filter_tbl mc_fltr_tbl; bool sdp_link; /* SDP flag */ + /* Inline IPsec params */ + uint16_t ipsec_in_max_spi; } __rte_cache_aligned; struct otx2_eth_txq { diff --git a/drivers/net/octeontx2/otx2_ethdev_devargs.c b/drivers/net/octeontx2/otx2_ethdev_devargs.c index 04da1ab..a3f7598 100644 --- a/drivers/net/octeontx2/otx2_ethdev_devargs.c +++ b/drivers/net/octeontx2/otx2_ethdev_devargs.c @@ -64,6 +64,19 @@ parse_reta_size(const char *key, const char *value, void *extra_args) } static int +parse_ipsec_in_max_spi(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + uint32_t val; + + val = atoi(value); + + *(uint16_t *)extra_args = val; + + return 0; +} + +static int parse_flag(const char *key, const char *value, void *extra_args) { RTE_SET_USED(key); @@ -104,6 +117,7 @@ parse_switch_header_type(const char *key, const char *value, void *extra_args) } #define OTX2_RSS_RETA_SIZE "reta_size" +#define OTX2_IPSEC_IN_MAX_SPI "ipsec_in_max_spi" #define OTX2_SCL_ENABLE "scalar_enable" #define OTX2_MAX_SQB_COUNT "max_sqb_count" #define OTX2_FLOW_PREALLOC_SIZE "flow_prealloc_size" @@ -118,6 +132,7 @@ otx2_ethdev_parse_devargs(struct rte_devargs *devargs, struct otx2_eth_dev *dev) uint16_t flow_prealloc_size = 8; uint16_t switch_header_type = 0; uint16_t flow_max_priority = 3; + uint16_t ipsec_in_max_spi = 1; uint16_t scalar_enable = 0; struct rte_kvargs *kvlist; @@ -130,6 +145,8 @@ otx2_ethdev_parse_devargs(struct rte_devargs *devargs, struct otx2_eth_dev *dev) rte_kvargs_process(kvlist, OTX2_RSS_RETA_SIZE, &parse_reta_size, &rss_size); + rte_kvargs_process(kvlist, OTX2_IPSEC_IN_MAX_SPI, + &parse_ipsec_in_max_spi, &ipsec_in_max_spi); rte_kvargs_process(kvlist, OTX2_SCL_ENABLE, &parse_flag, &scalar_enable); rte_kvargs_process(kvlist, OTX2_MAX_SQB_COUNT, @@ -143,6 +160,7 @@ otx2_ethdev_parse_devargs(struct rte_devargs *devargs, struct otx2_eth_dev *dev) rte_kvargs_free(kvlist); null_devargs: + dev->ipsec_in_max_spi = ipsec_in_max_spi; dev->scalar_ena = scalar_enable; dev->max_sqb_count = sqb_count; dev->rss_info.rss_size = rss_size; @@ -157,6 +175,7 @@ otx2_ethdev_parse_devargs(struct rte_devargs *devargs, struct otx2_eth_dev *dev) RTE_PMD_REGISTER_PARAM_STRING(net_octeontx2, OTX2_RSS_RETA_SIZE "=<64|128|256>" + OTX2_IPSEC_IN_MAX_SPI "=<1-65535>" OTX2_SCL_ENABLE "=1" OTX2_MAX_SQB_COUNT "=<8-512>" OTX2_FLOW_PREALLOC_SIZE "=<1-32>" From patchwork Sun Dec 8 11:54:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 63631 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id DA6B9A04F1; Sun, 8 Dec 2019 12:56:34 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6C2231BF6F; Sun, 8 Dec 2019 12:56:12 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 36B0E1BF6D for ; Sun, 8 Dec 2019 12:56:10 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB8BqnmB007482; Sun, 8 Dec 2019 03:56:09 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=qWLlOsR+5hzaZkA9MI8dt17qZZJZ1qLyFpxxZulJEX4=; b=KBBwcd8O3EehwEmxnz74apn1oH5lada7XNbYjCGfHf2imwsai+XZVVhPGJtfVF3orKnu OQACYS3/+SV054KpLn45GKZFtvQq05k90ZRqXE0632vGl6LqrWcTpbnz/x2uxQBD5vkW HfQUy6mlox+F2B7Dr2jQn76Yc0tM70sBjTtNp8MoiQXvo5SjxVbJKrKStHSCW4flcBgA w/ZrhFrGwud5f+R6bXuf7sUn8xEhMcHsZZUZcXF9/HG6bRiTi5P84Ib+JdPfW4IdwLy3 TAXBDAZW3Y5BYoId9lE305u0OBvJOjsIC17E5VeMEhXF8J0bgWugPhprqJX0p1Dvf45X Nw== Received: from sc-exch02.marvell.com ([199.233.58.182]) by mx0a-0016f401.pphosted.com with ESMTP id 2wrbawjj2m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 08 Dec 2019 03:56:09 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Sun, 8 Dec 2019 03:56:08 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Sun, 8 Dec 2019 03:56:08 -0800 Received: from ajoseph83.caveonetworks.com.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id 3FF673F7043; Sun, 8 Dec 2019 03:56:02 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Declan Doherty , Thomas Monjalon CC: Ankur Dwivedi , Jerin Jacob , Narayana Prasad , Kiran Kumar K , Nithin Dabilpuram , "Pavan Nikhilesh" , Anoob Joseph , Archana Muniganti , Tejasree Kondoj , Vamsi Attunuru , "Lukasz Bartosik" , Date: Sun, 8 Dec 2019 17:24:45 +0530 Message-ID: <1575806094-28391-7-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1575806094-28391-1-git-send-email-anoobj@marvell.com> References: <1575806094-28391-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-08_03:2019-12-05,2019-12-08 signatures=0 Subject: [dpdk-dev] [PATCH 06/15] crypto/octeontx2: add eth security capabilities X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Ankur Dwivedi Adding security capabilities supported by the eth PMD. Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Archana Muniganti Signed-off-by: Tejasree Kondoj Signed-off-by: Vamsi Attunuru --- drivers/crypto/octeontx2/otx2_security.c | 124 +++++++++++++++++++++++++++++++ drivers/crypto/octeontx2/otx2_security.h | 18 +++++ 2 files changed, 142 insertions(+) diff --git a/drivers/crypto/octeontx2/otx2_security.c b/drivers/crypto/octeontx2/otx2_security.c index 9cd4683..46b94e5 100644 --- a/drivers/crypto/octeontx2/otx2_security.c +++ b/drivers/crypto/octeontx2/otx2_security.c @@ -2,11 +2,13 @@ * Copyright (C) 2019 Marvell International Ltd. */ +#include #include #include #include #include #include +#include #include "otx2_ethdev.h" #include "otx2_ipsec_fp.h" @@ -27,12 +29,133 @@ struct sec_eth_tag_const { }; }; +static struct rte_cryptodev_capabilities otx2_sec_eth_crypto_caps[] = { + { /* AES GCM */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AEAD, + {.aead = { + .algo = RTE_CRYPTO_AEAD_AES_GCM, + .block_size = 16, + .key_size = { + .min = 16, + .max = 32, + .increment = 8 + }, + .digest_size = { + .min = 16, + .max = 16, + .increment = 0 + }, + .aad_size = { + .min = 8, + .max = 12, + .increment = 4 + }, + .iv_size = { + .min = 12, + .max = 12, + .increment = 0 + } + }, } + }, } + }, + { /* AES CBC */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER, + {.cipher = { + .algo = RTE_CRYPTO_CIPHER_AES_CBC, + .block_size = 16, + .key_size = { + .min = 16, + .max = 32, + .increment = 8 + }, + .iv_size = { + .min = 16, + .max = 16, + .increment = 0 + } + }, } + }, } + }, + { /* SHA1 HMAC */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH, + {.auth = { + .algo = RTE_CRYPTO_AUTH_SHA1_HMAC, + .block_size = 64, + .key_size = { + .min = 20, + .max = 64, + .increment = 1 + }, + .digest_size = { + .min = 12, + .max = 12, + .increment = 0 + }, + }, } + }, } + }, + RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST() +}; + +static const struct rte_security_capability otx2_sec_eth_capabilities[] = { + { /* IPsec Inline Protocol ESP Tunnel Ingress */ + .action = RTE_SECURITY_ACTION_TYPE_INLINE_PROTOCOL, + .protocol = RTE_SECURITY_PROTOCOL_IPSEC, + .ipsec = { + .proto = RTE_SECURITY_IPSEC_SA_PROTO_ESP, + .mode = RTE_SECURITY_IPSEC_SA_MODE_TUNNEL, + .direction = RTE_SECURITY_IPSEC_SA_DIR_INGRESS, + .options = { 0 } + }, + .crypto_capabilities = otx2_sec_eth_crypto_caps, + .ol_flags = RTE_SECURITY_TX_OLOAD_NEED_MDATA + }, + { /* IPsec Inline Protocol ESP Tunnel Egress */ + .action = RTE_SECURITY_ACTION_TYPE_INLINE_PROTOCOL, + .protocol = RTE_SECURITY_PROTOCOL_IPSEC, + .ipsec = { + .proto = RTE_SECURITY_IPSEC_SA_PROTO_ESP, + .mode = RTE_SECURITY_IPSEC_SA_MODE_TUNNEL, + .direction = RTE_SECURITY_IPSEC_SA_DIR_EGRESS, + .options = { 0 } + }, + .crypto_capabilities = otx2_sec_eth_crypto_caps, + .ol_flags = RTE_SECURITY_TX_OLOAD_NEED_MDATA + }, + { + .action = RTE_SECURITY_ACTION_TYPE_NONE + } +}; + static inline void in_sa_mz_name_get(char *name, int size, uint16_t port) { snprintf(name, size, "otx2_ipsec_in_sadb_%u", port); } +static unsigned int +otx2_sec_eth_session_get_size(void *device __rte_unused) +{ + return sizeof(struct otx2_sec_session); +} + +static const struct rte_security_capability * +otx2_sec_eth_capabilities_get(void *device __rte_unused) +{ + return otx2_sec_eth_capabilities; +} + +static struct rte_security_ops otx2_sec_eth_ops = { + .session_get_size = otx2_sec_eth_session_get_size, + .capabilities_get = otx2_sec_eth_capabilities_get +}; + int otx2_sec_eth_ctx_create(struct rte_eth_dev *eth_dev) { @@ -46,6 +169,7 @@ otx2_sec_eth_ctx_create(struct rte_eth_dev *eth_dev) /* Populate ctx */ ctx->device = eth_dev; + ctx->ops = &otx2_sec_eth_ops; ctx->sess_cnt = 0; eth_dev->security_ctx = ctx; diff --git a/drivers/crypto/octeontx2/otx2_security.h b/drivers/crypto/octeontx2/otx2_security.h index 4704781..f819f01 100644 --- a/drivers/crypto/octeontx2/otx2_security.h +++ b/drivers/crypto/octeontx2/otx2_security.h @@ -7,6 +7,24 @@ #include +#include "otx2_ipsec_fp.h" + +/* + * Security session for inline IPsec protocol offload. This is private data of + * inline capable PMD. + */ +struct otx2_sec_session_ipsec_ip { + int dummy; +}; + +struct otx2_sec_session_ipsec { + struct otx2_sec_session_ipsec_ip ip; +}; + +struct otx2_sec_session { + struct otx2_sec_session_ipsec ipsec; +} __rte_cache_aligned; + int otx2_sec_eth_ctx_create(struct rte_eth_dev *eth_dev); void otx2_sec_eth_ctx_destroy(struct rte_eth_dev *eth_dev); From patchwork Sun Dec 8 11:54:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 63632 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id E5449A04F1; Sun, 8 Dec 2019 12:56:42 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0EB641BDFD; Sun, 8 Dec 2019 12:56:19 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 145C61BDFD for ; Sun, 8 Dec 2019 12:56:16 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB8BpXxI006265; Sun, 8 Dec 2019 03:56:16 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=qyiz6Pzs6s11hZ+DvdOYD49WJR/aZYAb8f5rSyLwJcg=; b=Mwm/3bJw20B6JLKFTi5DscEcpM6r3XU9iTQmOiOX61sgZeVCyaUfcpaFyorP+W9WtUL0 AswxzkG7Qd29ot2Dl/DhTKCMIoBMhPgiB5ciXrXbFLEybqD3VYQrnCuX1etweM+Mu9CC dZopx8cJwxzuxsaVxxLoI8S8ehFNJOR5NG8iR1GbswEtaAa6FNfpKLurqIj9vo+H3Wjb vifruLRggtAVZ1/FzxAbPrv2mlyZ1RhAXfpU1EGBgwfJphc8k0FP1me/1J3ZSwGit/UE c8WSTrcgznAAh6vQwX2WbiWYOXwZpDYgCyZnJHEH+Q9kQurIfGMSM0EEo+Jyjer7s2Vt rA== Received: from sc-exch01.marvell.com ([199.233.58.181]) by mx0a-0016f401.pphosted.com with ESMTP id 2wrbawjj2u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 08 Dec 2019 03:56:16 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Sun, 8 Dec 2019 03:56:14 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Sun, 8 Dec 2019 03:56:14 -0800 Received: from ajoseph83.caveonetworks.com.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id B7FED3F703F; Sun, 8 Dec 2019 03:56:09 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Declan Doherty , Thomas Monjalon CC: Anoob Joseph , Jerin Jacob , Narayana Prasad , Kiran Kumar K , Nithin Dabilpuram , "Pavan Nikhilesh" , Ankur Dwivedi , Archana Muniganti , Tejasree Kondoj , Vamsi Attunuru , "Lukasz Bartosik" , Date: Sun, 8 Dec 2019 17:24:46 +0530 Message-ID: <1575806094-28391-8-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1575806094-28391-1-git-send-email-anoobj@marvell.com> References: <1575806094-28391-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-08_03:2019-12-05,2019-12-08 signatures=0 Subject: [dpdk-dev] [PATCH 07/15] crypto/octeontx2: enable CPT to share QP with ethdev X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Adding the infrastructure to save one opaque pointer in idev and implement the consumer-producer in the PMDs which uses it accordingly. Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Archana Muniganti Signed-off-by: Tejasree Kondoj Signed-off-by: Vamsi Attunuru --- .../crypto/octeontx2/otx2_cryptodev_hw_access.h | 22 +---- drivers/crypto/octeontx2/otx2_cryptodev_ops.c | 18 ++++ drivers/crypto/octeontx2/otx2_cryptodev_qp.h | 35 ++++++++ drivers/crypto/octeontx2/otx2_security.c | 98 ++++++++++++++++++++++ drivers/crypto/octeontx2/otx2_security.h | 20 +++++ 5 files changed, 172 insertions(+), 21 deletions(-) create mode 100644 drivers/crypto/octeontx2/otx2_cryptodev_qp.h diff --git a/drivers/crypto/octeontx2/otx2_cryptodev_hw_access.h b/drivers/crypto/octeontx2/otx2_cryptodev_hw_access.h index 6f78aa4..43db6a6 100644 --- a/drivers/crypto/octeontx2/otx2_cryptodev_hw_access.h +++ b/drivers/crypto/octeontx2/otx2_cryptodev_hw_access.h @@ -15,6 +15,7 @@ #include "cpt_mcode_defines.h" #include "otx2_dev.h" +#include "otx2_cryptodev_qp.h" /* CPT instruction queue length */ #define OTX2_CPT_IQ_LEN 8200 @@ -135,27 +136,6 @@ enum cpt_9x_comp_e { CPT_9X_COMP_E_LAST_ENTRY = 0x06 }; -struct otx2_cpt_qp { - uint32_t id; - /**< Queue pair id */ - uintptr_t base; - /**< Base address where BAR is mapped */ - void *lmtline; - /**< Address of LMTLINE */ - rte_iova_t lf_nq_reg; - /**< LF enqueue register address */ - struct pending_queue pend_q; - /**< Pending queue */ - struct rte_mempool *sess_mp; - /**< Session mempool */ - struct rte_mempool *sess_mp_priv; - /**< Session private data mempool */ - struct cpt_qp_meta_info meta_info; - /**< Metabuf info required to support operations on the queue pair */ - rte_iova_t iq_dma_addr; - /**< Instruction queue address */ -}; - void otx2_cpt_err_intr_unregister(const struct rte_cryptodev *dev); int otx2_cpt_err_intr_register(const struct rte_cryptodev *dev); diff --git a/drivers/crypto/octeontx2/otx2_cryptodev_ops.c b/drivers/crypto/octeontx2/otx2_cryptodev_ops.c index a1213ca..702a653 100644 --- a/drivers/crypto/octeontx2/otx2_cryptodev_ops.c +++ b/drivers/crypto/octeontx2/otx2_cryptodev_ops.c @@ -13,6 +13,7 @@ #include "otx2_cryptodev_hw_access.h" #include "otx2_cryptodev_mbox.h" #include "otx2_cryptodev_ops.h" +#include "otx2_security.h" #include "otx2_mbox.h" #include "cpt_hw_types.h" @@ -146,6 +147,11 @@ otx2_cpt_qp_inline_cfg(const struct rte_cryptodev *dev, struct otx2_cpt_qp *qp) if (ret) return ret; + /* Publish inline Tx QP to eth dev security */ + ret = otx2_sec_tx_cpt_qp_add(port_id, qp); + if (ret) + return ret; + return 0; } @@ -240,6 +246,12 @@ otx2_cpt_qp_create(const struct rte_cryptodev *dev, uint16_t qp_id, qp->lf_nq_reg = qp->base + OTX2_CPT_LF_NQ(0); + ret = otx2_sec_tx_cpt_qp_remove(qp); + if (ret && (ret != -ENOENT)) { + CPT_LOG_ERR("Could not delete inline configuration"); + goto mempool_destroy; + } + otx2_cpt_iq_disable(qp); ret = otx2_cpt_qp_inline_cfg(dev, qp); @@ -273,6 +285,12 @@ otx2_cpt_qp_destroy(const struct rte_cryptodev *dev, struct otx2_cpt_qp *qp) char name[RTE_MEMZONE_NAMESIZE]; int ret; + ret = otx2_sec_tx_cpt_qp_remove(qp); + if (ret && (ret != -ENOENT)) { + CPT_LOG_ERR("Could not delete inline configuration"); + return ret; + } + otx2_cpt_iq_disable(qp); otx2_cpt_metabuf_mempool_destroy(qp); diff --git a/drivers/crypto/octeontx2/otx2_cryptodev_qp.h b/drivers/crypto/octeontx2/otx2_cryptodev_qp.h new file mode 100644 index 0000000..caf8272 --- /dev/null +++ b/drivers/crypto/octeontx2/otx2_cryptodev_qp.h @@ -0,0 +1,35 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (C) 2019 Marvell International Ltd. + */ + +#ifndef _OTX2_CRYPTODEV_QP_H_ +#define _OTX2_CRYPTODEV_QP_H_ + +#include +#include +#include + +#include "cpt_common.h" + +struct otx2_cpt_qp { + uint32_t id; + /**< Queue pair id */ + uintptr_t base; + /**< Base address where BAR is mapped */ + void *lmtline; + /**< Address of LMTLINE */ + rte_iova_t lf_nq_reg; + /**< LF enqueue register address */ + struct pending_queue pend_q; + /**< Pending queue */ + struct rte_mempool *sess_mp; + /**< Session mempool */ + struct rte_mempool *sess_mp_priv; + /**< Session private data mempool */ + struct cpt_qp_meta_info meta_info; + /**< Metabuf info required to support operations on the queue pair */ + rte_iova_t iq_dma_addr; + /**< Instruction queue address */ +}; + +#endif /* _OTX2_CRYPTODEV_QP_H_ */ diff --git a/drivers/crypto/octeontx2/otx2_security.c b/drivers/crypto/octeontx2/otx2_security.c index 46b94e5..6e9063e 100644 --- a/drivers/crypto/octeontx2/otx2_security.c +++ b/drivers/crypto/octeontx2/otx2_security.c @@ -10,6 +10,7 @@ #include #include +#include "otx2_cryptodev_qp.h" #include "otx2_ethdev.h" #include "otx2_ipsec_fp.h" #include "otx2_security.h" @@ -29,6 +30,8 @@ struct sec_eth_tag_const { }; }; +static struct otx2_sec_eth_cfg sec_cfg[OTX2_MAX_INLINE_PORTS]; + static struct rte_cryptodev_capabilities otx2_sec_eth_crypto_caps[] = { { /* AES GCM */ .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, @@ -156,16 +159,41 @@ static struct rte_security_ops otx2_sec_eth_ops = { .capabilities_get = otx2_sec_eth_capabilities_get }; +static int +otx2_sec_eth_cfg_init(int port_id) +{ + struct otx2_sec_eth_cfg *cfg; + int i; + + cfg = &sec_cfg[port_id]; + cfg->tx_cpt_idx = 0; + rte_spinlock_init(&cfg->tx_cpt_lock); + + for (i = 0; i < OTX2_MAX_CPT_QP_PER_PORT; i++) { + cfg->tx_cpt[i].qp = NULL; + rte_atomic16_set(&cfg->tx_cpt[i].ref_cnt, 0); + } + + return 0; +} + int otx2_sec_eth_ctx_create(struct rte_eth_dev *eth_dev) { struct rte_security_ctx *ctx; + int ret; ctx = rte_malloc("otx2_sec_eth_ctx", sizeof(struct rte_security_ctx), 0); if (ctx == NULL) return -ENOMEM; + ret = otx2_sec_eth_cfg_init(eth_dev->data->port_id); + if (ret) { + rte_free(ctx); + return ret; + } + /* Populate ctx */ ctx->device = eth_dev; @@ -279,3 +307,73 @@ otx2_sec_eth_fini(struct rte_eth_dev *eth_dev) in_sa_mz_name_get(name, RTE_MEMZONE_NAMESIZE, port); rte_memzone_free(rte_memzone_lookup(name)); } + +int +otx2_sec_tx_cpt_qp_add(uint16_t port_id, struct otx2_cpt_qp *qp) +{ + struct otx2_sec_eth_cfg *cfg; + int i, ret; + + if (qp == NULL || port_id > OTX2_MAX_INLINE_PORTS) + return -EINVAL; + + cfg = &sec_cfg[port_id]; + + /* Find a free slot to save CPT LF */ + + rte_spinlock_lock(&cfg->tx_cpt_lock); + + for (i = 0; i < OTX2_MAX_CPT_QP_PER_PORT; i++) { + if (cfg->tx_cpt[i].qp == NULL) { + cfg->tx_cpt[i].qp = qp; + ret = 0; + goto unlock; + } + } + + ret = -EINVAL; + +unlock: + rte_spinlock_unlock(&cfg->tx_cpt_lock); + return ret; +} + +int +otx2_sec_tx_cpt_qp_remove(struct otx2_cpt_qp *qp) +{ + struct otx2_sec_eth_cfg *cfg; + uint16_t port_id; + int i, ret; + + if (qp == NULL) + return -EINVAL; + + for (port_id = 0; port_id < OTX2_MAX_INLINE_PORTS; port_id++) { + cfg = &sec_cfg[port_id]; + + rte_spinlock_lock(&cfg->tx_cpt_lock); + + for (i = 0; i < OTX2_MAX_CPT_QP_PER_PORT; i++) { + if (cfg->tx_cpt[i].qp != qp) + continue; + + /* Don't free if the QP is in use by any sec session */ + if (rte_atomic16_read(&cfg->tx_cpt[i].ref_cnt)) { + ret = -EBUSY; + } else { + cfg->tx_cpt[i].qp = NULL; + ret = 0; + } + + goto unlock; + } + + rte_spinlock_unlock(&cfg->tx_cpt_lock); + } + + return -ENOENT; + +unlock: + rte_spinlock_unlock(&cfg->tx_cpt_lock); + return ret; +} diff --git a/drivers/crypto/octeontx2/otx2_security.h b/drivers/crypto/octeontx2/otx2_security.h index f819f01..c8aca7b 100644 --- a/drivers/crypto/octeontx2/otx2_security.h +++ b/drivers/crypto/octeontx2/otx2_security.h @@ -5,10 +5,27 @@ #ifndef __OTX2_SECURITY_H__ #define __OTX2_SECURITY_H__ +#include #include +#include #include "otx2_ipsec_fp.h" +#define OTX2_MAX_CPT_QP_PER_PORT 64 +#define OTX2_MAX_INLINE_PORTS 64 + +struct otx2_cpt_qp; + +struct otx2_sec_eth_cfg { + struct { + struct otx2_cpt_qp *qp; + rte_atomic16_t ref_cnt; + } tx_cpt[OTX2_MAX_CPT_QP_PER_PORT]; + + uint16_t tx_cpt_idx; + rte_spinlock_t tx_cpt_lock; +}; + /* * Security session for inline IPsec protocol offload. This is private data of * inline capable PMD. @@ -33,4 +50,7 @@ int otx2_sec_eth_init(struct rte_eth_dev *eth_dev); void otx2_sec_eth_fini(struct rte_eth_dev *eth_dev); +int otx2_sec_tx_cpt_qp_add(uint16_t port_id, struct otx2_cpt_qp *qp); + +int otx2_sec_tx_cpt_qp_remove(struct otx2_cpt_qp *qp); #endif /* __OTX2_SECURITY_H__ */ From patchwork Sun Dec 8 11:54:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 63633 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 67B6AA04F1; Sun, 8 Dec 2019 12:56:52 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 40BA81BF74; Sun, 8 Dec 2019 12:56:25 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id E17D61BC25 for ; Sun, 8 Dec 2019 12:56:23 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB8BrP5G007317; Sun, 8 Dec 2019 03:56:23 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=EPrpOxShvtU/rB9Y9PO0fj0SZ6TolZcf1ZUAllk8elw=; b=AkK8ptyHzA/y769JIMpZYoVXvzUPp8ZXyyUP5Ar3ZT+JjgKQ9veXe+jtMZBIBrbu/5N1 eNgxZayksfhsXTYdayrHaLWwfuSWSawmZw3r3cO5kZ6MNSP9em5ObVWOg4ZOlFeMdEbH O/wADjQVsem8KR1MhxiEHTwdnY1U+WFuwulNt22nKbOG/RxWM0Q18E0s8BO9iMzn2ozt syRDQJSgjiu2d/GOSmngwc8iMNTcbfKSpmFANRGLiZqvAWZ33MXOmwmWtuuLDQ8o8ttn +27iZ8/OFRIuM+dcfJbXpBxTbhUEL5XkmfiCsS1mHSoAfrQajjVKFoeEUFELDchU69nF fg== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0b-0016f401.pphosted.com with ESMTP id 2wrcfptc21-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 08 Dec 2019 03:56:23 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Sun, 8 Dec 2019 03:56:21 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Sun, 8 Dec 2019 03:56:20 -0800 Received: from ajoseph83.caveonetworks.com.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id 134453F703F; Sun, 8 Dec 2019 03:56:15 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Declan Doherty , Thomas Monjalon CC: Anoob Joseph , Jerin Jacob , Narayana Prasad , Kiran Kumar K , Nithin Dabilpuram , "Pavan Nikhilesh" , Ankur Dwivedi , Archana Muniganti , Tejasree Kondoj , Vamsi Attunuru , "Lukasz Bartosik" , Date: Sun, 8 Dec 2019 17:24:47 +0530 Message-ID: <1575806094-28391-9-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1575806094-28391-1-git-send-email-anoobj@marvell.com> References: <1575806094-28391-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-08_03:2019-12-05,2019-12-08 signatures=0 Subject: [dpdk-dev] [PATCH 08/15] crypto/octeontx2: add eth security session operations X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Adding security session operations in eth security ctx. Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Archana Muniganti Signed-off-by: Tejasree Kondoj Signed-off-by: Vamsi Attunuru --- drivers/crypto/octeontx2/otx2_ipsec_fp.h | 293 ++++++++++++++++++++++++++ drivers/crypto/octeontx2/otx2_security.c | 340 +++++++++++++++++++++++++++++++ drivers/crypto/octeontx2/otx2_security.h | 23 ++- 3 files changed, 655 insertions(+), 1 deletion(-) diff --git a/drivers/crypto/octeontx2/otx2_ipsec_fp.h b/drivers/crypto/octeontx2/otx2_ipsec_fp.h index 520a3fe..be83c23 100644 --- a/drivers/crypto/octeontx2/otx2_ipsec_fp.h +++ b/drivers/crypto/octeontx2/otx2_ipsec_fp.h @@ -5,6 +5,67 @@ #ifndef __OTX2_IPSEC_FP_H__ #define __OTX2_IPSEC_FP_H__ +#include +#include + +enum { + OTX2_IPSEC_FP_SA_DIRECTION_INBOUND = 0, + OTX2_IPSEC_FP_SA_DIRECTION_OUTBOUND = 1, +}; + +enum { + OTX2_IPSEC_FP_SA_IP_VERSION_4 = 0, + OTX2_IPSEC_FP_SA_IP_VERSION_6 = 1, +}; + +enum { + OTX2_IPSEC_FP_SA_MODE_TRANSPORT = 0, + OTX2_IPSEC_FP_SA_MODE_TUNNEL = 1, +}; + +enum { + OTX2_IPSEC_FP_SA_PROTOCOL_AH = 0, + OTX2_IPSEC_FP_SA_PROTOCOL_ESP = 1, +}; + +enum { + OTX2_IPSEC_FP_SA_AES_KEY_LEN_128 = 1, + OTX2_IPSEC_FP_SA_AES_KEY_LEN_192 = 2, + OTX2_IPSEC_FP_SA_AES_KEY_LEN_256 = 3, +}; + +enum { + OTX2_IPSEC_FP_SA_ENC_NULL = 0, + OTX2_IPSEC_FP_SA_ENC_DES_CBC = 1, + OTX2_IPSEC_FP_SA_ENC_3DES_CBC = 2, + OTX2_IPSEC_FP_SA_ENC_AES_CBC = 3, + OTX2_IPSEC_FP_SA_ENC_AES_CTR = 4, + OTX2_IPSEC_FP_SA_ENC_AES_GCM = 5, + OTX2_IPSEC_FP_SA_ENC_AES_CCM = 6, +}; + +enum { + OTX2_IPSEC_FP_SA_AUTH_NULL = 0, + OTX2_IPSEC_FP_SA_AUTH_MD5 = 1, + OTX2_IPSEC_FP_SA_AUTH_SHA1 = 2, + OTX2_IPSEC_FP_SA_AUTH_SHA2_224 = 3, + OTX2_IPSEC_FP_SA_AUTH_SHA2_256 = 4, + OTX2_IPSEC_FP_SA_AUTH_SHA2_384 = 5, + OTX2_IPSEC_FP_SA_AUTH_SHA2_512 = 6, + OTX2_IPSEC_FP_SA_AUTH_AES_GMAC = 7, + OTX2_IPSEC_FP_SA_AUTH_AES_XCBC_128 = 8, +}; + +enum { + OTX2_IPSEC_FP_SA_FRAG_POST = 0, + OTX2_IPSEC_FP_SA_FRAG_PRE = 1, +}; + +enum { + OTX2_IPSEC_FP_SA_ENCAP_NONE = 0, + OTX2_IPSEC_FP_SA_ENCAP_UDP = 1, +}; + struct otx2_ipsec_fp_sa_ctl { rte_be32_t spi : 32; uint64_t exp_proto_inter_frag : 8; @@ -24,6 +85,26 @@ struct otx2_ipsec_fp_sa_ctl { uint64_t aes_key_len : 2; }; +struct otx2_ipsec_fp_out_sa { + /* w0 */ + struct otx2_ipsec_fp_sa_ctl ctl; + + /* w1 */ + uint8_t nonce[4]; + uint16_t udp_src; + uint16_t udp_dst; + + /* w2 */ + uint32_t ip_src; + uint32_t ip_dst; + + /* w3-w6 */ + uint8_t cipher_key[32]; + + /* w7-w12 */ + uint8_t hmac_key[48]; +}; + struct otx2_ipsec_fp_in_sa { /* w0 */ struct otx2_ipsec_fp_sa_ctl ctl; @@ -52,4 +133,216 @@ struct otx2_ipsec_fp_in_sa { uint64_t reserved2; }; +static inline int +ipsec_fp_xform_cipher_verify(struct rte_crypto_sym_xform *xform) +{ + if (xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CBC) { + switch (xform->cipher.key.length) { + case 16: + case 24: + case 32: + break; + default: + return -ENOTSUP; + } + return 0; + } + + return -ENOTSUP; +} + +static inline int +ipsec_fp_xform_auth_verify(struct rte_crypto_sym_xform *xform) +{ + if (xform->auth.algo == RTE_CRYPTO_AUTH_SHA1_HMAC) { + if (xform->auth.key.length == 64) + return 0; + } + + return -ENOTSUP; +} + +static inline int +ipsec_fp_xform_aead_verify(struct rte_security_ipsec_xform *ipsec, + struct rte_crypto_sym_xform *xform) +{ + if (ipsec->direction == RTE_SECURITY_IPSEC_SA_DIR_EGRESS && + xform->aead.op != RTE_CRYPTO_AEAD_OP_ENCRYPT) + return -EINVAL; + + if (ipsec->direction == RTE_SECURITY_IPSEC_SA_DIR_INGRESS && + xform->aead.op != RTE_CRYPTO_AEAD_OP_DECRYPT) + return -EINVAL; + + if (xform->aead.algo == RTE_CRYPTO_AEAD_AES_GCM) { + switch (xform->aead.key.length) { + case 16: + case 24: + case 32: + break; + default: + return -EINVAL; + } + return 0; + } + + return -ENOTSUP; +} + +static inline int +ipsec_fp_xform_verify(struct rte_security_ipsec_xform *ipsec, + struct rte_crypto_sym_xform *xform) +{ + struct rte_crypto_sym_xform *auth_xform, *cipher_xform; + int ret; + + if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) + return ipsec_fp_xform_aead_verify(ipsec, xform); + + if (xform->next == NULL) + return -EINVAL; + + if (ipsec->direction == RTE_SECURITY_IPSEC_SA_DIR_INGRESS) { + /* Ingress */ + if (xform->type != RTE_CRYPTO_SYM_XFORM_AUTH || + xform->next->type != RTE_CRYPTO_SYM_XFORM_CIPHER) + return -EINVAL; + auth_xform = xform; + cipher_xform = xform->next; + } else { + /* Egress */ + if (xform->type != RTE_CRYPTO_SYM_XFORM_CIPHER || + xform->next->type != RTE_CRYPTO_SYM_XFORM_AUTH) + return -EINVAL; + cipher_xform = xform; + auth_xform = xform->next; + } + + ret = ipsec_fp_xform_cipher_verify(cipher_xform); + if (ret) + return ret; + + ret = ipsec_fp_xform_auth_verify(auth_xform); + if (ret) + return ret; + + return 0; +} + +static inline int +ipsec_fp_sa_ctl_set(struct rte_security_ipsec_xform *ipsec, + struct rte_crypto_sym_xform *xform, + struct otx2_ipsec_fp_sa_ctl *ctl) +{ + struct rte_crypto_sym_xform *cipher_xform, *auth_xform; + int aes_key_len; + + if (ipsec->direction == RTE_SECURITY_IPSEC_SA_DIR_EGRESS) { + ctl->direction = OTX2_IPSEC_FP_SA_DIRECTION_OUTBOUND; + cipher_xform = xform; + auth_xform = xform->next; + } else if (ipsec->direction == RTE_SECURITY_IPSEC_SA_DIR_INGRESS) { + ctl->direction = OTX2_IPSEC_FP_SA_DIRECTION_INBOUND; + auth_xform = xform; + cipher_xform = xform->next; + } else { + return -EINVAL; + } + + if (ipsec->mode == RTE_SECURITY_IPSEC_SA_MODE_TUNNEL) { + if (ipsec->tunnel.type == RTE_SECURITY_IPSEC_TUNNEL_IPV4) + ctl->outer_ip_ver = OTX2_IPSEC_FP_SA_IP_VERSION_4; + else if (ipsec->tunnel.type == RTE_SECURITY_IPSEC_TUNNEL_IPV6) + ctl->outer_ip_ver = OTX2_IPSEC_FP_SA_IP_VERSION_6; + else + return -EINVAL; + } + + ctl->inner_ip_ver = OTX2_IPSEC_FP_SA_IP_VERSION_4; + + if (ipsec->mode == RTE_SECURITY_IPSEC_SA_MODE_TRANSPORT) + ctl->ipsec_mode = OTX2_IPSEC_FP_SA_MODE_TRANSPORT; + else if (ipsec->mode == RTE_SECURITY_IPSEC_SA_MODE_TUNNEL) + ctl->ipsec_mode = OTX2_IPSEC_FP_SA_MODE_TUNNEL; + else + return -EINVAL; + + if (ipsec->proto == RTE_SECURITY_IPSEC_SA_PROTO_AH) + ctl->ipsec_proto = OTX2_IPSEC_FP_SA_PROTOCOL_AH; + else if (ipsec->proto == RTE_SECURITY_IPSEC_SA_PROTO_ESP) + ctl->ipsec_proto = OTX2_IPSEC_FP_SA_PROTOCOL_ESP; + else + return -EINVAL; + + if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) { + if (xform->aead.algo == RTE_CRYPTO_AEAD_AES_GCM) { + ctl->enc_type = OTX2_IPSEC_FP_SA_ENC_AES_GCM; + aes_key_len = xform->aead.key.length; + } else { + return -ENOTSUP; + } + } else if (cipher_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CBC) { + ctl->enc_type = OTX2_IPSEC_FP_SA_ENC_AES_CCM; + aes_key_len = xform->cipher.key.length; + } else { + return -ENOTSUP; + } + + switch (aes_key_len) { + case 16: + ctl->aes_key_len = OTX2_IPSEC_FP_SA_AES_KEY_LEN_128; + break; + case 24: + ctl->aes_key_len = OTX2_IPSEC_FP_SA_AES_KEY_LEN_192; + break; + case 32: + ctl->aes_key_len = OTX2_IPSEC_FP_SA_AES_KEY_LEN_256; + break; + default: + return -EINVAL; + } + + if (xform->type != RTE_CRYPTO_SYM_XFORM_AEAD) { + switch (auth_xform->auth.algo) { + case RTE_CRYPTO_AUTH_NULL: + ctl->auth_type = OTX2_IPSEC_FP_SA_AUTH_NULL; + break; + case RTE_CRYPTO_AUTH_MD5_HMAC: + ctl->auth_type = OTX2_IPSEC_FP_SA_AUTH_MD5; + break; + case RTE_CRYPTO_AUTH_SHA1_HMAC: + ctl->auth_type = OTX2_IPSEC_FP_SA_AUTH_SHA1; + break; + case RTE_CRYPTO_AUTH_SHA224_HMAC: + ctl->auth_type = OTX2_IPSEC_FP_SA_AUTH_SHA2_224; + break; + case RTE_CRYPTO_AUTH_SHA256_HMAC: + ctl->auth_type = OTX2_IPSEC_FP_SA_AUTH_SHA2_256; + break; + case RTE_CRYPTO_AUTH_SHA384_HMAC: + ctl->auth_type = OTX2_IPSEC_FP_SA_AUTH_SHA2_384; + break; + case RTE_CRYPTO_AUTH_SHA512_HMAC: + ctl->auth_type = OTX2_IPSEC_FP_SA_AUTH_SHA2_512; + break; + case RTE_CRYPTO_AUTH_AES_GMAC: + ctl->auth_type = OTX2_IPSEC_FP_SA_AUTH_AES_GMAC; + break; + case RTE_CRYPTO_AUTH_AES_XCBC_MAC: + ctl->auth_type = OTX2_IPSEC_FP_SA_AUTH_AES_XCBC_128; + break; + default: + return -ENOTSUP; + } + } + + if (ipsec->options.esn == 1) + ctl->esn_en = 1; + + ctl->spi = rte_cpu_to_be_32(ipsec->spi); + ctl->valid = 1; + + return 0; +} + #endif /* __OTX2_IPSEC_FP_H__ */ diff --git a/drivers/crypto/octeontx2/otx2_security.c b/drivers/crypto/octeontx2/otx2_security.c index 6e9063e..393bc12 100644 --- a/drivers/crypto/octeontx2/otx2_security.c +++ b/drivers/crypto/octeontx2/otx2_security.c @@ -136,12 +136,350 @@ static const struct rte_security_capability otx2_sec_eth_capabilities[] = { } }; +static int +otx2_sec_eth_tx_cpt_qp_get(uint16_t port_id, struct otx2_cpt_qp **qp) +{ + struct otx2_sec_eth_cfg *cfg; + uint16_t index; + int i, ret; + + if (port_id > OTX2_MAX_INLINE_PORTS || qp == NULL) + return -EINVAL; + + cfg = &sec_cfg[port_id]; + + rte_spinlock_lock(&cfg->tx_cpt_lock); + + index = cfg->tx_cpt_idx; + + /* Get the next index with valid data */ + for (i = 0; i < OTX2_MAX_CPT_QP_PER_PORT; i++) { + if (cfg->tx_cpt[index].qp != NULL) + break; + index = (index + 1) % OTX2_MAX_CPT_QP_PER_PORT; + } + + if (i >= OTX2_MAX_CPT_QP_PER_PORT) { + ret = -EINVAL; + goto unlock; + } + + *qp = cfg->tx_cpt[index].qp; + rte_atomic16_inc(&cfg->tx_cpt[index].ref_cnt); + + cfg->tx_cpt_idx = (index + 1) % OTX2_MAX_CPT_QP_PER_PORT; + + ret = 0; + +unlock: + rte_spinlock_unlock(&cfg->tx_cpt_lock); + return ret; +} + +static int +otx2_sec_eth_tx_cpt_put(struct otx2_cpt_qp *qp) +{ + struct otx2_sec_eth_cfg *cfg; + uint16_t port_id; + int i; + + if (qp == NULL) + return -EINVAL; + + for (port_id = 0; port_id < OTX2_MAX_INLINE_PORTS; port_id++) { + cfg = &sec_cfg[port_id]; + for (i = 0; i < OTX2_MAX_CPT_QP_PER_PORT; i++) { + if (cfg->tx_cpt[i].qp == qp) { + rte_atomic16_dec(&cfg->tx_cpt[i].ref_cnt); + return 0; + } + } + } + + return -EINVAL; +} + static inline void in_sa_mz_name_get(char *name, int size, uint16_t port) { snprintf(name, size, "otx2_ipsec_in_sadb_%u", port); } +static struct otx2_ipsec_fp_in_sa * +in_sa_get(uint16_t port, int sa_index) +{ + char name[RTE_MEMZONE_NAMESIZE]; + struct otx2_ipsec_fp_in_sa *sa; + const struct rte_memzone *mz; + + in_sa_mz_name_get(name, RTE_MEMZONE_NAMESIZE, port); + mz = rte_memzone_lookup(name); + if (mz == NULL) { + otx2_err("Could not get the memzone reserved for IN SA DB"); + return NULL; + } + + sa = mz->addr; + + return sa + sa_index; +} + +static int +sec_eth_ipsec_out_sess_create(struct rte_eth_dev *eth_dev, + struct rte_security_ipsec_xform *ipsec, + struct rte_crypto_sym_xform *crypto_xform, + struct rte_security_session *sec_sess) +{ + struct rte_crypto_sym_xform *auth_xform, *cipher_xform; + struct otx2_sec_session_ipsec_ip *sess; + uint16_t port = eth_dev->data->port_id; + int cipher_key_len, auth_key_len, ret; + const uint8_t *cipher_key, *auth_key; + struct otx2_ipsec_fp_sa_ctl *ctl; + struct otx2_ipsec_fp_out_sa *sa; + struct otx2_sec_session *priv; + struct otx2_cpt_qp *qp; + + priv = get_sec_session_private_data(sec_sess); + sess = &priv->ipsec.ip; + + sa = &sess->out_sa; + ctl = &sa->ctl; + if (ctl->valid) { + otx2_err("SA already registered"); + return -EINVAL; + } + + memset(sess, 0, sizeof(struct otx2_sec_session_ipsec_ip)); + + memcpy(sa->nonce, &ipsec->salt, 4); + + if (ipsec->options.udp_encap == 1) { + sa->udp_src = 4500; + sa->udp_dst = 4500; + } + + if (ipsec->mode == RTE_SECURITY_IPSEC_SA_MODE_TUNNEL) { + if (ipsec->tunnel.type == RTE_SECURITY_IPSEC_TUNNEL_IPV4) { + memcpy(&sa->ip_src, &ipsec->tunnel.ipv4.src_ip, + sizeof(struct in_addr)); + memcpy(&sa->ip_dst, &ipsec->tunnel.ipv4.dst_ip, + sizeof(struct in_addr)); + } else { + return -EINVAL; + } + } else { + return -EINVAL; + } + + cipher_xform = crypto_xform; + auth_xform = crypto_xform->next; + + cipher_key_len = 0; + auth_key_len = 0; + + if (crypto_xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) { + cipher_key = crypto_xform->aead.key.data; + cipher_key_len = crypto_xform->aead.key.length; + } else { + cipher_key = cipher_xform->cipher.key.data; + cipher_key_len = cipher_xform->cipher.key.length; + auth_key = auth_xform->auth.key.data; + auth_key_len = auth_xform->auth.key.length; + } + + if (cipher_key_len != 0) + memcpy(sa->cipher_key, cipher_key, cipher_key_len); + else + return -EINVAL; + + /* Use OPAD & IPAD */ + RTE_SET_USED(auth_key); + RTE_SET_USED(auth_key_len); + + /* Get CPT QP to be used for this SA */ + ret = otx2_sec_eth_tx_cpt_qp_get(port, &qp); + if (ret) + return ret; + + sess->qp = qp; + + sess->cpt_lmtline = qp->lmtline; + sess->cpt_nq_reg = qp->lf_nq_reg; + + /* Populate control word */ + ret = ipsec_fp_sa_ctl_set(ipsec, crypto_xform, ctl); + if (ret) + goto cpt_put; + + return 0; +cpt_put: + otx2_sec_eth_tx_cpt_put(sess->qp); + return ret; +} + +static int +sec_eth_ipsec_in_sess_create(struct rte_eth_dev *eth_dev, + struct rte_security_ipsec_xform *ipsec, + struct rte_crypto_sym_xform *crypto_xform, + struct rte_security_session *sec_sess) +{ + struct rte_crypto_sym_xform *auth_xform, *cipher_xform; + struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev); + struct otx2_sec_session_ipsec_ip *sess; + uint16_t port = eth_dev->data->port_id; + const uint8_t *cipher_key, *auth_key; + int cipher_key_len, auth_key_len; + struct otx2_ipsec_fp_sa_ctl *ctl; + struct otx2_ipsec_fp_in_sa *sa; + struct otx2_sec_session *priv; + + if (ipsec->spi >= dev->ipsec_in_max_spi) { + otx2_err("SPI exceeds max supported"); + return -EINVAL; + } + + sa = in_sa_get(port, ipsec->spi); + ctl = &sa->ctl; + + priv = get_sec_session_private_data(sec_sess); + sess = &priv->ipsec.ip; + + if (ctl->valid) { + otx2_err("SA already registered"); + return -EINVAL; + } + + memset(sa, 0, sizeof(struct otx2_ipsec_fp_in_sa)); + + auth_xform = crypto_xform; + cipher_xform = crypto_xform->next; + + cipher_key_len = 0; + auth_key_len = 0; + + if (crypto_xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) { + if (crypto_xform->aead.algo == RTE_CRYPTO_AEAD_AES_GCM) + memcpy(sa->nonce, &ipsec->salt, 4); + cipher_key = crypto_xform->aead.key.data; + cipher_key_len = crypto_xform->aead.key.length; + } else { + cipher_key = cipher_xform->cipher.key.data; + cipher_key_len = cipher_xform->cipher.key.length; + auth_key = auth_xform->auth.key.data; + auth_key_len = auth_xform->auth.key.length; + } + + if (cipher_key_len != 0) + memcpy(sa->cipher_key, cipher_key, cipher_key_len); + else + return -EINVAL; + + /* Use OPAD & IPAD */ + RTE_SET_USED(auth_key); + RTE_SET_USED(auth_key_len); + + sess->in_sa = sa; + + sa->userdata = priv->userdata; + + return ipsec_fp_sa_ctl_set(ipsec, crypto_xform, ctl); + +} + +static int +sec_eth_ipsec_sess_create(struct rte_eth_dev *eth_dev, + struct rte_security_ipsec_xform *ipsec, + struct rte_crypto_sym_xform *crypto_xform, + struct rte_security_session *sess) +{ + int ret; + + ret = ipsec_fp_xform_verify(ipsec, crypto_xform); + if (ret) + return ret; + + if (ipsec->direction == RTE_SECURITY_IPSEC_SA_DIR_INGRESS) + return sec_eth_ipsec_in_sess_create(eth_dev, ipsec, + crypto_xform, sess); + else + return sec_eth_ipsec_out_sess_create(eth_dev, ipsec, + crypto_xform, sess); +} + +static int +otx2_sec_eth_session_create(void *device, + struct rte_security_session_conf *conf, + struct rte_security_session *sess, + struct rte_mempool *mempool) +{ + struct otx2_sec_session *priv; + int ret; + + if (conf->action_type != RTE_SECURITY_ACTION_TYPE_INLINE_PROTOCOL) + return -ENOTSUP; + + if (rte_mempool_get(mempool, (void **)&priv)) { + otx2_err("Could not allocate security session private data"); + return -ENOMEM; + } + + set_sec_session_private_data(sess, priv); + + /* + * Save userdata provided by the application. For ingress packets, this + * could be used to identify the SA. + */ + priv->userdata = conf->userdata; + + if (conf->protocol == RTE_SECURITY_PROTOCOL_IPSEC) + ret = sec_eth_ipsec_sess_create(device, &conf->ipsec, + conf->crypto_xform, + sess); + else + ret = -ENOTSUP; + + if (ret) + goto mempool_put; + + return 0; + +mempool_put: + rte_mempool_put(mempool, priv); + set_sec_session_private_data(sess, NULL); + return ret; +} + +static int +otx2_sec_eth_session_destroy(void *device __rte_unused, + struct rte_security_session *sess) +{ + struct otx2_sec_session_ipsec_ip *sess_ip; + struct otx2_sec_session *priv; + struct rte_mempool *sess_mp; + int ret; + + priv = get_sec_session_private_data(sess); + if (priv == NULL) + return -EINVAL; + + sess_ip = &priv->ipsec.ip; + + /* Release CPT LF used for this session */ + if (sess_ip->qp != NULL) { + ret = otx2_sec_eth_tx_cpt_put(sess_ip->qp); + if (ret) + return ret; + } + + sess_mp = rte_mempool_from_obj(priv); + + set_sec_session_private_data(sess, NULL); + rte_mempool_put(sess_mp, priv); + + return 0; +} + static unsigned int otx2_sec_eth_session_get_size(void *device __rte_unused) { @@ -155,6 +493,8 @@ otx2_sec_eth_capabilities_get(void *device __rte_unused) } static struct rte_security_ops otx2_sec_eth_ops = { + .session_create = otx2_sec_eth_session_create, + .session_destroy = otx2_sec_eth_session_destroy, .session_get_size = otx2_sec_eth_session_get_size, .capabilities_get = otx2_sec_eth_capabilities_get }; diff --git a/drivers/crypto/octeontx2/otx2_security.h b/drivers/crypto/octeontx2/otx2_security.h index c8aca7b..adca00b 100644 --- a/drivers/crypto/octeontx2/otx2_security.h +++ b/drivers/crypto/octeontx2/otx2_security.h @@ -31,7 +31,26 @@ struct otx2_sec_eth_cfg { * inline capable PMD. */ struct otx2_sec_session_ipsec_ip { - int dummy; + RTE_STD_C11 + union { + /* + * Inbound SA would accessed by crypto block. And so the memory + * is allocated differently and shared with the h/w. Only + * holding a pointer to this memory in the session private + * space. + */ + void *in_sa; + /* Outbound SA */ + struct otx2_ipsec_fp_out_sa out_sa; + }; + + /* Address of CPT LMTLINE */ + void *cpt_lmtline; + /* CPT LF enqueue register address */ + rte_iova_t cpt_nq_reg; + + /* CPT QP used by SA */ + struct otx2_cpt_qp *qp; }; struct otx2_sec_session_ipsec { @@ -40,6 +59,8 @@ struct otx2_sec_session_ipsec { struct otx2_sec_session { struct otx2_sec_session_ipsec ipsec; + void *userdata; + /**< Userdata registered by the application */ } __rte_cache_aligned; int otx2_sec_eth_ctx_create(struct rte_eth_dev *eth_dev); From patchwork Sun Dec 8 11:54:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 63634 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 276F3A04F1; Sun, 8 Dec 2019 12:57:07 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 395941BF6D; Sun, 8 Dec 2019 12:56:31 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id EE4C8374E for ; Sun, 8 Dec 2019 12:56:29 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB8Bq8Th005638; Sun, 8 Dec 2019 03:56:29 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=nyhCD/pkdMGHzYiM1obT2VQM87rAsBfzPTPHTB0JYJ0=; b=CTy/0B+pTtqZZ1qKiD0tdP3X3XLfsQmgry8Xo7K/NMIOHjMKOWC6oLS4zB4S2ZRr6ZN/ J2orhF69JSz6zEtye8h1WnJ1t85aAN+M2WgRAkc6/pXptlEJWmFqYQezsmCtHsyn3o4z Is8IMYEpV9broGoL4qqlkXAVxTsNsB8FRfu1hDvvTk76N1FZgHnFMkOoFhCm+n7UvDk7 xBCxqjJNcJ/FHeVdJc7Mk2swE0WkZm9kqKGKLLpSWu4/p6bHwHMc3pef9uO3YJUMHrcu z2JTDNBSZwdh+Ihmor6cerh2mewqFqcxvSPI2iC25awkY+5BXVJSq76S65aSABF/+Jwe cw== Received: from sc-exch01.marvell.com ([199.233.58.181]) by mx0b-0016f401.pphosted.com with ESMTP id 2wrcfptc26-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 08 Dec 2019 03:56:29 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Sun, 8 Dec 2019 03:56:27 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Sun, 8 Dec 2019 03:56:27 -0800 Received: from ajoseph83.caveonetworks.com.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id 7E64A3F703F; Sun, 8 Dec 2019 03:56:22 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Declan Doherty , Thomas Monjalon CC: Ankur Dwivedi , Jerin Jacob , Narayana Prasad , Kiran Kumar K , Nithin Dabilpuram , Pavan Nikhilesh , Anoob Joseph , Archana Muniganti , "Tejasree Kondoj" , Vamsi Attunuru , Lukasz Bartosik , Date: Sun, 8 Dec 2019 17:24:48 +0530 Message-ID: <1575806094-28391-10-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1575806094-28391-1-git-send-email-anoobj@marvell.com> References: <1575806094-28391-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-08_03:2019-12-05,2019-12-08 signatures=0 Subject: [dpdk-dev] [PATCH 09/15] crypto/octeontx2: add datapath ops in eth security ctx X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Ankur Dwivedi Adding data path ops in eth security ctx. Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Archana Muniganti Signed-off-by: Tejasree Kondoj Signed-off-by: Vamsi Attunuru --- drivers/crypto/octeontx2/otx2_security.c | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/drivers/crypto/octeontx2/otx2_security.c b/drivers/crypto/octeontx2/otx2_security.c index 393bc12..545c806 100644 --- a/drivers/crypto/octeontx2/otx2_security.c +++ b/drivers/crypto/octeontx2/otx2_security.c @@ -486,6 +486,27 @@ otx2_sec_eth_session_get_size(void *device __rte_unused) return sizeof(struct otx2_sec_session); } +static int +otx2_sec_eth_set_pkt_mdata(void *device __rte_unused, + struct rte_security_session *session, + struct rte_mbuf *m, void *params __rte_unused) +{ + /* Set security session as the pkt metadata */ + m->udata64 = (uint64_t)session; + + return 0; +} + +static int +otx2_sec_eth_get_userdata(void *device __rte_unused, uint64_t md, + void **userdata) +{ + /* Retrieve userdata */ + *userdata = (void *)md; + + return 0; +} + static const struct rte_security_capability * otx2_sec_eth_capabilities_get(void *device __rte_unused) { @@ -496,6 +517,8 @@ static struct rte_security_ops otx2_sec_eth_ops = { .session_create = otx2_sec_eth_session_create, .session_destroy = otx2_sec_eth_session_destroy, .session_get_size = otx2_sec_eth_session_get_size, + .set_pkt_metadata = otx2_sec_eth_set_pkt_mdata, + .get_userdata = otx2_sec_eth_get_userdata, .capabilities_get = otx2_sec_eth_capabilities_get }; From patchwork Sun Dec 8 11:54:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 63635 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7C4C2A04F1; Sun, 8 Dec 2019 12:57:16 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 9711C1BF6E; Sun, 8 Dec 2019 12:56:37 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id F320C1BE82 for ; Sun, 8 Dec 2019 12:56:35 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB8BrTmX008036; Sun, 8 Dec 2019 03:56:35 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=n48SzD5Eo/rrzOk0Uf9wGA/bfBj+vcHZ/DMLKfUVF0c=; b=EMhoXdg96kPV42ZqBdvYqmhEH8OhBG7kgLGAnRfFNi1/K9AzJFm/GthfTwq1g7uEyR0o QA/xZI25AjxumU4qGK4za48BfVF9RNNotG1XaJHi6PKdbLSiZYy8+eyua7age199AS11 T02IlzI6agyYOTYf2WZnMQd6jOeFt3MedBVrSfHLe4WbPi6YUo1XUU4DA4LZvi35Id6Q e2oVvE//mKxsm2mR31l0JffatpI6Bj8+wfClmzQclF39ZF6G1PYEiV61HldKXnreMfco GyjrwuJKiOjNcP7Vk1J1PElZCR1W+cVaH8H16DITqdwBxgd8wQNedCoSsyi4WyEfOlij nA== Received: from sc-exch02.marvell.com ([199.233.58.182]) by mx0a-0016f401.pphosted.com with ESMTP id 2wrbawjj3w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 08 Dec 2019 03:56:35 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Sun, 8 Dec 2019 03:56:33 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Sun, 8 Dec 2019 03:56:33 -0800 Received: from ajoseph83.caveonetworks.com.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id ED6B73F703F; Sun, 8 Dec 2019 03:56:28 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Declan Doherty , Thomas Monjalon CC: Archana Muniganti , Jerin Jacob , Narayana Prasad , Kiran Kumar K , Nithin Dabilpuram , "Pavan Nikhilesh" , Ankur Dwivedi , Anoob Joseph , Tejasree Kondoj , Vamsi Attunuru , Lukasz Bartosik , Date: Sun, 8 Dec 2019 17:24:49 +0530 Message-ID: <1575806094-28391-11-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1575806094-28391-1-git-send-email-anoobj@marvell.com> References: <1575806094-28391-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-08_03:2019-12-05,2019-12-08 signatures=0 Subject: [dpdk-dev] [PATCH 10/15] crypto/octeontx2: add lookup mem changes to hold sa indices X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Archana Muniganti lookup_mem provides fast accessing of data path fields. Storing sa indices in lookup_mem which are required in inline rx data path. Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Archana Muniganti Signed-off-by: Tejasree Kondoj Signed-off-by: Vamsi Attunuru --- drivers/common/octeontx2/otx2_common.h | 20 +++++++++++ drivers/crypto/octeontx2/otx2_security.c | 59 +++++++++++++++++++++++++++++++- drivers/net/octeontx2/otx2_lookup.c | 9 +---- drivers/net/octeontx2/otx2_rx.h | 10 ++---- 4 files changed, 81 insertions(+), 17 deletions(-) diff --git a/drivers/common/octeontx2/otx2_common.h b/drivers/common/octeontx2/otx2_common.h index 9705a8d..6456c4b 100644 --- a/drivers/common/octeontx2/otx2_common.h +++ b/drivers/common/octeontx2/otx2_common.h @@ -170,4 +170,24 @@ extern int otx2_logtype_dpi; #include "otx2_io_generic.h" #endif +/* Fastpath lookup */ +#define OTX2_NIX_FASTPATH_LOOKUP_MEM "otx2_nix_fastpath_lookup_mem" +#define PTYPE_NON_TUNNEL_WIDTH 16 +#define PTYPE_TUNNEL_WIDTH 12 +#define PTYPE_NON_TUNNEL_ARRAY_SZ BIT(PTYPE_NON_TUNNEL_WIDTH) +#define PTYPE_TUNNEL_ARRAY_SZ BIT(PTYPE_TUNNEL_WIDTH) +#define PTYPE_ARRAY_SZ ((PTYPE_NON_TUNNEL_ARRAY_SZ +\ + PTYPE_TUNNEL_ARRAY_SZ) *\ + sizeof(uint16_t)) + +/* NIX_RX_PARSE_S's ERRCODE + ERRLEV (12 bits) */ +#define ERRCODE_ERRLEN_WIDTH 12 +#define ERR_ARRAY_SZ ((BIT(ERRCODE_ERRLEN_WIDTH)) *\ + sizeof(uint32_t)) + +#define PORT_ARRAY_SZ (RTE_MAX_ETHPORTS * sizeof(uint64_t)) + +#define LOOKUP_ARRAY_SZ (PTYPE_ARRAY_SZ + ERR_ARRAY_SZ +\ + PORT_ARRAY_SZ) + #endif /* _OTX2_COMMON_H_ */ diff --git a/drivers/crypto/octeontx2/otx2_security.c b/drivers/crypto/octeontx2/otx2_security.c index 545c806..4d762d9 100644 --- a/drivers/crypto/octeontx2/otx2_security.c +++ b/drivers/crypto/octeontx2/otx2_security.c @@ -136,6 +136,59 @@ static const struct rte_security_capability otx2_sec_eth_capabilities[] = { } }; +static void +lookup_mem_sa_tbl_clear(struct rte_eth_dev *eth_dev) +{ + static const char name[] = OTX2_NIX_FASTPATH_LOOKUP_MEM; + uint16_t port = eth_dev->data->port_id; + const struct rte_memzone *mz; + uint64_t **sa_tbl; + uint8_t *mem; + + mz = rte_memzone_lookup(name); + if (mz == NULL) + return; + + mem = mz->addr; + + sa_tbl = (uint64_t **)(mem + PTYPE_ARRAY_SZ + ERR_ARRAY_SZ); + if (sa_tbl[port] == NULL) + return; + + rte_free(sa_tbl[port]); + sa_tbl[port] = NULL; +} + +static int +lookup_mem_sa_index_update(struct rte_eth_dev *eth_dev, int spi, void *sa) +{ + static const char name[] = OTX2_NIX_FASTPATH_LOOKUP_MEM; + struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev); + uint16_t port = eth_dev->data->port_id; + const struct rte_memzone *mz; + uint64_t **sa_tbl; + uint8_t *mem; + + mz = rte_memzone_lookup(name); + if (mz == NULL) { + otx2_err("Could not find fastpath lookup table"); + return -EINVAL; + } + + mem = mz->addr; + + sa_tbl = (uint64_t **)(mem + PTYPE_ARRAY_SZ + ERR_ARRAY_SZ); + + if (sa_tbl[port] == NULL) { + sa_tbl[port] = rte_malloc(NULL, dev->ipsec_in_max_spi * + sizeof(uint64_t), 0); + } + + sa_tbl[port][spi] = (uint64_t)sa; + + return 0; +} + static int otx2_sec_eth_tx_cpt_qp_get(uint16_t port_id, struct otx2_cpt_qp **qp) { @@ -383,8 +436,10 @@ sec_eth_ipsec_in_sess_create(struct rte_eth_dev *eth_dev, sa->userdata = priv->userdata; - return ipsec_fp_sa_ctl_set(ipsec, crypto_xform, ctl); + if (lookup_mem_sa_index_update(eth_dev, ipsec->spi, sa)) + return -EINVAL; + return ipsec_fp_sa_ctl_set(ipsec, crypto_xform, ctl); } static int @@ -667,6 +722,8 @@ otx2_sec_eth_fini(struct rte_eth_dev *eth_dev) !(dev->rx_offloads & DEV_RX_OFFLOAD_SECURITY)) return; + lookup_mem_sa_tbl_clear(eth_dev); + in_sa_mz_name_get(name, RTE_MEMZONE_NAMESIZE, port); rte_memzone_free(rte_memzone_lookup(name)); } diff --git a/drivers/net/octeontx2/otx2_lookup.c b/drivers/net/octeontx2/otx2_lookup.c index bcf2ff4..46fdbc8 100644 --- a/drivers/net/octeontx2/otx2_lookup.c +++ b/drivers/net/octeontx2/otx2_lookup.c @@ -7,13 +7,6 @@ #include "otx2_ethdev.h" -/* NIX_RX_PARSE_S's ERRCODE + ERRLEV (12 bits) */ -#define ERRCODE_ERRLEN_WIDTH 12 -#define ERR_ARRAY_SZ ((BIT(ERRCODE_ERRLEN_WIDTH)) *\ - sizeof(uint32_t)) - -#define LOOKUP_ARRAY_SZ (PTYPE_ARRAY_SZ + ERR_ARRAY_SZ) - const uint32_t * otx2_nix_supported_ptypes_get(struct rte_eth_dev *eth_dev) { @@ -314,7 +307,7 @@ nix_create_rx_ol_flags_array(void *mem) void * otx2_nix_fastpath_lookup_mem_get(void) { - const char name[] = "otx2_nix_fastpath_lookup_mem"; + const char name[] = OTX2_NIX_FASTPATH_LOOKUP_MEM; const struct rte_memzone *mz; void *mem; diff --git a/drivers/net/octeontx2/otx2_rx.h b/drivers/net/octeontx2/otx2_rx.h index 351ad0f..5e1d5a2 100644 --- a/drivers/net/octeontx2/otx2_rx.h +++ b/drivers/net/octeontx2/otx2_rx.h @@ -5,17 +5,11 @@ #ifndef __OTX2_RX_H__ #define __OTX2_RX_H__ +#include "otx2_common.h" + /* Default mark value used when none is provided. */ #define OTX2_FLOW_ACTION_FLAG_DEFAULT 0xffff -#define PTYPE_NON_TUNNEL_WIDTH 16 -#define PTYPE_TUNNEL_WIDTH 12 -#define PTYPE_NON_TUNNEL_ARRAY_SZ BIT(PTYPE_NON_TUNNEL_WIDTH) -#define PTYPE_TUNNEL_ARRAY_SZ BIT(PTYPE_TUNNEL_WIDTH) -#define PTYPE_ARRAY_SZ ((PTYPE_NON_TUNNEL_ARRAY_SZ +\ - PTYPE_TUNNEL_ARRAY_SZ) *\ - sizeof(uint16_t)) - #define NIX_RX_OFFLOAD_NONE (0) #define NIX_RX_OFFLOAD_RSS_F BIT(0) #define NIX_RX_OFFLOAD_PTYPE_F BIT(1) From patchwork Sun Dec 8 11:54:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 63636 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id E1117A04F1; Sun, 8 Dec 2019 12:57:25 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 16B3E37B0; Sun, 8 Dec 2019 12:56:44 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id D3DA3374E for ; Sun, 8 Dec 2019 12:56:42 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB8BqojI006663; Sun, 8 Dec 2019 03:56:42 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=5kxuU6VesmZBcNXx0gOLwfKrbguENbImVlKPfTQyTO4=; b=fNXhMKNP43+s7G7D8siNa9GMZ7x7Bc36uO3Mw3S1LgbGITSZJobGkpLVdCznBOeMTs2s oXE/w4zQVGET/cAB+vzq9FzPjbm6pk/TN+SH5vPoqBiwImQgZ1/yhECksxaH5SLRztKG yhHDcmFgUvW5sqxrUEw0yaD1ez6MfHZhxetJ4Msk2JzPBTmmM/SjqH3k1WeuydB2KyNQ 5PBgW/fp3d2XY6/ULc0Asqqu1bgQODNp3TNsYhkgsd7pCpRQZogyrMpFntkEfKvMRqOm 3mvPCiIyaTJdHMW2SqcnQ4ZEyc24ei5+4NW5YHV6BAzYe4f8+OOgsdosvp9qchN8pCAi XA== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0b-0016f401.pphosted.com with ESMTP id 2wrcfptc2p-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 08 Dec 2019 03:56:42 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Sun, 8 Dec 2019 03:56:40 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Sun, 8 Dec 2019 03:56:40 -0800 Received: from ajoseph83.caveonetworks.com.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id 353603F703F; Sun, 8 Dec 2019 03:56:34 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Declan Doherty , Thomas Monjalon CC: Tejasree Kondoj , Jerin Jacob , Narayana Prasad , Kiran Kumar K , Nithin Dabilpuram , "Pavan Nikhilesh" , Ankur Dwivedi , Anoob Joseph , Archana Muniganti , Vamsi Attunuru , Lukasz Bartosik , Date: Sun, 8 Dec 2019 17:24:50 +0530 Message-ID: <1575806094-28391-12-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1575806094-28391-1-git-send-email-anoobj@marvell.com> References: <1575806094-28391-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-08_03:2019-12-05,2019-12-08 signatures=0 Subject: [dpdk-dev] [PATCH 11/15] net/octeontx2: add inline ipsec rx path changes X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Tejasree Kondoj Adding post-processing required for inline IPsec inbound packets. Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Archana Muniganti Signed-off-by: Tejasree Kondoj Signed-off-by: Vamsi Attunuru --- drivers/crypto/octeontx2/Makefile | 1 + drivers/crypto/octeontx2/otx2_security.h | 19 +++++++++ drivers/event/octeontx2/Makefile | 1 + drivers/event/octeontx2/meson.build | 2 + drivers/net/octeontx2/Makefile | 1 + drivers/net/octeontx2/meson.build | 3 ++ drivers/net/octeontx2/otx2_rx.h | 72 ++++++++++++++++++++++++++++++++ 7 files changed, 99 insertions(+) diff --git a/drivers/crypto/octeontx2/Makefile b/drivers/crypto/octeontx2/Makefile index 5966ddc..62b630e 100644 --- a/drivers/crypto/octeontx2/Makefile +++ b/drivers/crypto/octeontx2/Makefile @@ -20,6 +20,7 @@ VPATH += $(RTE_SDK)/drivers/crypto/octeontx2 CFLAGS += -O3 CFLAGS += -I$(RTE_SDK)/drivers/common/cpt CFLAGS += -I$(RTE_SDK)/drivers/common/octeontx2 +CFLAGS += -I$(RTE_SDK)/drivers/crypto/octeontx2 CFLAGS += -I$(RTE_SDK)/drivers/mempool/octeontx2 CFLAGS += -I$(RTE_SDK)/drivers/net/octeontx2 CFLAGS += -DALLOW_EXPERIMENTAL_API diff --git a/drivers/crypto/octeontx2/otx2_security.h b/drivers/crypto/octeontx2/otx2_security.h index adca00b..1229685 100644 --- a/drivers/crypto/octeontx2/otx2_security.h +++ b/drivers/crypto/octeontx2/otx2_security.h @@ -26,6 +26,25 @@ struct otx2_sec_eth_cfg { rte_spinlock_t tx_cpt_lock; }; +#define OTX2_SEC_CPT_COMP_GOOD 0x1 +#define OTX2_SEC_UC_COMP_GOOD 0x0 +#define OTX2_SEC_COMP_GOOD (OTX2_SEC_UC_COMP_GOOD << 8 | \ + OTX2_SEC_CPT_COMP_GOOD) + +/* CPT Result */ +struct otx2_cpt_res { + union { + struct { + uint64_t compcode:8; + uint64_t uc_compcode:8; + uint64_t doneint:1; + uint64_t reserved_17_63:47; + uint64_t reserved_64_127; + }; + uint16_t u16[8]; + }; +}; + /* * Security session for inline IPsec protocol offload. This is private data of * inline capable PMD. diff --git a/drivers/event/octeontx2/Makefile b/drivers/event/octeontx2/Makefile index 6dab69c..bcd22ee 100644 --- a/drivers/event/octeontx2/Makefile +++ b/drivers/event/octeontx2/Makefile @@ -11,6 +11,7 @@ LIB = librte_pmd_octeontx2_event.a CFLAGS += $(WERROR_FLAGS) CFLAGS += -I$(RTE_SDK)/drivers/common/octeontx2 +CFLAGS += -I$(RTE_SDK)/drivers/crypto/octeontx2 CFLAGS += -I$(RTE_SDK)/drivers/mempool/octeontx2 CFLAGS += -I$(RTE_SDK)/drivers/event/octeontx2 CFLAGS += -I$(RTE_SDK)/drivers/net/octeontx2 diff --git a/drivers/event/octeontx2/meson.build b/drivers/event/octeontx2/meson.build index 807818b..56febb8 100644 --- a/drivers/event/octeontx2/meson.build +++ b/drivers/event/octeontx2/meson.build @@ -32,3 +32,5 @@ foreach flag: extra_flags endforeach deps += ['bus_pci', 'common_octeontx2', 'mempool_octeontx2', 'pmd_octeontx2'] + +includes += include_directories('../../crypto/octeontx2') diff --git a/drivers/net/octeontx2/Makefile b/drivers/net/octeontx2/Makefile index 68f5765..d31ce0a 100644 --- a/drivers/net/octeontx2/Makefile +++ b/drivers/net/octeontx2/Makefile @@ -11,6 +11,7 @@ LIB = librte_pmd_octeontx2.a CFLAGS += $(WERROR_FLAGS) CFLAGS += -I$(RTE_SDK)/drivers/common/octeontx2 +CFLAGS += -I$(RTE_SDK)/drivers/crypto/octeontx2 CFLAGS += -I$(RTE_SDK)/drivers/mempool/octeontx2 CFLAGS += -I$(RTE_SDK)/drivers/net/octeontx2 CFLAGS += -O3 diff --git a/drivers/net/octeontx2/meson.build b/drivers/net/octeontx2/meson.build index fad3076..4a06eb2 100644 --- a/drivers/net/octeontx2/meson.build +++ b/drivers/net/octeontx2/meson.build @@ -25,6 +25,7 @@ sources = files('otx2_rx.c', ) deps += ['bus_pci', 'common_octeontx2', 'mempool_octeontx2'] +deps += ['cryptodev', 'security'] cflags += ['-flax-vector-conversions'] @@ -39,3 +40,5 @@ foreach flag: extra_flags cflags += flag endif endforeach + +includes += include_directories('../../crypto/octeontx2') diff --git a/drivers/net/octeontx2/otx2_rx.h b/drivers/net/octeontx2/otx2_rx.h index 5e1d5a2..f1dbfb7 100644 --- a/drivers/net/octeontx2/otx2_rx.h +++ b/drivers/net/octeontx2/otx2_rx.h @@ -5,7 +5,11 @@ #ifndef __OTX2_RX_H__ #define __OTX2_RX_H__ +#include + #include "otx2_common.h" +#include "otx2_ipsec_fp.h" +#include "otx2_security.h" /* Default mark value used when none is provided. */ #define OTX2_FLOW_ACTION_FLAG_DEFAULT 0xffff @@ -25,6 +29,12 @@ #define NIX_RX_MULTI_SEG_F BIT(15) #define NIX_TIMESYNC_RX_OFFSET 8 +/* Inline IPsec offsets */ + +#define INLINE_INB_RPTR_HDR 16 +/* nix_cqe_hdr_s + nix_rx_parse_s + nix_rx_sg_s + nix_iova_s */ +#define INLINE_CPT_RESULT_OFFSET 80 + struct otx2_timesync_info { uint64_t rx_tstamp; rte_iova_t tx_tstamp_iova; @@ -184,6 +194,61 @@ nix_cqe_xtract_mseg(const struct nix_rx_parse_s *rx, } } +static __rte_always_inline uint16_t +nix_rx_sec_cptres_get(const void *cq) +{ + volatile const struct otx2_cpt_res *res; + + res = (volatile const struct otx2_cpt_res *)((const char *)cq + + INLINE_CPT_RESULT_OFFSET); + + return res->u16[0]; +} + +static __rte_always_inline void * +nix_rx_sec_sa_get(const void * const lookup_mem, int spi, uint16_t port) +{ + const uint64_t *const *sa_tbl = (const uint64_t * const *) + ((const uint8_t *)lookup_mem + PTYPE_ARRAY_SZ + + ERR_ARRAY_SZ); + + return (void *)sa_tbl[port][spi]; +} + +static __rte_always_inline uint64_t +nix_rx_sec_mbuf_update(const struct nix_cqe_hdr_s *cq, struct rte_mbuf *m, + const void * const lookup_mem) +{ + struct otx2_ipsec_fp_in_sa *sa; + struct rte_ipv4_hdr *ipv4; + uint16_t m_len; + uint32_t spi; + char *data; + + if (unlikely(nix_rx_sec_cptres_get(cq) != OTX2_SEC_COMP_GOOD)) + return PKT_RX_SEC_OFFLOAD | PKT_RX_SEC_OFFLOAD_FAILED; + + /* 20 bits of tag would have the SPI */ + spi = cq->tag & 0xFFFFF; + + sa = nix_rx_sec_sa_get(lookup_mem, spi, m->port); + m->udata64 = (uint64_t)sa->userdata; + + data = rte_pktmbuf_mtod(m, char *); + memcpy(data + INLINE_INB_RPTR_HDR, data, RTE_ETHER_HDR_LEN); + + m->data_off += INLINE_INB_RPTR_HDR; + + ipv4 = (struct rte_ipv4_hdr *)(data + INLINE_INB_RPTR_HDR + + RTE_ETHER_HDR_LEN); + + m_len = rte_be_to_cpu_16(ipv4->total_length) + RTE_ETHER_HDR_LEN; + + m->data_len = m_len; + m->pkt_len = m_len; + return PKT_RX_SEC_OFFLOAD; +} + static __rte_always_inline void otx2_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, struct rte_mbuf *mbuf, const void *lookup_mem, @@ -225,6 +290,13 @@ otx2_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, if (flag & NIX_RX_OFFLOAD_MARK_UPDATE_F) ol_flags = nix_update_match_id(rx->match_id, ol_flags, mbuf); + if (cq->cqe_type == NIX_XQE_TYPE_RX_IPSECH) { + *(uint64_t *)(&mbuf->rearm_data) = val; + ol_flags |= nix_rx_sec_mbuf_update(cq, mbuf, lookup_mem); + mbuf->ol_flags = ol_flags; + return; + } + mbuf->ol_flags = ol_flags; *(uint64_t *)(&mbuf->rearm_data) = val; mbuf->pkt_len = len; From patchwork Sun Dec 8 11:54:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 63637 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 729A6A04F1; Sun, 8 Dec 2019 12:57:35 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A0D9A1BF70; Sun, 8 Dec 2019 12:56:50 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 6BD5D1BE82 for ; Sun, 8 Dec 2019 12:56:49 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB8BpS6W005483; Sun, 8 Dec 2019 03:56:48 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=pXFCI0k6+/zJeQIOksA/GJCJihpElDaSd9wiCYydnuk=; b=ojHJnMkBnzNRQcXD7AVzF3o3Owfd6TjQVzJH/MXiRYioW8SSmwtwnpoNDFcTCDYrbu2t WzsHnvLLqWWDJeW1tkvEzguVzRF+IAvvXZnFtYJWvHSrAQrwJasBun2FNzPJJuviy5+P CLQw/UTPFyGk4wH+4JKOjVK0ojcrvXwjaQGjBgeLBAxB0qV4wBAjFE7DGvsS1VY1I8u+ q5w0amUT5WWd9qY+SdqG7fC45qeBBPkDnNTkEnJodCMZ6Bzx9Qo2y3GCDgIfC1QvefMZ BT3QPJtVT3isQVGGrd3uQua24+/F9PM4hk58OUjXlQFR+DWosqYD9uWGAMoS1P/e8E6f cg== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0b-0016f401.pphosted.com with ESMTP id 2wrcfptc31-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 08 Dec 2019 03:56:48 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Sun, 8 Dec 2019 03:56:46 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Sun, 8 Dec 2019 03:56:46 -0800 Received: from ajoseph83.caveonetworks.com.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id 8E9D53F703F; Sun, 8 Dec 2019 03:56:41 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Declan Doherty , Thomas Monjalon CC: Archana Muniganti , Jerin Jacob , Narayana Prasad , Kiran Kumar K , Nithin Dabilpuram , "Pavan Nikhilesh" , Ankur Dwivedi , Anoob Joseph , Tejasree Kondoj , Vamsi Attunuru , Lukasz Bartosik , Date: Sun, 8 Dec 2019 17:24:51 +0530 Message-ID: <1575806094-28391-13-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1575806094-28391-1-git-send-email-anoobj@marvell.com> References: <1575806094-28391-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-08_03:2019-12-05,2019-12-08 signatures=0 Subject: [dpdk-dev] [PATCH 12/15] drivers/octeontx2: add sec in compiler optimized RX fastpath framework X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Archana Muniganti Added new flag for SECURITY in RX compiler optimized fastpath framework. With this, compiler autogenerates functions which have security enabled. Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Archana Muniganti Signed-off-by: Tejasree Kondoj Signed-off-by: Vamsi Attunuru --- drivers/event/octeontx2/otx2_evdev.c | 134 ++++++++----- drivers/event/octeontx2/otx2_evdev.h | 2 +- drivers/event/octeontx2/otx2_worker.c | 2 +- drivers/event/octeontx2/otx2_worker_dual.c | 2 +- drivers/net/octeontx2/otx2_ethdev.c | 3 + drivers/net/octeontx2/otx2_rx.c | 27 +-- drivers/net/octeontx2/otx2_rx.h | 306 ++++++++++++++++++++--------- 7 files changed, 320 insertions(+), 156 deletions(-) diff --git a/drivers/event/octeontx2/otx2_evdev.c b/drivers/event/octeontx2/otx2_evdev.c index 2daeba4..f6c641a 100644 --- a/drivers/event/octeontx2/otx2_evdev.c +++ b/drivers/event/octeontx2/otx2_evdev.c @@ -44,61 +44,64 @@ sso_fastpath_fns_set(struct rte_eventdev *event_dev) { struct otx2_sso_evdev *dev = sso_pmd_priv(event_dev); /* Single WS modes */ - const event_dequeue_t ssogws_deq[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_ssogws_deq_ ##name, + const event_dequeue_t ssogws_deq[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = otx2_ssogws_deq_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; - const event_dequeue_burst_t ssogws_deq_burst[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_ssogws_deq_burst_ ##name, + const event_dequeue_burst_t ssogws_deq_burst[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = otx2_ssogws_deq_burst_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; - const event_dequeue_t ssogws_deq_timeout[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_ssogws_deq_timeout_ ##name, + const event_dequeue_t ssogws_deq_timeout[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = otx2_ssogws_deq_timeout_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; const event_dequeue_burst_t - ssogws_deq_timeout_burst[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = \ + ssogws_deq_timeout_burst[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = \ otx2_ssogws_deq_timeout_burst_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; - const event_dequeue_t ssogws_deq_seg[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_ssogws_deq_seg_ ##name, + const event_dequeue_t ssogws_deq_seg[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = otx2_ssogws_deq_seg_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; - const event_dequeue_burst_t ssogws_deq_seg_burst[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_ssogws_deq_seg_burst_ ##name, + const event_dequeue_burst_t + ssogws_deq_seg_burst[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = \ + otx2_ssogws_deq_seg_burst_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; - const event_dequeue_t ssogws_deq_seg_timeout[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_ssogws_deq_seg_timeout_ ##name, + const event_dequeue_t ssogws_deq_seg_timeout[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = \ + otx2_ssogws_deq_seg_timeout_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; const event_dequeue_burst_t - ssogws_deq_seg_timeout_burst[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = \ + ssogws_deq_seg_timeout_burst[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = \ otx2_ssogws_deq_seg_timeout_burst_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R @@ -106,64 +109,69 @@ SSO_RX_ADPTR_ENQ_FASTPATH_FUNC /* Dual WS modes */ - const event_dequeue_t ssogws_dual_deq[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_ssogws_dual_deq_ ##name, + const event_dequeue_t ssogws_dual_deq[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = otx2_ssogws_dual_deq_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; - const event_dequeue_burst_t ssogws_dual_deq_burst[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_ssogws_dual_deq_burst_ ##name, + const event_dequeue_burst_t + ssogws_dual_deq_burst[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = \ + otx2_ssogws_dual_deq_burst_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; - const event_dequeue_t ssogws_dual_deq_timeout[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_ssogws_dual_deq_timeout_ ##name, + const event_dequeue_t ssogws_dual_deq_timeout[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = \ + otx2_ssogws_dual_deq_timeout_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; const event_dequeue_burst_t - ssogws_dual_deq_timeout_burst[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_ssogws_dual_deq_timeout_burst_ ##name, + ssogws_dual_deq_timeout_burst[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = \ + otx2_ssogws_dual_deq_timeout_burst_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; - const event_dequeue_t ssogws_dual_deq_seg[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_ssogws_dual_deq_seg_ ##name, + const event_dequeue_t ssogws_dual_deq_seg[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = otx2_ssogws_dual_deq_seg_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; const event_dequeue_burst_t - ssogws_dual_deq_seg_burst[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = \ - otx2_ssogws_dual_deq_seg_burst_ ##name, + ssogws_dual_deq_seg_burst[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = \ + otx2_ssogws_dual_deq_seg_burst_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; - const event_dequeue_t ssogws_dual_deq_seg_timeout[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = \ - otx2_ssogws_dual_deq_seg_timeout_ ##name, + const event_dequeue_t + ssogws_dual_deq_seg_timeout[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = \ + otx2_ssogws_dual_deq_seg_timeout_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; const event_dequeue_burst_t - ssogws_dual_deq_seg_timeout_burst[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = \ - otx2_ssogws_dual_deq_seg_timeout_burst_ ##name, + ssogws_dual_deq_seg_timeout_burst[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = \ + otx2_ssogws_dual_deq_seg_timeout_burst_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; @@ -209,6 +217,7 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC event_dev->enqueue_forward_burst = otx2_ssogws_enq_fwd_burst; if (dev->rx_offloads & NIX_RX_MULTI_SEG_F) { event_dev->dequeue = ssogws_deq_seg + [!!(dev->rx_offloads & NIX_RX_OFFLOAD_SECURITY_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_VLAN_STRIP_F)] @@ -216,6 +225,7 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC [!!(dev->rx_offloads & NIX_RX_OFFLOAD_PTYPE_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_RSS_F)]; event_dev->dequeue_burst = ssogws_deq_seg_burst + [!!(dev->rx_offloads & NIX_RX_OFFLOAD_SECURITY_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_VLAN_STRIP_F)] @@ -224,6 +234,7 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC [!!(dev->rx_offloads & NIX_RX_OFFLOAD_RSS_F)]; if (dev->is_timeout_deq) { event_dev->dequeue = ssogws_deq_seg_timeout + [!!(dev->rx_offloads & NIX_RX_OFFLOAD_SECURITY_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_VLAN_STRIP_F)] @@ -232,6 +243,7 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC [!!(dev->rx_offloads & NIX_RX_OFFLOAD_RSS_F)]; event_dev->dequeue_burst = ssogws_deq_seg_timeout_burst + [!!(dev->rx_offloads & NIX_RX_OFFLOAD_SECURITY_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_VLAN_STRIP_F)] @@ -241,6 +253,7 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC } } else { event_dev->dequeue = ssogws_deq + [!!(dev->rx_offloads & NIX_RX_OFFLOAD_SECURITY_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_VLAN_STRIP_F)] @@ -248,6 +261,7 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC [!!(dev->rx_offloads & NIX_RX_OFFLOAD_PTYPE_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_RSS_F)]; event_dev->dequeue_burst = ssogws_deq_burst + [!!(dev->rx_offloads & NIX_RX_OFFLOAD_SECURITY_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_VLAN_STRIP_F)] @@ -256,6 +270,7 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC [!!(dev->rx_offloads & NIX_RX_OFFLOAD_RSS_F)]; if (dev->is_timeout_deq) { event_dev->dequeue = ssogws_deq_timeout + [!!(dev->rx_offloads & NIX_RX_OFFLOAD_SECURITY_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_VLAN_STRIP_F)] @@ -264,6 +279,7 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC [!!(dev->rx_offloads & NIX_RX_OFFLOAD_RSS_F)]; event_dev->dequeue_burst = ssogws_deq_timeout_burst + [!!(dev->rx_offloads & NIX_RX_OFFLOAD_SECURITY_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_VLAN_STRIP_F)] @@ -303,6 +319,8 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC if (dev->rx_offloads & NIX_RX_MULTI_SEG_F) { event_dev->dequeue = ssogws_dual_deq_seg [!!(dev->rx_offloads & + NIX_RX_OFFLOAD_SECURITY_F)] + [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] @@ -313,6 +331,8 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC [!!(dev->rx_offloads & NIX_RX_OFFLOAD_PTYPE_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_RSS_F)]; event_dev->dequeue_burst = ssogws_dual_deq_seg_burst + [!!(dev->rx_offloads & + NIX_RX_OFFLOAD_SECURITY_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] @@ -326,6 +346,8 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC event_dev->dequeue = ssogws_dual_deq_seg_timeout [!!(dev->rx_offloads & + NIX_RX_OFFLOAD_SECURITY_F)] + [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] @@ -340,6 +362,8 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC event_dev->dequeue_burst = ssogws_dual_deq_seg_timeout_burst [!!(dev->rx_offloads & + NIX_RX_OFFLOAD_SECURITY_F)] + [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] @@ -355,6 +379,8 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC } else { event_dev->dequeue = ssogws_dual_deq [!!(dev->rx_offloads & + NIX_RX_OFFLOAD_SECURITY_F)] + [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] @@ -366,6 +392,8 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC [!!(dev->rx_offloads & NIX_RX_OFFLOAD_RSS_F)]; event_dev->dequeue_burst = ssogws_dual_deq_burst [!!(dev->rx_offloads & + NIX_RX_OFFLOAD_SECURITY_F)] + [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] @@ -379,6 +407,8 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC event_dev->dequeue = ssogws_dual_deq_timeout [!!(dev->rx_offloads & + NIX_RX_OFFLOAD_SECURITY_F)] + [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] @@ -393,6 +423,8 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC event_dev->dequeue_burst = ssogws_dual_deq_timeout_burst [!!(dev->rx_offloads & + NIX_RX_OFFLOAD_SECURITY_F)] + [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] diff --git a/drivers/event/octeontx2/otx2_evdev.h b/drivers/event/octeontx2/otx2_evdev.h index 231a12a..e20e251 100644 --- a/drivers/event/octeontx2/otx2_evdev.h +++ b/drivers/event/octeontx2/otx2_evdev.h @@ -278,7 +278,7 @@ uint16_t otx2_ssogws_dual_enq_fwd_burst(void *port, const struct rte_event ev[], uint16_t nb_events); /* Auto generated API's */ -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ uint16_t otx2_ssogws_deq_ ##name(void *port, struct rte_event *ev, \ uint64_t timeout_ticks); \ uint16_t otx2_ssogws_deq_burst_ ##name(void *port, struct rte_event ev[], \ diff --git a/drivers/event/octeontx2/otx2_worker.c b/drivers/event/octeontx2/otx2_worker.c index 34d39f4..12445d9 100644 --- a/drivers/event/octeontx2/otx2_worker.c +++ b/drivers/event/octeontx2/otx2_worker.c @@ -81,7 +81,7 @@ otx2_ssogws_release_event(struct otx2_ssogws *ws) otx2_ssogws_swtag_flush(ws); } -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ uint16_t __hot \ otx2_ssogws_deq_ ##name(void *port, struct rte_event *ev, \ uint64_t timeout_ticks) \ diff --git a/drivers/event/octeontx2/otx2_worker_dual.c b/drivers/event/octeontx2/otx2_worker_dual.c index 7016eee..22a4889 100644 --- a/drivers/event/octeontx2/otx2_worker_dual.c +++ b/drivers/event/octeontx2/otx2_worker_dual.c @@ -140,7 +140,7 @@ otx2_ssogws_dual_enq_fwd_burst(void *port, const struct rte_event ev[], return 1; } -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ uint16_t __hot \ otx2_ssogws_dual_deq_ ##name(void *port, struct rte_event *ev, \ uint64_t timeout_ticks) \ diff --git a/drivers/net/octeontx2/otx2_ethdev.c b/drivers/net/octeontx2/otx2_ethdev.c index 131e883..c215078 100644 --- a/drivers/net/octeontx2/otx2_ethdev.c +++ b/drivers/net/octeontx2/otx2_ethdev.c @@ -626,6 +626,9 @@ nix_rx_offload_flags(struct rte_eth_dev *eth_dev) if ((dev->rx_offloads & DEV_RX_OFFLOAD_TIMESTAMP)) flags |= NIX_RX_OFFLOAD_TSTAMP_F; + if (dev->rx_offloads & DEV_RX_OFFLOAD_SECURITY) + flags |= NIX_RX_OFFLOAD_SECURITY_F; + if (!dev->ptype_disable) flags |= NIX_RX_OFFLOAD_PTYPE_F; diff --git a/drivers/net/octeontx2/otx2_rx.c b/drivers/net/octeontx2/otx2_rx.c index 48565db..ffe6996 100644 --- a/drivers/net/octeontx2/otx2_rx.c +++ b/drivers/net/octeontx2/otx2_rx.c @@ -320,7 +320,7 @@ nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, #endif -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ static uint16_t __rte_noinline __hot \ otx2_nix_recv_pkts_ ## name(void *rx_queue, \ struct rte_mbuf **rx_pkts, uint16_t pkts) \ @@ -351,12 +351,13 @@ NIX_RX_FASTPATH_MODES static inline void pick_rx_func(struct rte_eth_dev *eth_dev, - const eth_rx_burst_t rx_burst[2][2][2][2][2][2]) + const eth_rx_burst_t rx_burst[2][2][2][2][2][2][2]) { struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev); - /* [TSTMP] [MARK] [VLAN] [CKSUM] [PTYPE] [RSS] */ + /* [SEC] [TSTMP] [MARK] [VLAN] [CKSUM] [PTYPE] [RSS] */ eth_dev->rx_pkt_burst = rx_burst + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_SECURITY_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_MARK_UPDATE_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_VLAN_STRIP_F)] @@ -370,25 +371,25 @@ otx2_eth_set_rx_function(struct rte_eth_dev *eth_dev) { struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev); - const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_nix_recv_pkts_ ## name, + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = otx2_nix_recv_pkts_ ## name, NIX_RX_FASTPATH_MODES #undef R }; - const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_nix_recv_pkts_mseg_ ## name, + const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = otx2_nix_recv_pkts_mseg_ ## name, NIX_RX_FASTPATH_MODES #undef R }; - const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_nix_recv_pkts_vec_ ## name, + const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = otx2_nix_recv_pkts_vec_ ## name, NIX_RX_FASTPATH_MODES #undef R @@ -408,6 +409,6 @@ NIX_RX_FASTPATH_MODES /* Copy multi seg version with no offload for tear down sequence */ if (rte_eal_process_type() == RTE_PROC_PRIMARY) dev->rx_pkt_burst_no_offload = - nix_eth_rx_burst_mseg[0][0][0][0][0][0]; + nix_eth_rx_burst_mseg[0][0][0][0][0][0][0]; rte_mb(); } diff --git a/drivers/net/octeontx2/otx2_rx.h b/drivers/net/octeontx2/otx2_rx.h index f1dbfb7..a523503 100644 --- a/drivers/net/octeontx2/otx2_rx.h +++ b/drivers/net/octeontx2/otx2_rx.h @@ -21,6 +21,7 @@ #define NIX_RX_OFFLOAD_VLAN_STRIP_F BIT(3) #define NIX_RX_OFFLOAD_MARK_UPDATE_F BIT(4) #define NIX_RX_OFFLOAD_TSTAMP_F BIT(5) +#define NIX_RX_OFFLOAD_SECURITY_F BIT(6) /* Flags to control cqe_to_mbuf conversion function. * Defining it from backwards to denote its been @@ -290,7 +291,8 @@ otx2_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, if (flag & NIX_RX_OFFLOAD_MARK_UPDATE_F) ol_flags = nix_update_match_id(rx->match_id, ol_flags, mbuf); - if (cq->cqe_type == NIX_XQE_TYPE_RX_IPSECH) { + if ((flag & NIX_RX_OFFLOAD_SECURITY_F) && + cq->cqe_type == NIX_XQE_TYPE_RX_IPSECH) { *(uint64_t *)(&mbuf->rearm_data) = val; ol_flags |= nix_rx_sec_mbuf_update(cq, mbuf, lookup_mem); mbuf->ol_flags = ol_flags; @@ -313,94 +315,220 @@ otx2_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, #define RX_VLAN_F NIX_RX_OFFLOAD_VLAN_STRIP_F #define MARK_F NIX_RX_OFFLOAD_MARK_UPDATE_F #define TS_F NIX_RX_OFFLOAD_TSTAMP_F +#define RX_SEC_F NIX_RX_OFFLOAD_SECURITY_F -/* [TSMP] [MARK] [VLAN] [CKSUM] [PTYPE] [RSS] */ +/* [SEC] [TSMP] [MARK] [VLAN] [CKSUM] [PTYPE] [RSS] */ #define NIX_RX_FASTPATH_MODES \ -R(no_offload, 0, 0, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ -R(rss, 0, 0, 0, 0, 0, 1, RSS_F) \ -R(ptype, 0, 0, 0, 0, 1, 0, PTYPE_F) \ -R(ptype_rss, 0, 0, 0, 0, 1, 1, PTYPE_F | RSS_F) \ -R(cksum, 0, 0, 0, 1, 0, 0, CKSUM_F) \ -R(cksum_rss, 0, 0, 0, 1, 0, 1, CKSUM_F | RSS_F) \ -R(cksum_ptype, 0, 0, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ -R(cksum_ptype_rss, 0, 0, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F)\ -R(vlan, 0, 0, 1, 0, 0, 0, RX_VLAN_F) \ -R(vlan_rss, 0, 0, 1, 0, 0, 1, RX_VLAN_F | RSS_F) \ -R(vlan_ptype, 0, 0, 1, 0, 1, 0, RX_VLAN_F | PTYPE_F) \ -R(vlan_ptype_rss, 0, 0, 1, 0, 1, 1, RX_VLAN_F | PTYPE_F | RSS_F)\ -R(vlan_cksum, 0, 0, 1, 1, 0, 0, RX_VLAN_F | CKSUM_F) \ -R(vlan_cksum_rss, 0, 0, 1, 1, 0, 1, RX_VLAN_F | CKSUM_F | RSS_F)\ -R(vlan_cksum_ptype, 0, 0, 1, 1, 1, 0, \ - RX_VLAN_F | CKSUM_F | PTYPE_F) \ -R(vlan_cksum_ptype_rss, 0, 0, 1, 1, 1, 1, \ - RX_VLAN_F | CKSUM_F | PTYPE_F | RSS_F) \ -R(mark, 0, 1, 0, 0, 0, 0, MARK_F) \ -R(mark_rss, 0, 1, 0, 0, 0, 1, MARK_F | RSS_F) \ -R(mark_ptype, 0, 1, 0, 0, 1, 0, MARK_F | PTYPE_F) \ -R(mark_ptype_rss, 0, 1, 0, 0, 1, 1, MARK_F | PTYPE_F | RSS_F)\ -R(mark_cksum, 0, 1, 0, 1, 0, 0, MARK_F | CKSUM_F) \ -R(mark_cksum_rss, 0, 1, 0, 1, 0, 1, MARK_F | CKSUM_F | RSS_F)\ -R(mark_cksum_ptype, 0, 1, 0, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F)\ -R(mark_cksum_ptype_rss, 0, 1, 0, 1, 1, 1, \ - MARK_F | CKSUM_F | PTYPE_F | RSS_F) \ -R(mark_vlan, 0, 1, 1, 0, 0, 0, MARK_F | RX_VLAN_F) \ -R(mark_vlan_rss, 0, 1, 1, 0, 0, 1, MARK_F | RX_VLAN_F | RSS_F)\ -R(mark_vlan_ptype, 0, 1, 1, 0, 1, 0, \ - MARK_F | RX_VLAN_F | PTYPE_F) \ -R(mark_vlan_ptype_rss, 0, 1, 1, 0, 1, 1, \ - MARK_F | RX_VLAN_F | PTYPE_F | RSS_F) \ -R(mark_vlan_cksum, 0, 1, 1, 1, 0, 0, \ - MARK_F | RX_VLAN_F | CKSUM_F) \ -R(mark_vlan_cksum_rss, 0, 1, 1, 1, 0, 1, \ - MARK_F | RX_VLAN_F | CKSUM_F | RSS_F) \ -R(mark_vlan_cksum_ptype, 0, 1, 1, 1, 1, 0, \ - MARK_F | RX_VLAN_F | CKSUM_F | PTYPE_F) \ -R(mark_vlan_cksum_ptype_rss, 0, 1, 1, 1, 1, 1, \ - MARK_F | RX_VLAN_F | CKSUM_F | PTYPE_F | RSS_F) \ -R(ts, 1, 0, 0, 0, 0, 0, TS_F) \ -R(ts_rss, 1, 0, 0, 0, 0, 1, TS_F | RSS_F) \ -R(ts_ptype, 1, 0, 0, 0, 1, 0, TS_F | PTYPE_F) \ -R(ts_ptype_rss, 1, 0, 0, 0, 1, 1, TS_F | PTYPE_F | RSS_F)\ -R(ts_cksum, 1, 0, 0, 1, 0, 0, TS_F | CKSUM_F) \ -R(ts_cksum_rss, 1, 0, 0, 1, 0, 1, TS_F | CKSUM_F | RSS_F)\ -R(ts_cksum_ptype, 1, 0, 0, 1, 1, 0, TS_F | CKSUM_F | PTYPE_F)\ -R(ts_cksum_ptype_rss, 1, 0, 0, 1, 1, 1, \ - TS_F | CKSUM_F | PTYPE_F | RSS_F) \ -R(ts_vlan, 1, 0, 1, 0, 0, 0, TS_F | RX_VLAN_F) \ -R(ts_vlan_rss, 1, 0, 1, 0, 0, 1, TS_F | RX_VLAN_F | RSS_F)\ -R(ts_vlan_ptype, 1, 0, 1, 0, 1, 0, TS_F | RX_VLAN_F | PTYPE_F)\ -R(ts_vlan_ptype_rss, 1, 0, 1, 0, 1, 1, \ - TS_F | RX_VLAN_F | PTYPE_F | RSS_F) \ -R(ts_vlan_cksum, 1, 0, 1, 1, 0, 0, \ - TS_F | RX_VLAN_F | CKSUM_F) \ -R(ts_vlan_cksum_rss, 1, 0, 1, 1, 0, 1, \ - MARK_F | RX_VLAN_F | CKSUM_F | RSS_F) \ -R(ts_vlan_cksum_ptype, 1, 0, 1, 1, 1, 0, \ - TS_F | RX_VLAN_F | CKSUM_F | PTYPE_F) \ -R(ts_vlan_cksum_ptype_rss, 1, 0, 1, 1, 1, 1, \ - TS_F | RX_VLAN_F | CKSUM_F | PTYPE_F | RSS_F) \ -R(ts_mark, 1, 1, 0, 0, 0, 0, TS_F | MARK_F) \ -R(ts_mark_rss, 1, 1, 0, 0, 0, 1, TS_F | MARK_F | RSS_F)\ -R(ts_mark_ptype, 1, 1, 0, 0, 1, 0, TS_F | MARK_F | PTYPE_F)\ -R(ts_mark_ptype_rss, 1, 1, 0, 0, 1, 1, \ - TS_F | MARK_F | PTYPE_F | RSS_F) \ -R(ts_mark_cksum, 1, 1, 0, 1, 0, 0, TS_F | MARK_F | CKSUM_F)\ -R(ts_mark_cksum_rss, 1, 1, 0, 1, 0, 1, \ - TS_F | MARK_F | CKSUM_F | RSS_F)\ -R(ts_mark_cksum_ptype, 1, 1, 0, 1, 1, 0, \ - TS_F | MARK_F | CKSUM_F | PTYPE_F) \ -R(ts_mark_cksum_ptype_rss, 1, 1, 0, 1, 1, 1, \ - TS_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) \ -R(ts_mark_vlan, 1, 1, 1, 0, 0, 0, TS_F | MARK_F | RX_VLAN_F)\ -R(ts_mark_vlan_rss, 1, 1, 1, 0, 0, 1, \ - TS_F | MARK_F | RX_VLAN_F | RSS_F)\ -R(ts_mark_vlan_ptype, 1, 1, 1, 0, 1, 0, \ - TS_F | MARK_F | RX_VLAN_F | PTYPE_F) \ -R(ts_mark_vlan_ptype_rss, 1, 1, 1, 0, 1, 1, \ - TS_F | MARK_F | RX_VLAN_F | PTYPE_F | RSS_F) \ -R(ts_mark_vlan_cksum_ptype, 1, 1, 1, 1, 1, 0, \ - TS_F | MARK_F | RX_VLAN_F | CKSUM_F | PTYPE_F) \ -R(ts_mark_vlan_cksum_ptype_rss, 1, 1, 1, 1, 1, 1, \ - TS_F | MARK_F | RX_VLAN_F | CKSUM_F | PTYPE_F | RSS_F) - +R(no_offload, 0, 0, 0, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ +R(rss, 0, 0, 0, 0, 0, 0, 1, RSS_F) \ +R(ptype, 0, 0, 0, 0, 0, 1, 0, PTYPE_F) \ +R(ptype_rss, 0, 0, 0, 0, 0, 1, 1, PTYPE_F | RSS_F) \ +R(cksum, 0, 0, 0, 0, 1, 0, 0, CKSUM_F) \ +R(cksum_rss, 0, 0, 0, 0, 1, 0, 1, CKSUM_F | RSS_F) \ +R(cksum_ptype, 0, 0, 0, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ +R(cksum_ptype_rss, 0, 0, 0, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F)\ +R(vlan, 0, 0, 0, 1, 0, 0, 0, RX_VLAN_F) \ +R(vlan_rss, 0, 0, 0, 1, 0, 0, 1, RX_VLAN_F | RSS_F) \ +R(vlan_ptype, 0, 0, 0, 1, 0, 1, 0, RX_VLAN_F | PTYPE_F) \ +R(vlan_ptype_rss, 0, 0, 0, 1, 0, 1, 1, \ + RX_VLAN_F | PTYPE_F | RSS_F) \ +R(vlan_cksum, 0, 0, 0, 1, 1, 0, 0, RX_VLAN_F | CKSUM_F) \ +R(vlan_cksum_rss, 0, 0, 0, 1, 1, 0, 1, \ + RX_VLAN_F | CKSUM_F | RSS_F) \ +R(vlan_cksum_ptype, 0, 0, 0, 1, 1, 1, 0, \ + RX_VLAN_F | CKSUM_F | PTYPE_F) \ +R(vlan_cksum_ptype_rss, 0, 0, 0, 1, 1, 1, 1, \ + RX_VLAN_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(mark, 0, 0, 1, 0, 0, 0, 0, MARK_F) \ +R(mark_rss, 0, 0, 1, 0, 0, 0, 1, MARK_F | RSS_F) \ +R(mark_ptype, 0, 0, 1, 0, 0, 1, 0, MARK_F | PTYPE_F) \ +R(mark_ptype_rss, 0, 0, 1, 0, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ +R(mark_cksum, 0, 0, 1, 0, 1, 0, 0, MARK_F | CKSUM_F) \ +R(mark_cksum_rss, 0, 0, 1, 0, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ +R(mark_cksum_ptype, 0, 0, 1, 0, 1, 1, 0, \ + MARK_F | CKSUM_F | PTYPE_F) \ +R(mark_cksum_ptype_rss, 0, 0, 1, 0, 1, 1, 1, \ + MARK_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(mark_vlan, 0, 0, 1, 1, 0, 0, 0, MARK_F | RX_VLAN_F) \ +R(mark_vlan_rss, 0, 0, 1, 1, 0, 0, 1, \ + MARK_F | RX_VLAN_F | RSS_F) \ +R(mark_vlan_ptype, 0, 0, 1, 1, 0, 1, 0, \ + MARK_F | RX_VLAN_F | PTYPE_F) \ +R(mark_vlan_ptype_rss, 0, 0, 1, 1, 0, 1, 1, \ + MARK_F | RX_VLAN_F | PTYPE_F | RSS_F) \ +R(mark_vlan_cksum, 0, 0, 1, 1, 1, 0, 0, \ + MARK_F | RX_VLAN_F | CKSUM_F) \ +R(mark_vlan_cksum_rss, 0, 0, 1, 1, 1, 0, 1, \ + MARK_F | RX_VLAN_F | CKSUM_F | RSS_F) \ +R(mark_vlan_cksum_ptype, 0, 0, 1, 1, 1, 1, 0, \ + MARK_F | RX_VLAN_F | CKSUM_F | PTYPE_F) \ +R(mark_vlan_cksum_ptype_rss, 0, 0, 1, 1, 1, 1, 1, \ + MARK_F | RX_VLAN_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(ts, 0, 1, 0, 0, 0, 0, 0, TS_F) \ +R(ts_rss, 0, 1, 0, 0, 0, 0, 1, TS_F | RSS_F) \ +R(ts_ptype, 0, 1, 0, 0, 0, 1, 0, TS_F | PTYPE_F) \ +R(ts_ptype_rss, 0, 1, 0, 0, 0, 1, 1, TS_F | PTYPE_F | RSS_F) \ +R(ts_cksum, 0, 1, 0, 0, 1, 0, 0, TS_F | CKSUM_F) \ +R(ts_cksum_rss, 0, 1, 0, 0, 1, 0, 1, TS_F | CKSUM_F | RSS_F) \ +R(ts_cksum_ptype, 0, 1, 0, 0, 1, 1, 0, TS_F | CKSUM_F | PTYPE_F) \ +R(ts_cksum_ptype_rss, 0, 1, 0, 0, 1, 1, 1, \ + TS_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(ts_vlan, 0, 1, 0, 1, 0, 0, 0, TS_F | RX_VLAN_F) \ +R(ts_vlan_rss, 0, 1, 0, 1, 0, 0, 1, TS_F | RX_VLAN_F | RSS_F) \ +R(ts_vlan_ptype, 0, 1, 0, 1, 0, 1, 0, \ + TS_F | RX_VLAN_F | PTYPE_F) \ +R(ts_vlan_ptype_rss, 0, 1, 0, 1, 0, 1, 1, \ + TS_F | RX_VLAN_F | PTYPE_F | RSS_F) \ +R(ts_vlan_cksum, 0, 1, 0, 1, 1, 0, 0, \ + TS_F | RX_VLAN_F | CKSUM_F) \ +R(ts_vlan_cksum_rss, 0, 1, 0, 1, 1, 0, 1, \ + MARK_F | RX_VLAN_F | CKSUM_F | RSS_F) \ +R(ts_vlan_cksum_ptype, 0, 1, 0, 1, 1, 1, 0, \ + TS_F | RX_VLAN_F | CKSUM_F | PTYPE_F) \ +R(ts_vlan_cksum_ptype_rss, 0, 1, 0, 1, 1, 1, 1, \ + TS_F | RX_VLAN_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(ts_mark, 0, 1, 1, 0, 0, 0, 0, TS_F | MARK_F) \ +R(ts_mark_rss, 0, 1, 1, 0, 0, 0, 1, TS_F | MARK_F | RSS_F) \ +R(ts_mark_ptype, 0, 1, 1, 0, 0, 1, 0, TS_F | MARK_F | PTYPE_F) \ +R(ts_mark_ptype_rss, 0, 1, 1, 0, 0, 1, 1, \ + TS_F | MARK_F | PTYPE_F | RSS_F) \ +R(ts_mark_cksum, 0, 1, 1, 0, 1, 0, 0, TS_F | MARK_F | CKSUM_F) \ +R(ts_mark_cksum_rss, 0, 1, 1, 0, 1, 0, 1, \ + TS_F | MARK_F | CKSUM_F | RSS_F) \ +R(ts_mark_cksum_ptype, 0, 1, 1, 0, 1, 1, 0, \ + TS_F | MARK_F | CKSUM_F | PTYPE_F) \ +R(ts_mark_cksum_ptype_rss, 0, 1, 1, 0, 1, 1, 1, \ + TS_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(ts_mark_vlan, 0, 1, 1, 1, 0, 0, 0, TS_F | MARK_F | RX_VLAN_F)\ +R(ts_mark_vlan_rss, 0, 1, 1, 1, 0, 0, 1, \ + TS_F | MARK_F | RX_VLAN_F | RSS_F) \ +R(ts_mark_vlan_ptype, 0, 1, 1, 1, 0, 1, 0, \ + TS_F | MARK_F | RX_VLAN_F | PTYPE_F) \ +R(ts_mark_vlan_ptype_rss, 0, 1, 1, 1, 0, 1, 1, \ + TS_F | MARK_F | RX_VLAN_F | PTYPE_F | RSS_F) \ +R(ts_mark_vlan_cksum_ptype, 0, 1, 1, 1, 1, 1, 0, \ + TS_F | MARK_F | RX_VLAN_F | CKSUM_F | PTYPE_F) \ +R(ts_mark_vlan_cksum_ptype_rss, 0, 1, 1, 1, 1, 1, 1, \ + TS_F | MARK_F | RX_VLAN_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(sec, 1, 0, 0, 0, 0, 0, 0, RX_SEC_F) \ +R(sec_rss, 1, 0, 0, 0, 0, 0, 1, RX_SEC_F | RSS_F) \ +R(sec_ptype, 1, 0, 0, 0, 0, 1, 0, RX_SEC_F | PTYPE_F) \ +R(sec_ptype_rss, 1, 0, 0, 0, 0, 1, 1, \ + RX_SEC_F | PTYPE_F | RSS_F) \ +R(sec_cksum, 1, 0, 0, 0, 1, 0, 0, RX_SEC_F | CKSUM_F) \ +R(sec_cksum_rss, 1, 0, 0, 0, 1, 0, 1, \ + RX_SEC_F | CKSUM_F | RSS_F) \ +R(sec_cksum_ptype, 1, 0, 0, 0, 1, 1, 0, \ + RX_SEC_F | CKSUM_F | PTYPE_F) \ +R(sec_cksum_ptype_rss, 1, 0, 0, 0, 1, 1, 1, \ + RX_SEC_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(sec_vlan, 1, 0, 0, 1, 0, 0, 0, RX_SEC_F | RX_VLAN_F) \ +R(sec_vlan_rss, 1, 0, 0, 1, 0, 0, 1, \ + RX_SEC_F | RX_VLAN_F | RSS_F) \ +R(sec_vlan_ptype, 1, 0, 0, 1, 0, 1, 0, \ + RX_SEC_F | RX_VLAN_F | PTYPE_F) \ +R(sec_vlan_ptype_rss, 1, 0, 0, 1, 0, 1, 1, \ + RX_SEC_F | RX_VLAN_F | PTYPE_F | RSS_F) \ +R(sec_vlan_cksum, 1, 0, 0, 1, 1, 0, 0, \ + RX_SEC_F | RX_VLAN_F | CKSUM_F) \ +R(sec_vlan_cksum_rss, 1, 0, 0, 1, 1, 0, 1, \ + RX_SEC_F | RX_VLAN_F | CKSUM_F | RSS_F) \ +R(sec_vlan_cksum_ptype, 1, 0, 0, 1, 1, 1, 0, \ + RX_SEC_F | RX_VLAN_F | CKSUM_F | PTYPE_F) \ +R(sec_vlan_cksum_ptype_rss, 1, 0, 0, 1, 1, 1, 1, \ + RX_SEC_F | RX_VLAN_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(sec_mark, 1, 0, 1, 0, 0, 0, 0, RX_SEC_F | MARK_F) \ +R(sec_mark_rss, 1, 0, 1, 0, 0, 0, 1, RX_SEC_F | MARK_F | RSS_F)\ +R(sec_mark_ptype, 1, 0, 1, 0, 0, 1, 0, \ + RX_SEC_F | MARK_F | PTYPE_F) \ +R(sec_mark_ptype_rss, 1, 0, 1, 0, 0, 1, 1, \ + RX_SEC_F | MARK_F | PTYPE_F | RSS_F) \ +R(sec_mark_cksum, 1, 0, 1, 0, 1, 0, 0, \ + RX_SEC_F | MARK_F | CKSUM_F) \ +R(sec_mark_cksum_rss, 1, 0, 1, 0, 1, 0, 1, \ + RX_SEC_F | MARK_F | CKSUM_F | RSS_F) \ +R(sec_mark_cksum_ptype, 1, 0, 1, 0, 1, 1, 0, \ + RX_SEC_F | MARK_F | CKSUM_F | PTYPE_F) \ +R(sec_mark_cksum_ptype_rss, 1, 0, 1, 0, 1, 1, 1, \ + RX_SEC_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(sec_mark_vlan, 1, 0, 1, 1, 0, 0, 0, RX_SEC_F | RX_VLAN_F) \ +R(sec_mark_vlan_rss, 1, 0, 1, 1, 0, 0, 1, \ + RX_SEC_F | MARK_F | RX_VLAN_F | RSS_F) \ +R(sec_mark_vlan_ptype, 1, 0, 1, 1, 0, 1, 0, \ + RX_SEC_F | MARK_F | RX_VLAN_F | PTYPE_F) \ +R(sec_mark_vlan_ptype_rss, 1, 0, 1, 1, 0, 1, 1, \ + RX_SEC_F | MARK_F | RX_VLAN_F | PTYPE_F | RSS_F) \ +R(sec_mark_vlan_cksum, 1, 0, 1, 1, 1, 0, 0, \ + RX_SEC_F | MARK_F | RX_VLAN_F | CKSUM_F) \ +R(sec_mark_vlan_cksum_rss, 1, 0, 1, 1, 1, 0, 1, \ + RX_SEC_F | MARK_F | RX_VLAN_F | CKSUM_F | RSS_F) \ +R(sec_mark_vlan_cksum_ptype, 1, 0, 1, 1, 1, 1, 0, \ + RX_SEC_F | MARK_F | RX_VLAN_F | CKSUM_F | PTYPE_F) \ +R(sec_mark_vlan_cksum_ptype_rss, \ + 1, 0, 1, 1, 1, 1, 1, \ + RX_SEC_F | MARK_F | RX_VLAN_F | CKSUM_F | PTYPE_F | \ + RSS_F) \ +R(sec_ts, 1, 1, 0, 0, 0, 0, 0, RX_SEC_F | TS_F) \ +R(sec_ts_rss, 1, 1, 0, 0, 0, 0, 1, RX_SEC_F | TS_F | RSS_F) \ +R(sec_ts_ptype, 1, 1, 0, 0, 0, 1, 0, RX_SEC_F | TS_F | PTYPE_F)\ +R(sec_ts_ptype_rss, 1, 1, 0, 0, 0, 1, 1, \ + RX_SEC_F | TS_F | PTYPE_F | RSS_F) \ +R(sec_ts_cksum, 1, 1, 0, 0, 1, 0, 0, RX_SEC_F | TS_F | CKSUM_F)\ +R(sec_ts_cksum_rss, 1, 1, 0, 0, 1, 0, 1, \ + RX_SEC_F | TS_F | CKSUM_F | RSS_F) \ +R(sec_ts_cksum_ptype, 1, 1, 0, 0, 1, 1, 0, \ + RX_SEC_F | CKSUM_F | PTYPE_F) \ +R(sec_ts_cksum_ptype_rss, 1, 1, 0, 0, 1, 1, 1, \ + RX_SEC_F | TS_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(sec_ts_vlan, 1, 1, 0, 1, 0, 0, 0, \ + RX_SEC_F | TS_F | RX_VLAN_F) \ +R(sec_ts_vlan_rss, 1, 1, 0, 1, 0, 0, 1, \ + RX_SEC_F | TS_F | RX_VLAN_F | RSS_F) \ +R(sec_ts_vlan_ptype, 1, 1, 0, 1, 0, 1, 0, \ + RX_SEC_F | TS_F | RX_VLAN_F | PTYPE_F) \ +R(sec_ts_vlan_ptype_rss, 1, 1, 0, 1, 0, 1, 1, \ + RX_SEC_F | TS_F | RX_VLAN_F | PTYPE_F | RSS_F) \ +R(sec_ts_vlan_cksum, 1, 1, 0, 1, 1, 0, 0, \ + RX_SEC_F | TS_F | RX_VLAN_F | CKSUM_F) \ +R(sec_ts_vlan_cksum_rss, 1, 1, 0, 1, 1, 0, 1, \ + RX_SEC_F | TS_F | RX_VLAN_F | CKSUM_F | RSS_F) \ +R(sec_ts_vlan_cksum_ptype, 1, 1, 0, 1, 1, 1, 0, \ + RX_SEC_F | TS_F | RX_VLAN_F | CKSUM_F | PTYPE_F) \ +R(sec_ts_vlan_cksum_ptype_rss, 1, 1, 0, 1, 1, 1, 1, \ + RX_SEC_F | TS_F | RX_VLAN_F | CKSUM_F | PTYPE_F | \ + RSS_F) \ +R(sec_ts_mark, 1, 1, 1, 0, 0, 0, 0, RX_SEC_F | TS_F | MARK_F) \ +R(sec_ts_mark_rss, 1, 1, 1, 0, 0, 0, 1, \ + RX_SEC_F | TS_F | MARK_F | RSS_F) \ +R(sec_ts_mark_ptype, 1, 1, 1, 0, 0, 1, 0, \ + RX_SEC_F | TS_F | MARK_F | PTYPE_F) \ +R(sec_ts_mark_ptype_rss, 1, 1, 1, 0, 0, 1, 1, \ + RX_SEC_F | TS_F | MARK_F | PTYPE_F | RSS_F) \ +R(sec_ts_mark_cksum, 1, 1, 1, 0, 1, 0, 0, \ + RX_SEC_F | TS_F | MARK_F | CKSUM_F) \ +R(sec_ts_mark_cksum_rss, 1, 1, 1, 0, 1, 0, 1, \ + RX_SEC_F | TS_F | MARK_F | CKSUM_F | RSS_F) \ +R(sec_ts_mark_cksum_ptype, 1, 1, 1, 0, 1, 1, 0, \ + RX_SEC_F | TS_F | MARK_F | CKSUM_F | PTYPE_F) \ +R(sec_ts_mark_cksum_ptype_rss, 1, 1, 1, 0, 1, 1, 1, \ + RX_SEC_F | TS_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(sec_ts_mark_vlan, 1, 1, 1, 1, 0, 0, 0, \ + RX_SEC_F | TS_F | MARK_F | RX_VLAN_F) \ +R(sec_ts_mark_vlan_rss, 1, 1, 1, 1, 0, 0, 1, \ + RX_SEC_F | RX_VLAN_F | RSS_F) \ +R(sec_ts_mark_vlan_ptype, 1, 1, 1, 1, 0, 1, 0, \ + RX_SEC_F | TS_F | MARK_F | RX_VLAN_F | PTYPE_F) \ +R(sec_ts_mark_vlan_ptype_rss, 1, 1, 1, 1, 0, 1, 1, \ + RX_SEC_F | TS_F | MARK_F | RX_VLAN_F | PTYPE_F | RSS_F)\ +R(sec_ts_mark_vlan_cksum, 1, 1, 1, 1, 1, 0, 0, \ + RX_SEC_F | TS_F | MARK_F | RX_VLAN_F | CKSUM_F) \ +R(sec_ts_mark_vlan_cksum_rss, 1, 1, 1, 1, 1, 0, 1, \ + RX_SEC_F | TS_F | MARK_F | RX_VLAN_F | CKSUM_F | RSS_F)\ +R(sec_ts_mark_vlan_cksum_ptype, 1, 1, 1, 1, 1, 1, 0, \ + RX_SEC_F | TS_F | MARK_F | RX_VLAN_F | CKSUM_F | \ + PTYPE_F) \ +R(sec_ts_mark_vlan_cksum_ptype_rss, \ + 1, 1, 1, 1, 1, 1, 1, \ + RX_SEC_F | TS_F | MARK_F | RX_VLAN_F | CKSUM_F | \ + PTYPE_F | RSS_F) #endif /* __OTX2_RX_H__ */ From patchwork Sun Dec 8 11:54:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 63638 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4AAB2A04F1; Sun, 8 Dec 2019 12:57:48 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2874A1BF9C; Sun, 8 Dec 2019 12:56:57 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 8C10E1BF90 for ; Sun, 8 Dec 2019 12:56:55 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB8Bq6Kb005629; Sun, 8 Dec 2019 03:56:55 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=cytwBCBTx/+VB88Emr+mue3WHkFAGlN7qi/mMhr0YMI=; b=xlWkmpolCJhbu8e8uOg1xm6eMBWgQtAJJoS2SsFVPrC0iyRbzfbiGQSzsVzLBrUjsjcf zpvkZuZaXB8yNPS+xUaDA7EQYJfy3mn3yG+nWCXE3hJTantu5A9Sk8dyijw7DltFJUcQ eCLw3dHhBNO4ByYtRgXE+VhFslwXjvp1AV6y25PCpjEmDnA4cmGBVpnbWSV0YLiDNXHj fFvKH73GpkraO0Jv8bNavHr8NFTJZNh+ux83pHvooSGmPijdLpdACfoXj1nfexEYmalZ WrM7X3FLTZZjaJI/82FjfLCSlNGjmpI7dacCeeDc7YlTH9/+nLOiq47VWHSCY2s1QW1P HA== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0b-0016f401.pphosted.com with ESMTP id 2wrcfptc38-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 08 Dec 2019 03:56:54 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Sun, 8 Dec 2019 03:56:52 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Sun, 8 Dec 2019 03:56:52 -0800 Received: from ajoseph83.caveonetworks.com.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id 07D063F703F; Sun, 8 Dec 2019 03:56:47 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Declan Doherty , Thomas Monjalon CC: Archana Muniganti , Jerin Jacob , Narayana Prasad , Kiran Kumar K , Nithin Dabilpuram , "Pavan Nikhilesh" , Ankur Dwivedi , Anoob Joseph , Tejasree Kondoj , Vamsi Attunuru , Lukasz Bartosik , Date: Sun, 8 Dec 2019 17:24:52 +0530 Message-ID: <1575806094-28391-14-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1575806094-28391-1-git-send-email-anoobj@marvell.com> References: <1575806094-28391-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-08_03:2019-12-05,2019-12-08 signatures=0 Subject: [dpdk-dev] [PATCH 13/15] drivers/octeontx2: add sec in compiler optimized TX fastpath framework X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Archana Muniganti Added new flag for SECURITY in compiler optimized TX fastpath framework. With this, compiler autogenerates functions which have security enabled. Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Archana Muniganti Signed-off-by: Tejasree Kondoj Signed-off-by: Vamsi Attunuru --- drivers/event/octeontx2/otx2_evdev.c | 36 ++-- drivers/event/octeontx2/otx2_evdev.h | 2 +- drivers/event/octeontx2/otx2_worker.c | 4 +- drivers/event/octeontx2/otx2_worker_dual.c | 4 +- drivers/net/octeontx2/otx2_ethdev.c | 3 + drivers/net/octeontx2/otx2_tx.c | 29 +-- drivers/net/octeontx2/otx2_tx.h | 271 ++++++++++++++++++++++------- 7 files changed, 250 insertions(+), 99 deletions(-) diff --git a/drivers/event/octeontx2/otx2_evdev.c b/drivers/event/octeontx2/otx2_evdev.c index f6c641a..d20213d 100644 --- a/drivers/event/octeontx2/otx2_evdev.c +++ b/drivers/event/octeontx2/otx2_evdev.c @@ -177,35 +177,37 @@ SSO_RX_ADPTR_ENQ_FASTPATH_FUNC }; /* Tx modes */ - const event_tx_adapter_enqueue ssogws_tx_adptr_enq[2][2][2][2][2][2] = { -#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_ssogws_tx_adptr_enq_ ## name, + const event_tx_adapter_enqueue + ssogws_tx_adptr_enq[2][2][2][2][2][2][2] = { +#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = \ + otx2_ssogws_tx_adptr_enq_ ## name, SSO_TX_ADPTR_ENQ_FASTPATH_FUNC #undef T }; const event_tx_adapter_enqueue - ssogws_tx_adptr_enq_seg[2][2][2][2][2][2] = { -#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ - [f5][f4][f3][f2][f1][f0] = \ + ssogws_tx_adptr_enq_seg[2][2][2][2][2][2][2] = { +#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = \ otx2_ssogws_tx_adptr_enq_seg_ ## name, SSO_TX_ADPTR_ENQ_FASTPATH_FUNC #undef T }; const event_tx_adapter_enqueue - ssogws_dual_tx_adptr_enq[2][2][2][2][2][2] = { -#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ - [f5][f4][f3][f2][f1][f0] = \ + ssogws_dual_tx_adptr_enq[2][2][2][2][2][2][2] = { +#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = \ otx2_ssogws_dual_tx_adptr_enq_ ## name, SSO_TX_ADPTR_ENQ_FASTPATH_FUNC #undef T }; const event_tx_adapter_enqueue - ssogws_dual_tx_adptr_enq_seg[2][2][2][2][2][2] = { -#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ - [f5][f4][f3][f2][f1][f0] = \ + ssogws_dual_tx_adptr_enq_seg[2][2][2][2][2][2][2] = { +#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = \ otx2_ssogws_dual_tx_adptr_enq_seg_ ## name, SSO_TX_ADPTR_ENQ_FASTPATH_FUNC #undef T @@ -290,8 +292,9 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC } if (dev->tx_offloads & NIX_TX_MULTI_SEG_F) { - /* [TSMP] [MBUF_NOFF] [VLAN] [OL3_L4_CSUM] [L3_L4_CSUM] */ + /* [SEC] [TSMP] [MBUF_NOFF] [VLAN] [OL3_L4_CSUM] [L3_L4_CSUM] */ event_dev->txa_enqueue = ssogws_tx_adptr_enq_seg + [!!(dev->tx_offloads & NIX_TX_OFFLOAD_SECURITY_F)] [!!(dev->tx_offloads & NIX_TX_OFFLOAD_TSO_F)] [!!(dev->tx_offloads & NIX_TX_OFFLOAD_TSTAMP_F)] [!!(dev->tx_offloads & NIX_TX_OFFLOAD_MBUF_NOFF_F)] @@ -300,6 +303,7 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC [!!(dev->tx_offloads & NIX_TX_OFFLOAD_L3_L4_CSUM_F)]; } else { event_dev->txa_enqueue = ssogws_tx_adptr_enq + [!!(dev->tx_offloads & NIX_TX_OFFLOAD_SECURITY_F)] [!!(dev->tx_offloads & NIX_TX_OFFLOAD_TSO_F)] [!!(dev->tx_offloads & NIX_TX_OFFLOAD_TSTAMP_F)] [!!(dev->tx_offloads & NIX_TX_OFFLOAD_MBUF_NOFF_F)] @@ -440,8 +444,10 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC } if (dev->tx_offloads & NIX_TX_MULTI_SEG_F) { - /* [TSMP] [MBUF_NOFF] [VLAN] [OL3_L4_CSUM] [L3_L4_CSUM] */ + /* [SEC] [TSMP] [MBUF_NOFF] [VLAN] [OL3_L4_CSUM] [L3_L4_CSUM] */ event_dev->txa_enqueue = ssogws_dual_tx_adptr_enq_seg + [!!(dev->tx_offloads & + NIX_TX_OFFLOAD_SECURITY_F)] [!!(dev->tx_offloads & NIX_TX_OFFLOAD_TSO_F)] [!!(dev->tx_offloads & NIX_TX_OFFLOAD_TSTAMP_F)] [!!(dev->tx_offloads & @@ -454,6 +460,8 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC NIX_TX_OFFLOAD_L3_L4_CSUM_F)]; } else { event_dev->txa_enqueue = ssogws_dual_tx_adptr_enq + [!!(dev->tx_offloads & + NIX_TX_OFFLOAD_SECURITY_F)] [!!(dev->tx_offloads & NIX_TX_OFFLOAD_TSO_F)] [!!(dev->tx_offloads & NIX_TX_OFFLOAD_TSTAMP_F)] [!!(dev->tx_offloads & diff --git a/drivers/event/octeontx2/otx2_evdev.h b/drivers/event/octeontx2/otx2_evdev.h index e20e251..019e7a2 100644 --- a/drivers/event/octeontx2/otx2_evdev.h +++ b/drivers/event/octeontx2/otx2_evdev.h @@ -335,7 +335,7 @@ uint16_t otx2_ssogws_dual_deq_seg_timeout_burst_ ##name(void *port, \ SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R -#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t otx2_ssogws_tx_adptr_enq_ ## name(void *port, struct rte_event ev[],\ uint16_t nb_events); \ uint16_t otx2_ssogws_tx_adptr_enq_seg_ ## name(void *port, \ diff --git a/drivers/event/octeontx2/otx2_worker.c b/drivers/event/octeontx2/otx2_worker.c index 12445d9..8bec59e 100644 --- a/drivers/event/octeontx2/otx2_worker.c +++ b/drivers/event/octeontx2/otx2_worker.c @@ -267,7 +267,7 @@ otx2_ssogws_enq_fwd_burst(void *port, const struct rte_event ev[], return 1; } -#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __hot \ otx2_ssogws_tx_adptr_enq_ ## name(void *port, struct rte_event ev[], \ uint16_t nb_events) \ @@ -281,7 +281,7 @@ otx2_ssogws_tx_adptr_enq_ ## name(void *port, struct rte_event ev[], \ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC #undef T -#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __hot \ otx2_ssogws_tx_adptr_enq_seg_ ## name(void *port, struct rte_event ev[],\ uint16_t nb_events) \ diff --git a/drivers/event/octeontx2/otx2_worker_dual.c b/drivers/event/octeontx2/otx2_worker_dual.c index 22a4889..3cba09c 100644 --- a/drivers/event/octeontx2/otx2_worker_dual.c +++ b/drivers/event/octeontx2/otx2_worker_dual.c @@ -307,7 +307,7 @@ otx2_ssogws_dual_deq_seg_timeout_burst_ ##name(void *port, \ SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R -#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __hot \ otx2_ssogws_dual_tx_adptr_enq_ ## name(void *port, \ struct rte_event ev[], \ @@ -324,7 +324,7 @@ otx2_ssogws_dual_tx_adptr_enq_ ## name(void *port, \ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC #undef T -#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __hot \ otx2_ssogws_dual_tx_adptr_enq_seg_ ## name(void *port, \ struct rte_event ev[], \ diff --git a/drivers/net/octeontx2/otx2_ethdev.c b/drivers/net/octeontx2/otx2_ethdev.c index c215078..8801173 100644 --- a/drivers/net/octeontx2/otx2_ethdev.c +++ b/drivers/net/octeontx2/otx2_ethdev.c @@ -698,6 +698,9 @@ nix_tx_offload_flags(struct rte_eth_dev *eth_dev) NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | NIX_TX_OFFLOAD_L3_L4_CSUM_F); + if (conf & DEV_TX_OFFLOAD_SECURITY) + flags |= NIX_TX_OFFLOAD_SECURITY_F; + if ((dev->rx_offloads & DEV_RX_OFFLOAD_TIMESTAMP)) flags |= NIX_TX_OFFLOAD_TSTAMP_F; diff --git a/drivers/net/octeontx2/otx2_tx.c b/drivers/net/octeontx2/otx2_tx.c index fa53300..e43590e 100644 --- a/drivers/net/octeontx2/otx2_tx.c +++ b/drivers/net/octeontx2/otx2_tx.c @@ -945,7 +945,7 @@ nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, } #endif -#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ static uint16_t __rte_noinline __hot \ otx2_nix_xmit_pkts_ ## name(void *tx_queue, \ struct rte_mbuf **tx_pkts, uint16_t pkts) \ @@ -962,7 +962,7 @@ otx2_nix_xmit_pkts_ ## name(void *tx_queue, \ NIX_TX_FASTPATH_MODES #undef T -#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ static uint16_t __rte_noinline __hot \ otx2_nix_xmit_pkts_mseg_ ## name(void *tx_queue, \ struct rte_mbuf **tx_pkts, uint16_t pkts) \ @@ -980,7 +980,7 @@ otx2_nix_xmit_pkts_mseg_ ## name(void *tx_queue, \ NIX_TX_FASTPATH_MODES #undef T -#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ static uint16_t __rte_noinline __hot \ otx2_nix_xmit_pkts_vec_ ## name(void *tx_queue, \ struct rte_mbuf **tx_pkts, uint16_t pkts) \ @@ -998,12 +998,13 @@ NIX_TX_FASTPATH_MODES static inline void pick_tx_func(struct rte_eth_dev *eth_dev, - const eth_tx_burst_t tx_burst[2][2][2][2][2][2]) + const eth_tx_burst_t tx_burst[2][2][2][2][2][2][2]) { struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev); - /* [TSTMP] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ + /* [SEC] [TSTMP] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ eth_dev->tx_pkt_burst = tx_burst + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_SECURITY_F)] [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSO_F)] [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSTAMP_F)] [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)] @@ -1017,25 +1018,25 @@ otx2_eth_set_tx_function(struct rte_eth_dev *eth_dev) { struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev); - const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2][2] = { -#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_nix_xmit_pkts_ ## name, + const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2][2][2] = { +#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = otx2_nix_xmit_pkts_ ## name, NIX_TX_FASTPATH_MODES #undef T }; - const eth_tx_burst_t nix_eth_tx_burst_mseg[2][2][2][2][2][2] = { -#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_nix_xmit_pkts_mseg_ ## name, + const eth_tx_burst_t nix_eth_tx_burst_mseg[2][2][2][2][2][2][2] = { +#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = otx2_nix_xmit_pkts_mseg_ ## name, NIX_TX_FASTPATH_MODES #undef T }; - const eth_tx_burst_t nix_eth_tx_vec_burst[2][2][2][2][2][2] = { -#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_nix_xmit_pkts_vec_ ## name, + const eth_tx_burst_t nix_eth_tx_vec_burst[2][2][2][2][2][2][2] = { +#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = otx2_nix_xmit_pkts_vec_ ## name, NIX_TX_FASTPATH_MODES #undef T diff --git a/drivers/net/octeontx2/otx2_tx.h b/drivers/net/octeontx2/otx2_tx.h index 04e859b..3c43170 100644 --- a/drivers/net/octeontx2/otx2_tx.h +++ b/drivers/net/octeontx2/otx2_tx.h @@ -12,6 +12,7 @@ #define NIX_TX_OFFLOAD_MBUF_NOFF_F BIT(3) #define NIX_TX_OFFLOAD_TSTAMP_F BIT(4) #define NIX_TX_OFFLOAD_TSO_F BIT(5) +#define NIX_TX_OFFLOAD_SECURITY_F BIT(6) /* Flags to control xmit_prepare function. * Defining it from backwards to denote its been @@ -470,136 +471,274 @@ otx2_nix_xmit_mseg_one(uint64_t *cmd, void *lmt_addr, #define NOFF_F NIX_TX_OFFLOAD_MBUF_NOFF_F #define TSP_F NIX_TX_OFFLOAD_TSTAMP_F #define TSO_F NIX_TX_OFFLOAD_TSO_F +#define TX_SEC_F NIX_TX_OFFLOAD_SECURITY_F -/* [TSO] [TSTMP] [NOFF] [VLAN] [OL3OL4CSUM] [L3L4CSUM] */ +/* [SEC] [TSO] [TSTMP] [NOFF] [VLAN] [OL3OL4CSUM] [L3L4CSUM] */ #define NIX_TX_FASTPATH_MODES \ -T(no_offload, 0, 0, 0, 0, 0, 0, 4, \ +T(no_offload, 0, 0, 0, 0, 0, 0, 0, 4, \ NIX_TX_OFFLOAD_NONE) \ -T(l3l4csum, 0, 0, 0, 0, 0, 1, 4, \ +T(l3l4csum, 0, 0, 0, 0, 0, 0, 1, 4, \ L3L4CSUM_F) \ -T(ol3ol4csum, 0, 0, 0, 0, 1, 0, 4, \ +T(ol3ol4csum, 0, 0, 0, 0, 0, 1, 0, 4, \ OL3OL4CSUM_F) \ -T(ol3ol4csum_l3l4csum, 0, 0, 0, 0, 1, 1, 4, \ +T(ol3ol4csum_l3l4csum, 0, 0, 0, 0, 0, 1, 1, 4, \ OL3OL4CSUM_F | L3L4CSUM_F) \ -T(vlan, 0, 0, 0, 1, 0, 0, 6, \ +T(vlan, 0, 0, 0, 0, 1, 0, 0, 6, \ VLAN_F) \ -T(vlan_l3l4csum, 0, 0, 0, 1, 0, 1, 6, \ +T(vlan_l3l4csum, 0, 0, 0, 0, 1, 0, 1, 6, \ VLAN_F | L3L4CSUM_F) \ -T(vlan_ol3ol4csum, 0, 0, 0, 1, 1, 0, 6, \ +T(vlan_ol3ol4csum, 0, 0, 0, 0, 1, 1, 0, 6, \ VLAN_F | OL3OL4CSUM_F) \ -T(vlan_ol3ol4csum_l3l4csum, 0, 0, 0, 1, 1, 1, 6, \ +T(vlan_ol3ol4csum_l3l4csum, 0, 0, 0, 0, 1, 1, 1, 6, \ VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(noff, 0, 0, 1, 0, 0, 0, 4, \ +T(noff, 0, 0, 0, 1, 0, 0, 0, 4, \ NOFF_F) \ -T(noff_l3l4csum, 0, 0, 1, 0, 0, 1, 4, \ +T(noff_l3l4csum, 0, 0, 0, 1, 0, 0, 1, 4, \ NOFF_F | L3L4CSUM_F) \ -T(noff_ol3ol4csum, 0, 0, 1, 0, 1, 0, 4, \ +T(noff_ol3ol4csum, 0, 0, 0, 1, 0, 1, 0, 4, \ NOFF_F | OL3OL4CSUM_F) \ -T(noff_ol3ol4csum_l3l4csum, 0, 0, 1, 0, 1, 1, 4, \ +T(noff_ol3ol4csum_l3l4csum, 0, 0, 0, 1, 0, 1, 1, 4, \ NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(noff_vlan, 0, 0, 1, 1, 0, 0, 6, \ +T(noff_vlan, 0, 0, 0, 1, 1, 0, 0, 6, \ NOFF_F | VLAN_F) \ -T(noff_vlan_l3l4csum, 0, 0, 1, 1, 0, 1, 6, \ +T(noff_vlan_l3l4csum, 0, 0, 0, 1, 1, 0, 1, 6, \ NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(noff_vlan_ol3ol4csum, 0, 0, 1, 1, 1, 0, 6, \ +T(noff_vlan_ol3ol4csum, 0, 0, 0, 1, 1, 1, 0, 6, \ NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(noff_vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 1, 1, 6, \ +T(noff_vlan_ol3ol4csum_l3l4csum, 0, 0, 0, 1, 1, 1, 1, 6, \ NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(ts, 0, 1, 0, 0, 0, 0, 8, \ +T(ts, 0, 0, 1, 0, 0, 0, 0, 8, \ TSP_F) \ -T(ts_l3l4csum, 0, 1, 0, 0, 0, 1, 8, \ +T(ts_l3l4csum, 0, 0, 1, 0, 0, 0, 1, 8, \ TSP_F | L3L4CSUM_F) \ -T(ts_ol3ol4csum, 0, 1, 0, 0, 1, 0, 8, \ +T(ts_ol3ol4csum, 0, 0, 1, 0, 0, 1, 0, 8, \ TSP_F | OL3OL4CSUM_F) \ -T(ts_ol3ol4csum_l3l4csum, 0, 1, 0, 0, 1, 1, 8, \ +T(ts_ol3ol4csum_l3l4csum, 0, 0, 1, 0, 0, 1, 1, 8, \ TSP_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(ts_vlan, 0, 1, 0, 1, 0, 0, 8, \ +T(ts_vlan, 0, 0, 1, 0, 1, 0, 0, 8, \ TSP_F | VLAN_F) \ -T(ts_vlan_l3l4csum, 0, 1, 0, 1, 0, 1, 8, \ +T(ts_vlan_l3l4csum, 0, 0, 1, 0, 1, 0, 1, 8, \ TSP_F | VLAN_F | L3L4CSUM_F) \ -T(ts_vlan_ol3ol4csum, 0, 1, 0, 1, 1, 0, 8, \ +T(ts_vlan_ol3ol4csum, 0, 0, 1, 0, 1, 1, 0, 8, \ TSP_F | VLAN_F | OL3OL4CSUM_F) \ -T(ts_vlan_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 1, 1, 8, \ +T(ts_vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 0, 1, 1, 1, 8, \ TSP_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(ts_noff, 0, 1, 1, 0, 0, 0, 8, \ +T(ts_noff, 0, 0, 1, 1, 0, 0, 0, 8, \ TSP_F | NOFF_F) \ -T(ts_noff_l3l4csum, 0, 1, 1, 0, 0, 1, 8, \ +T(ts_noff_l3l4csum, 0, 0, 1, 1, 0, 0, 1, 8, \ TSP_F | NOFF_F | L3L4CSUM_F) \ -T(ts_noff_ol3ol4csum, 0, 1, 1, 0, 1, 0, 8, \ +T(ts_noff_ol3ol4csum, 0, 0, 1, 1, 0, 1, 0, 8, \ TSP_F | NOFF_F | OL3OL4CSUM_F) \ -T(ts_noff_ol3ol4csum_l3l4csum, 0, 1, 1, 0, 1, 1, 8, \ +T(ts_noff_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 0, 1, 1, 8, \ TSP_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(ts_noff_vlan, 0, 1, 1, 1, 0, 0, 8, \ +T(ts_noff_vlan, 0, 0, 1, 1, 1, 0, 0, 8, \ TSP_F | NOFF_F | VLAN_F) \ -T(ts_noff_vlan_l3l4csum, 0, 1, 1, 1, 0, 1, 8, \ +T(ts_noff_vlan_l3l4csum, 0, 0, 1, 1, 1, 0, 1, 8, \ TSP_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(ts_noff_vlan_ol3ol4csum, 0, 1, 1, 1, 1, 0, 8, \ +T(ts_noff_vlan_ol3ol4csum, 0, 0, 1, 1, 1, 1, 0, 8, \ TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(ts_noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 1, 1, 8, \ +T(ts_noff_vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 1, 1, 1, 8, \ TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ \ -T(tso, 1, 0, 0, 0, 0, 0, 6, \ +T(tso, 0, 1, 0, 0, 0, 0, 0, 6, \ TSO_F) \ -T(tso_l3l4csum, 1, 0, 0, 0, 0, 1, 6, \ +T(tso_l3l4csum, 0, 1, 0, 0, 0, 0, 1, 6, \ TSO_F | L3L4CSUM_F) \ -T(tso_ol3ol4csum, 1, 0, 0, 0, 1, 0, 6, \ +T(tso_ol3ol4csum, 0, 1, 0, 0, 0, 1, 0, 6, \ TSO_F | OL3OL4CSUM_F) \ -T(tso_ol3ol4csum_l3l4csum, 1, 0, 0, 0, 1, 1, 6, \ +T(tso_ol3ol4csum_l3l4csum, 0, 1, 0, 0, 0, 1, 1, 6, \ TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_vlan, 1, 0, 0, 1, 0, 0, 6, \ +T(tso_vlan, 0, 1, 0, 0, 1, 0, 0, 6, \ TSO_F | VLAN_F) \ -T(tso_vlan_l3l4csum, 1, 0, 0, 1, 0, 1, 6, \ +T(tso_vlan_l3l4csum, 0, 1, 0, 0, 1, 0, 1, 6, \ TSO_F | VLAN_F | L3L4CSUM_F) \ -T(tso_vlan_ol3ol4csum, 1, 0, 0, 1, 1, 0, 6, \ +T(tso_vlan_ol3ol4csum, 0, 1, 0, 0, 1, 1, 0, 6, \ TSO_F | VLAN_F | OL3OL4CSUM_F) \ -T(tso_vlan_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 1, 1, 6, \ +T(tso_vlan_ol3ol4csum_l3l4csum, 0, 1, 0, 0, 1, 1, 1, 6, \ TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_noff, 1, 0, 1, 0, 0, 0, 6, \ +T(tso_noff, 0, 1, 0, 1, 0, 0, 0, 6, \ TSO_F | NOFF_F) \ -T(tso_noff_l3l4csum, 1, 0, 1, 0, 0, 1, 6, \ +T(tso_noff_l3l4csum, 0, 1, 0, 1, 0, 0, 1, 6, \ TSO_F | NOFF_F | L3L4CSUM_F) \ -T(tso_noff_ol3ol4csum, 1, 0, 1, 0, 1, 0, 6, \ +T(tso_noff_ol3ol4csum, 0, 1, 0, 1, 0, 1, 0, 6, \ TSO_F | NOFF_F | OL3OL4CSUM_F) \ -T(tso_noff_ol3ol4csum_l3l4csum, 1, 0, 1, 0, 1, 1, 6, \ +T(tso_noff_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 0, 1, 1, 6, \ TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_noff_vlan, 1, 0, 1, 1, 0, 0, 6, \ +T(tso_noff_vlan, 0, 1, 0, 1, 1, 0, 0, 6, \ TSO_F | NOFF_F | VLAN_F) \ -T(tso_noff_vlan_l3l4csum, 1, 0, 1, 1, 0, 1, 6, \ +T(tso_noff_vlan_l3l4csum, 0, 1, 0, 1, 1, 0, 1, 6, \ TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(tso_noff_vlan_ol3ol4csum, 1, 0, 1, 1, 1, 0, 6, \ +T(tso_noff_vlan_ol3ol4csum, 0, 1, 0, 1, 1, 1, 0, 6, \ TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 1, 1, 6, \ +T(tso_noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 1, 1, 1, 6, \ TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_ts, 1, 1, 0, 0, 0, 0, 8, \ +T(tso_ts, 0, 1, 1, 0, 0, 0, 0, 8, \ TSO_F | TSP_F) \ -T(tso_ts_l3l4csum, 1, 1, 0, 0, 0, 1, 8, \ +T(tso_ts_l3l4csum, 0, 1, 1, 0, 0, 0, 1, 8, \ TSO_F | TSP_F | L3L4CSUM_F) \ -T(tso_ts_ol3ol4csum, 1, 1, 0, 0, 1, 0, 8, \ +T(tso_ts_ol3ol4csum, 0, 1, 1, 0, 0, 1, 0, 8, \ TSO_F | TSP_F | OL3OL4CSUM_F) \ -T(tso_ts_ol3ol4csum_l3l4csum, 1, 1, 0, 0, 1, 1, 8, \ +T(tso_ts_ol3ol4csum_l3l4csum, 0, 1, 1, 0, 0, 1, 1, 8, \ TSO_F | TSP_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_ts_vlan, 1, 1, 0, 1, 0, 0, 8, \ +T(tso_ts_vlan, 0, 1, 1, 0, 1, 0, 0, 8, \ TSO_F | TSP_F | VLAN_F) \ -T(tso_ts_vlan_l3l4csum, 1, 1, 0, 1, 0, 1, 8, \ +T(tso_ts_vlan_l3l4csum, 0, 1, 1, 0, 1, 0, 1, 8, \ TSO_F | TSP_F | VLAN_F | L3L4CSUM_F) \ -T(tso_ts_vlan_ol3ol4csum, 1, 1, 0, 1, 1, 0, 8, \ +T(tso_ts_vlan_ol3ol4csum, 0, 1, 1, 0, 1, 1, 0, 8, \ TSO_F | TSP_F | VLAN_F | OL3OL4CSUM_F) \ -T(tso_ts_vlan_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 1, 1, 8, \ +T(tso_ts_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 0, 1, 1, 1, 8, \ TSO_F | TSP_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_ts_noff, 1, 1, 1, 0, 0, 0, 8, \ +T(tso_ts_noff, 0, 1, 1, 1, 0, 0, 0, 8, \ TSO_F | TSP_F | NOFF_F) \ -T(tso_ts_noff_l3l4csum, 1, 1, 1, 0, 0, 1, 8, \ +T(tso_ts_noff_l3l4csum, 0, 1, 1, 1, 0, 0, 1, 8, \ TSO_F | TSP_F | NOFF_F | L3L4CSUM_F) \ -T(tso_ts_noff_ol3ol4csum, 1, 1, 1, 0, 1, 0, 8, \ +T(tso_ts_noff_ol3ol4csum, 0, 1, 1, 1, 0, 1, 0, 8, \ TSO_F | TSP_F | NOFF_F | OL3OL4CSUM_F) \ -T(tso_ts_noff_ol3ol4csum_l3l4csum, 1, 1, 1, 0, 1, 1, 8, \ +T(tso_ts_noff_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 0, 1, 1, 8, \ TSO_F | TSP_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_ts_noff_vlan, 1, 1, 1, 1, 0, 0, 8, \ +T(tso_ts_noff_vlan, 0, 1, 1, 1, 1, 0, 0, 8, \ TSO_F | TSP_F | NOFF_F | VLAN_F) \ -T(tso_ts_noff_vlan_l3l4csum, 1, 1, 1, 1, 0, 1, 8, \ +T(tso_ts_noff_vlan_l3l4csum, 0, 1, 1, 1, 1, 0, 1, 8, \ TSO_F | TSP_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(tso_ts_noff_vlan_ol3ol4csum, 1, 1, 1, 1, 1, 0, 8, \ +T(tso_ts_noff_vlan_ol3ol4csum, 0, 1, 1, 1, 1, 1, 0, 8, \ TSO_F | TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(tso_ts_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 1, 8, \ - TSO_F | TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) +T(tso_ts_noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 1, 1, 1, 8, \ + TSO_F | TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | \ + L3L4CSUM_F) \ +T(sec, 1, 0, 0, 0, 0, 0, 0, 8, \ + TX_SEC_F) \ +T(sec_l3l4csum, 1, 0, 0, 0, 0, 0, 1, 8, \ + TX_SEC_F | L3L4CSUM_F) \ +T(sec_ol3ol4csum, 1, 0, 0, 0, 0, 1, 0, 8, \ + TX_SEC_F | OL3OL4CSUM_F) \ +T(sec_ol3ol4csum_l3l4csum, 1, 0, 0, 0, 0, 1, 1, 8, \ + TX_SEC_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(sec_vlan, 1, 0, 0, 0, 1, 0, 0, 8, \ + TX_SEC_F | VLAN_F) \ +T(sec_vlan_l3l4csum, 1, 0, 0, 0, 1, 0, 1, 8, \ + TX_SEC_F | VLAN_F | L3L4CSUM_F) \ +T(sec_vlan_ol3ol4csum, 1, 0, 0, 0, 1, 1, 0, 8, \ + TX_SEC_F | VLAN_F | OL3OL4CSUM_F) \ +T(sec_vlan_ol3ol4csum_l3l4csum, 1, 0, 0, 0, 1, 1, 1, 8, \ + TX_SEC_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(sec_noff, 1, 0, 0, 1, 0, 0, 0, 8, \ + TX_SEC_F | NOFF_F) \ +T(sec_noff_l3l4csum, 1, 0, 0, 1, 0, 0, 1, 8, \ + TX_SEC_F | NOFF_F | L3L4CSUM_F) \ +T(sec_noff_ol3ol4csum, 1, 0, 0, 1, 0, 1, 0, 8, \ + TX_SEC_F | NOFF_F | OL3OL4CSUM_F) \ +T(sec_noff_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 0, 1, 1, 8, \ + TX_SEC_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(sec_noff_vlan, 1, 0, 0, 1, 1, 0, 0, 8, \ + TX_SEC_F | NOFF_F | VLAN_F) \ +T(sec_noff_vlan_l3l4csum, 1, 0, 0, 1, 1, 0, 1, 8, \ + TX_SEC_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(sec_noff_vlan_ol3ol4csum, 1, 0, 0, 1, 1, 1, 0, 8, \ + TX_SEC_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(sec_noff_vlan_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 1, 1, 1, 8, \ + TX_SEC_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(sec_ts, 1, 0, 1, 0, 0, 0, 0, 8, \ + TX_SEC_F | TSP_F) \ +T(sec_ts_l3l4csum, 1, 0, 1, 0, 0, 0, 1, 8, \ + TX_SEC_F | TSP_F | L3L4CSUM_F) \ +T(sec_ts_ol3ol4csum, 1, 0, 1, 0, 0, 1, 0, 8, \ + TX_SEC_F | TSP_F | OL3OL4CSUM_F) \ +T(sec_ts_ol3ol4csum_l3l4csum, 1, 0, 1, 0, 0, 1, 1, 8, \ + TX_SEC_F | TSP_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(sec_ts_vlan, 1, 0, 1, 0, 1, 0, 0, 8, \ + TX_SEC_F | TSP_F | VLAN_F) \ +T(sec_ts_vlan_l3l4csum, 1, 0, 1, 0, 1, 0, 1, 8, \ + TX_SEC_F | TSP_F | VLAN_F | L3L4CSUM_F) \ +T(sec_ts_vlan_ol3ol4csum, 1, 0, 1, 0, 1, 1, 0, 8, \ + TX_SEC_F | TSP_F | VLAN_F | OL3OL4CSUM_F) \ +T(sec_ts_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 0, 1, 1, 1, 8, \ + TX_SEC_F | TSP_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(sec_ts_noff, 1, 0, 1, 1, 0, 0, 0, 8, \ + TX_SEC_F | TSP_F | NOFF_F) \ +T(sec_ts_noff_l3l4csum, 1, 0, 1, 1, 0, 0, 1, 8, \ + TX_SEC_F | TSP_F | NOFF_F | L3L4CSUM_F) \ +T(sec_ts_noff_ol3ol4csum, 1, 0, 1, 1, 0, 1, 0, 8, \ + TX_SEC_F | TSP_F | NOFF_F | OL3OL4CSUM_F) \ +T(sec_ts_noff_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 0, 1, 1, 8, \ + TX_SEC_F | TSP_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(sec_ts_noff_vlan, 1, 0, 1, 1, 1, 0, 0, 8, \ + TX_SEC_F | TSP_F | NOFF_F | VLAN_F) \ +T(sec_ts_noff_vlan_l3l4csum, 1, 0, 1, 1, 1, 0, 1, 8, \ + TX_SEC_F | TSP_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(sec_ts_noff_vlan_ol3ol4csum, 1, 0, 1, 1, 1, 1, 0, 8, \ + TX_SEC_F | TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(sec_ts_noff_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 1, 1, 1, 8, \ + TX_SEC_F | TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | \ + L3L4CSUM_F) \ +T(sec_tso, 1, 1, 0, 0, 0, 0, 0, 8, \ + TX_SEC_F | TSO_F) \ +T(sec_tso_l3l4csum, 1, 1, 0, 0, 0, 0, 1, 8, \ + TX_SEC_F | TSO_F | L3L4CSUM_F) \ +T(sec_tso_ol3ol4csum, 1, 1, 0, 0, 0, 1, 0, 8, \ + TX_SEC_F | TSO_F | OL3OL4CSUM_F) \ +T(sec_tso_ol3ol4csum_l3l4csum, 1, 1, 0, 0, 0, 1, 1, 8, \ + TX_SEC_F | TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(sec_tso_vlan, 1, 1, 0, 0, 1, 0, 0, 8, \ + TX_SEC_F | TSO_F | VLAN_F) \ +T(sec_tso_vlan_l3l4csum, 1, 1, 0, 0, 1, 0, 1, 8, \ + TX_SEC_F | TSO_F | VLAN_F | L3L4CSUM_F) \ +T(sec_tso_vlan_ol3ol4csum, 1, 1, 0, 0, 1, 1, 0, 8, \ + TX_SEC_F | TSO_F | VLAN_F | OL3OL4CSUM_F) \ +T(sec_tso_vlan_ol3ol4csum_l3l4csum, 1, 1, 0, 0, 1, 1, 1, 8, \ + TX_SEC_F | TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(sec_tso_noff, 1, 1, 0, 1, 0, 0, 0, 8, \ + TX_SEC_F | TSO_F | NOFF_F) \ +T(sec_tso_noff_l3l4csum, 1, 1, 0, 1, 0, 0, 1, 8, \ + TX_SEC_F | TSO_F | NOFF_F | L3L4CSUM_F) \ +T(sec_tso_noff_ol3ol4csum, 1, 1, 0, 1, 0, 1, 0, 8, \ + TX_SEC_F | TSO_F | NOFF_F | OL3OL4CSUM_F) \ +T(sec_tso_noff_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 0, 1, 1, 8, \ + TX_SEC_F | TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(sec_tso_noff_vlan, 1, 1, 0, 1, 1, 0, 0, 8, \ + TX_SEC_F | TSO_F | NOFF_F | VLAN_F) \ +T(sec_tso_noff_vlan_l3l4csum, 1, 1, 0, 1, 1, 0, 1, 8, \ + TX_SEC_F | TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(sec_tso_noff_vlan_ol3ol4csum, 1, 1, 0, 1, 1, 1, 0, 8, \ + TX_SEC_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(sec_tso_noff_vlan_ol3ol4csum_l3l4csum, \ + 1, 1, 0, 1, 1, 1, 1, 8, \ + TX_SEC_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | \ + L3L4CSUM_F) \ +T(sec_tso_ts, 1, 1, 1, 0, 0, 0, 0, 8, \ + TX_SEC_F | TSO_F | TSP_F) \ +T(sec_tso_ts_l3l4csum, 1, 1, 1, 0, 0, 0, 1, 8, \ + TX_SEC_F | TSO_F | TSP_F | L3L4CSUM_F) \ +T(sec_tso_ts_ol3ol4csum, 1, 1, 1, 0, 0, 1, 0, 8, \ + TX_SEC_F | TSO_F | TSP_F | OL3OL4CSUM_F) \ +T(sec_tso_ts_ol3ol4csum_l3l4csum, 1, 1, 1, 0, 0, 1, 1, 8, \ + TX_SEC_F | TSO_F | TSP_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(sec_tso_ts_vlan, 1, 1, 1, 0, 1, 0, 0, 8, \ + TX_SEC_F | TSO_F | TSP_F | VLAN_F) \ +T(sec_tso_ts_vlan_l3l4csum, 1, 1, 1, 0, 1, 0, 1, 8, \ + TX_SEC_F | TSO_F | TSP_F | VLAN_F | L3L4CSUM_F) \ +T(sec_tso_ts_vlan_ol3ol4csum, 1, 1, 1, 0, 1, 1, 0, 8, \ + TX_SEC_F | TSO_F | TSP_F | VLAN_F | OL3OL4CSUM_F) \ +T(sec_tso_ts_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 0, 1, 1, 1, 8, \ + TX_SEC_F | TSO_F | TSP_F | VLAN_F | OL3OL4CSUM_F | \ + L3L4CSUM_F) \ +T(sec_tso_ts_noff, 1, 1, 1, 1, 0, 0, 0, 8, \ + TX_SEC_F | TSO_F | TSP_F | NOFF_F) \ +T(sec_tso_ts_noff_l3l4csum, 1, 1, 1, 1, 0, 0, 1, 8, \ + TX_SEC_F | TSO_F | TSP_F | NOFF_F | L3L4CSUM_F) \ +T(sec_tso_ts_noff_ol3ol4csum, 1, 1, 1, 1, 0, 1, 0, 8, \ + TX_SEC_F | TSO_F | TSP_F | NOFF_F | OL3OL4CSUM_F) \ +T(sec_tso_ts_noff_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 0, 1, 1, 8, \ + TX_SEC_F | TSO_F | TSP_F | NOFF_F | OL3OL4CSUM_F | \ + L3L4CSUM_F) \ +T(sec_tso_ts_noff_vlan, 1, 1, 1, 1, 1, 0, 0, 8, \ + TX_SEC_F | TSO_F | TSP_F | NOFF_F | VLAN_F) \ +T(sec_tso_ts_noff_vlan_l3l4csum, 1, 1, 1, 1, 1, 0, 1, 8, \ + TX_SEC_F | TSO_F | TSP_F | NOFF_F | VLAN_F | L3L4CSUM_F)\ +T(sec_tso_ts_noff_vlan_ol3ol4csum, 1, 1, 1, 1, 1, 1, 0, 8, \ + TX_SEC_F | TSO_F | TSP_F | NOFF_F | VLAN_F | \ + OL3OL4CSUM_F) \ +T(sec_tso_ts_noff_vlan_ol3ol4csum_l3l4csum, \ + 1, 1, 1, 1, 1, 1, 1, 8, \ + TX_SEC_F | TSO_F | TSP_F | NOFF_F | VLAN_F | \ + OL3OL4CSUM_F | L3L4CSUM_F) #endif /* __OTX2_TX_H__ */ From patchwork Sun Dec 8 11:54:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 63639 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 97C89A04F1; Sun, 8 Dec 2019 12:57:59 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2675D1BF7F; Sun, 8 Dec 2019 12:57:04 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 1AA701BFA1 for ; Sun, 8 Dec 2019 12:57:02 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB8Bqi8m006520; Sun, 8 Dec 2019 03:57:01 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=YJTfr7le9/tQ02ksQO11HI5z2RCzOer/ss3jPOvXyto=; b=keX/jFnb7xdA6fQJYFag9qz3eh/ML5UD5NAxBMg33Yxo5NGB531yMDe/eapyQemgUpiI CgqTQlIaAeeQ4HbyJ4s60KmuVor9hYfFOIJ8T9AccvXaLr5yGwPZAimEurXqSXzjJ7/Y jg5zUUBPOGev5yHjyTKnQlJQysajUH79Ok6jaQ0Ph+KJ2nH1sdYtSR+GBDTOvC4mVf4V UqzaxrMQb28vxl4MQejucyE/PxxOSmB2OombM9em+OfK55w62XxSe+2EnzXBBJgYeYmg QTpRDtJU1U+omIIALy9a+0ExbF5hDu2M+06Kz9lZVASjHr8sQUvrzIDzj4IPEWWeS8P5 FQ== Received: from sc-exch02.marvell.com ([199.233.58.182]) by mx0b-0016f401.pphosted.com with ESMTP id 2wrcfptc3k-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 08 Dec 2019 03:57:01 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Sun, 8 Dec 2019 03:56:59 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Sun, 8 Dec 2019 03:56:59 -0800 Received: from ajoseph83.caveonetworks.com.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id 6A19B3F703F; Sun, 8 Dec 2019 03:56:54 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Declan Doherty , Thomas Monjalon CC: Ankur Dwivedi , Jerin Jacob , Narayana Prasad , Kiran Kumar K , Nithin Dabilpuram , "Pavan Nikhilesh" , Anoob Joseph , Archana Muniganti , Tejasree Kondoj , Vamsi Attunuru , "Lukasz Bartosik" , Date: Sun, 8 Dec 2019 17:24:53 +0530 Message-ID: <1575806094-28391-15-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1575806094-28391-1-git-send-email-anoobj@marvell.com> References: <1575806094-28391-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-08_03:2019-12-05,2019-12-08 signatures=0 Subject: [dpdk-dev] [PATCH 14/15] crypto/octeontx2: add inline tx path changes X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Ankur Dwivedi Adding pre-processing required for inline IPsec outbound packets. Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Archana Muniganti Signed-off-by: Tejasree Kondoj Signed-off-by: Vamsi Attunuru Reviewed-by: Gavin Hu --- drivers/crypto/octeontx2/otx2_security.c | 82 +++++++++++++ drivers/crypto/octeontx2/otx2_security.h | 60 ++++++++++ drivers/crypto/octeontx2/otx2_security_tx.h | 176 ++++++++++++++++++++++++++++ drivers/event/octeontx2/meson.build | 3 +- drivers/event/octeontx2/otx2_worker.h | 6 + 5 files changed, 326 insertions(+), 1 deletion(-) create mode 100644 drivers/crypto/octeontx2/otx2_security_tx.h diff --git a/drivers/crypto/octeontx2/otx2_security.c b/drivers/crypto/octeontx2/otx2_security.c index 4d762d9..7bd5625 100644 --- a/drivers/crypto/octeontx2/otx2_security.c +++ b/drivers/crypto/octeontx2/otx2_security.c @@ -3,12 +3,15 @@ */ #include +#include #include #include +#include #include #include #include #include +#include #include "otx2_cryptodev_qp.h" #include "otx2_ethdev.h" @@ -17,6 +20,15 @@ #define SEC_ETH_MAX_PKT_LEN 1450 +#define AH_HDR_LEN 12 +#define AES_GCM_IV_LEN 8 +#define AES_GCM_MAC_LEN 16 +#define AES_CBC_IV_LEN 16 +#define SHA1_HMAC_LEN 12 + +#define AES_GCM_ROUNDUP_BYTE_LEN 4 +#define AES_CBC_ROUNDUP_BYTE_LEN 16 + struct sec_eth_tag_const { RTE_STD_C11 union { @@ -278,6 +290,60 @@ in_sa_get(uint16_t port, int sa_index) } static int +ipsec_sa_const_set(struct rte_security_ipsec_xform *ipsec, + struct rte_crypto_sym_xform *xform, + struct otx2_sec_session_ipsec_ip *sess) +{ + struct rte_crypto_sym_xform *cipher_xform, *auth_xform; + + sess->partial_len = sizeof(struct rte_ipv4_hdr); + + if (ipsec->proto == RTE_SECURITY_IPSEC_SA_PROTO_ESP) { + sess->partial_len += sizeof(struct rte_esp_hdr); + sess->roundup_len = sizeof(struct rte_esp_tail); + } else if (ipsec->proto == RTE_SECURITY_IPSEC_SA_PROTO_AH) { + sess->partial_len += AH_HDR_LEN; + } else { + return -EINVAL; + } + + if (ipsec->options.udp_encap) + sess->partial_len += sizeof(struct rte_udp_hdr); + + if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) { + if (xform->aead.algo == RTE_CRYPTO_AEAD_AES_GCM) { + sess->partial_len += AES_GCM_IV_LEN; + sess->partial_len += AES_GCM_MAC_LEN; + sess->roundup_byte = AES_GCM_ROUNDUP_BYTE_LEN; + } + return 0; + } + + if (ipsec->direction == RTE_SECURITY_IPSEC_SA_DIR_EGRESS) { + cipher_xform = xform; + auth_xform = xform->next; + } else if (ipsec->direction == RTE_SECURITY_IPSEC_SA_DIR_INGRESS) { + auth_xform = xform; + cipher_xform = xform->next; + } else { + return -EINVAL; + } + if (cipher_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CBC) { + sess->partial_len += AES_CBC_IV_LEN; + sess->roundup_byte = AES_CBC_ROUNDUP_BYTE_LEN; + } else { + return -EINVAL; + } + + if (auth_xform->auth.algo == RTE_CRYPTO_AUTH_SHA1_HMAC) + sess->partial_len += SHA1_HMAC_LEN; + else + return -EINVAL; + + return 0; +} + +static int sec_eth_ipsec_out_sess_create(struct rte_eth_dev *eth_dev, struct rte_security_ipsec_xform *ipsec, struct rte_crypto_sym_xform *crypto_xform, @@ -291,6 +357,7 @@ sec_eth_ipsec_out_sess_create(struct rte_eth_dev *eth_dev, struct otx2_ipsec_fp_sa_ctl *ctl; struct otx2_ipsec_fp_out_sa *sa; struct otx2_sec_session *priv; + struct otx2_cpt_inst_s inst; struct otx2_cpt_qp *qp; priv = get_sec_session_private_data(sec_sess); @@ -305,6 +372,12 @@ sec_eth_ipsec_out_sess_create(struct rte_eth_dev *eth_dev, memset(sess, 0, sizeof(struct otx2_sec_session_ipsec_ip)); + sess->seq = 1; + + ret = ipsec_sa_const_set(ipsec, crypto_xform, sess); + if (ret < 0) + return ret; + memcpy(sa->nonce, &ipsec->salt, 4); if (ipsec->options.udp_encap == 1) { @@ -313,6 +386,9 @@ sec_eth_ipsec_out_sess_create(struct rte_eth_dev *eth_dev, } if (ipsec->mode == RTE_SECURITY_IPSEC_SA_MODE_TUNNEL) { + /* Start ip id from 1 */ + sess->ip_id = 1; + if (ipsec->tunnel.type == RTE_SECURITY_IPSEC_TUNNEL_IPV4) { memcpy(&sa->ip_src, &ipsec->tunnel.ipv4.src_ip, sizeof(struct in_addr)); @@ -346,6 +422,12 @@ sec_eth_ipsec_out_sess_create(struct rte_eth_dev *eth_dev, else return -EINVAL; + /* Determine word 7 of CPT instruction */ + inst.u64[7] = 0; + inst.egrp = OTX2_CPT_EGRP_INLINE_IPSEC; + inst.cptr = rte_mempool_virt2iova(sa); + sess->inst_w7 = inst.u64[7]; + /* Use OPAD & IPAD */ RTE_SET_USED(auth_key); RTE_SET_USED(auth_key_len); diff --git a/drivers/crypto/octeontx2/otx2_security.h b/drivers/crypto/octeontx2/otx2_security.h index 1229685..e576f67 100644 --- a/drivers/crypto/octeontx2/otx2_security.h +++ b/drivers/crypto/octeontx2/otx2_security.h @@ -14,6 +14,15 @@ #define OTX2_MAX_CPT_QP_PER_PORT 64 #define OTX2_MAX_INLINE_PORTS 64 +#define OTX2_CPT_RES_ALIGN 16 +#define OTX2_NIX_SEND_DESC_ALIGN 16 +#define OTX2_CPT_INST_SIZE 64 + +#define OTX2_CPT_EGRP_INLINE_IPSEC 1 + +#define OTX2_CPT_OP_INLINE_IPSEC_OUTB (0x40 | 0x25) +#define OTX2_CPT_OP_INLINE_IPSEC_INB (0x40 | 0x26) + struct otx2_cpt_qp; struct otx2_sec_eth_cfg { @@ -45,6 +54,42 @@ struct otx2_cpt_res { }; }; +struct otx2_cpt_inst_s { + union { + struct { + /* W0 */ + uint64_t nixtxl : 3; + uint64_t doneint : 1; + uint64_t nixtx_addr : 60; + /* W1 */ + uint64_t res_addr : 64; + /* W2 */ + uint64_t tag : 32; + uint64_t tt : 2; + uint64_t grp : 10; + uint64_t rsvd_175_172 : 4; + uint64_t rvu_pf_func : 16; + /* W3 */ + uint64_t qord : 1; + uint64_t rsvd_194_193 : 2; + uint64_t wqe_ptr : 61; + /* W4 */ + uint64_t dlen : 16; + uint64_t param2 : 16; + uint64_t param1 : 16; + uint64_t opcode : 16; + /* W5 */ + uint64_t dptr : 64; + /* W6 */ + uint64_t rptr : 64; + /* W7 */ + uint64_t cptr : 61; + uint64_t egrp : 3; + }; + uint64_t u64[8]; + }; +}; + /* * Security session for inline IPsec protocol offload. This is private data of * inline capable PMD. @@ -68,6 +113,21 @@ struct otx2_sec_session_ipsec_ip { /* CPT LF enqueue register address */ rte_iova_t cpt_nq_reg; + /* Pre calculated lengths and data for a session */ + uint8_t partial_len; + uint8_t roundup_len; + uint8_t roundup_byte; + uint16_t ip_id; + union { + uint64_t esn; + struct { + uint32_t seq; + uint32_t esn_hi; + }; + }; + + uint64_t inst_w7; + /* CPT QP used by SA */ struct otx2_cpt_qp *qp; }; diff --git a/drivers/crypto/octeontx2/otx2_security_tx.h b/drivers/crypto/octeontx2/otx2_security_tx.h new file mode 100644 index 0000000..772ff90 --- /dev/null +++ b/drivers/crypto/octeontx2/otx2_security_tx.h @@ -0,0 +1,176 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2019 Marvell International Ltd. + */ + +#ifndef __OTX2_SECURITY_TX_H__ +#define __OTX2_SECURITY_TX_H__ + +#include +#include + +#include "otx2_security.h" + +struct otx2_ipsec_fp_out_hdr { + uint32_t ip_id; + uint32_t seq; + uint8_t iv[16]; +}; + +static __rte_always_inline int32_t +otx2_ipsec_fp_out_rlen_get(struct otx2_sec_session_ipsec_ip *sess, + uint32_t plen) +{ + uint32_t enc_payload_len; + + enc_payload_len = RTE_ALIGN_CEIL(plen + sess->roundup_len, + sess->roundup_byte); + + return sess->partial_len + enc_payload_len; +} + +static __rte_always_inline void +otx2_ssogws_head_wait(struct otx2_ssogws *ws); + +static __rte_always_inline int +otx2_sec_event_tx(struct otx2_ssogws *ws, struct rte_event *ev, + struct rte_mbuf *m, const struct otx2_eth_txq *txq, + const uint32_t offload_flags) +{ + uint32_t dlen, rlen, desc_headroom, extend_head, extend_tail; + struct otx2_sec_session_ipsec_ip *sess; + struct otx2_ipsec_fp_out_hdr *hdr; + struct otx2_ipsec_fp_out_sa *sa; + uint64_t data_addr, desc_addr; + struct otx2_sec_session *priv; + struct otx2_cpt_inst_s inst; + uint64_t lmt_status; + char *data; + + struct desc { + struct otx2_cpt_res cpt_res __rte_aligned(OTX2_CPT_RES_ALIGN); + struct nix_send_hdr_s nix_hdr + __rte_aligned(OTX2_NIX_SEND_DESC_ALIGN); + union nix_send_sg_s nix_sg; + struct nix_iova_s nix_iova; + } *sd; + + priv = get_sec_session_private_data((void *)(m->udata64)); + sess = &priv->ipsec.ip; + sa = &sess->out_sa; + + RTE_ASSERT(sess->cpt_lmtline != NULL); + RTE_ASSERT(!(offload_flags & (NIX_TX_OFFLOAD_MBUF_NOFF_F | + NIX_TX_OFFLOAD_VLAN_QINQ))); + + dlen = rte_pktmbuf_pkt_len(m) + sizeof(*hdr) - RTE_ETHER_HDR_LEN; + rlen = otx2_ipsec_fp_out_rlen_get(sess, dlen - sizeof(*hdr)); + + RTE_BUILD_BUG_ON(OTX2_CPT_RES_ALIGN % OTX2_NIX_SEND_DESC_ALIGN); + RTE_BUILD_BUG_ON(sizeof(sd->cpt_res) % OTX2_NIX_SEND_DESC_ALIGN); + + extend_head = sizeof(*hdr); + extend_tail = rlen - dlen; + + desc_headroom = (OTX2_CPT_RES_ALIGN - 1) + sizeof(*sd); + + if (unlikely(!rte_pktmbuf_is_contiguous(m)) || + unlikely(rte_pktmbuf_headroom(m) < extend_head + desc_headroom) || + unlikely(rte_pktmbuf_tailroom(m) < extend_tail)) { + goto drop; + } + + /* + * Extend mbuf data to point to the expected packet buffer for NIX. + * This includes the Ethernet header followed by the encrypted IPsec + * payload + */ + rte_pktmbuf_append(m, extend_tail); + data = rte_pktmbuf_prepend(m, extend_head); + data_addr = rte_pktmbuf_mtophys(m); + + /* + * Move the Ethernet header, to insert otx2_ipsec_fp_out_hdr prior + * to the IP header + */ + memcpy(data, data + sizeof(*hdr), RTE_ETHER_HDR_LEN); + + hdr = (struct otx2_ipsec_fp_out_hdr *)(data + RTE_ETHER_HDR_LEN); + + memcpy(hdr->iv, &sa->nonce, 4); + memset(hdr->iv + 4, 0, 12); //TODO: make it random + + /* Keep CPT result and NIX send descriptors in headroom */ + sd = (void *)RTE_PTR_ALIGN(data - desc_headroom, OTX2_CPT_RES_ALIGN); + desc_addr = data_addr - RTE_PTR_DIFF(data, sd); + + /* Prepare CPT instruction */ + + inst.nixtx_addr = (desc_addr + offsetof(struct desc, nix_hdr)) >> 4; + inst.doneint = 0; + inst.nixtxl = 1; + inst.res_addr = desc_addr + offsetof(struct desc, cpt_res); + inst.u64[2] = 0; + inst.u64[3] = 0; + inst.wqe_ptr = desc_addr >> 3; /* FIXME: Handle errors */ + inst.qord = 1; + inst.opcode = OTX2_CPT_OP_INLINE_IPSEC_OUTB; + inst.dlen = dlen; + inst.dptr = data_addr + RTE_ETHER_HDR_LEN; + inst.rptr = sa->ctl.spi; + inst.u64[7] = sess->inst_w7; + + /* First word contains 8 bit completion code & 8 bit uc comp code */ + sd->cpt_res.u16[0] = 0; + + /* Prepare NIX send descriptors for output expected from CPT */ + + sd->nix_hdr.w0.u = 0; + sd->nix_hdr.w1.u = 0; + sd->nix_hdr.w0.sq = txq->sq; + sd->nix_hdr.w0.sizem1 = 1; + sd->nix_hdr.w0.total = rte_pktmbuf_data_len(m); + sd->nix_hdr.w0.aura = npa_lf_aura_handle_to_aura(m->pool->pool_id); + + sd->nix_sg.u = 0; + sd->nix_sg.subdc = NIX_SUBDC_SG; + sd->nix_sg.ld_type = NIX_SENDLDTYPE_LDD; + sd->nix_sg.segs = 1; + sd->nix_sg.seg1_size = rte_pktmbuf_data_len(m); + + sd->nix_iova.addr = rte_mbuf_data_iova(m); + + /* Mark mempool object as "put" since it is freed by NIX */ + __mempool_check_cookies(m->pool, (void **)&m, 1, 0); + + if (!ev->sched_type) + otx2_ssogws_head_wait(ws); + + inst.param1 = sess->esn_hi >> 16; + inst.param2 = sess->esn_hi & 0xffff; + + hdr->seq = rte_cpu_to_be_32(sess->seq); + hdr->ip_id = rte_cpu_to_be_32(sess->ip_id); + + sess->ip_id++; + sess->esn++; + + rte_cio_wmb(); + + do { + otx2_lmt_mov(sess->cpt_lmtline, &inst, 2); + lmt_status = otx2_lmt_submit(sess->cpt_nq_reg); + } while (lmt_status == 0); + + return 1; + +drop: + if (offload_flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) { + /* Don't free if reference count > 1 */ + if (rte_pktmbuf_prefree_seg(m) == NULL) + return 0; + } + rte_pktmbuf_free(m); + return 0; +} + +#endif /* __OTX2_SECURITY_TX_H__ */ diff --git a/drivers/event/octeontx2/meson.build b/drivers/event/octeontx2/meson.build index 56febb8..be4b47a 100644 --- a/drivers/event/octeontx2/meson.build +++ b/drivers/event/octeontx2/meson.build @@ -31,6 +31,7 @@ foreach flag: extra_flags endif endforeach -deps += ['bus_pci', 'common_octeontx2', 'mempool_octeontx2', 'pmd_octeontx2'] +deps += ['bus_pci', 'common_octeontx2', 'cryptodev', 'mempool_octeontx2', 'pmd_octeontx2', + 'security'] includes += include_directories('../../crypto/octeontx2') diff --git a/drivers/event/octeontx2/otx2_worker.h b/drivers/event/octeontx2/otx2_worker.h index 7d161c8..c5ea4dd 100644 --- a/drivers/event/octeontx2/otx2_worker.h +++ b/drivers/event/octeontx2/otx2_worker.h @@ -10,6 +10,7 @@ #include #include "otx2_evdev.h" +#include "otx2_security_tx.h" /* SSO Operations */ @@ -281,6 +282,11 @@ otx2_ssogws_event_tx(struct otx2_ssogws *ws, struct rte_event ev[], const struct otx2_eth_txq *txq = otx2_ssogws_xtract_meta(m); rte_prefetch_non_temporal(txq); + + if ((flags & NIX_TX_OFFLOAD_SECURITY_F) && + (m->ol_flags & PKT_TX_SEC_OFFLOAD)) + return otx2_sec_event_tx(ws, ev, m, txq, flags); + /* Perform header writes before barrier for TSO */ otx2_nix_xmit_prepare_tso(m, flags); otx2_ssogws_order(ws, !ev->sched_type); From patchwork Sun Dec 8 11:54:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 63640 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 215A4A04F1; Sun, 8 Dec 2019 12:58:08 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7A8891BFB1; Sun, 8 Dec 2019 12:57:11 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 0EC5A1BFA1 for ; Sun, 8 Dec 2019 12:57:08 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB8Bq4sx005623; Sun, 8 Dec 2019 03:57:08 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=KwiFWfvEcRpO7oKXeoJOjViwvpCBbtwR6o+oo1A7CdY=; b=L5UFNFkSFgAi3UDncjol88cM1WeepHmZXXP7VhsEjaa8ch17wdXoLLqDRr9js2nZ/03v EnUIVFD6cfxeli9S8bkl8p+x0Tq0owOmMBDa2dDf+ipZrTe/TZL/QEPNVdeK+x9k7R7X lWMmboaBNnVzo2vOULG43w93i7bLNhMu303QDpD6b1jIYcYqshdmja3yDd0LcAagbv6G wGptE+udHLT2IiH+l04kNi2d8nf8hKMzXPBwS+djV4y044K0letfUvEzRpSRBUmlviHk vAj9d7QWHAkjipyBBNf7HzLuoEfdRpGdREs0WB/qVbPtsWqH0jXOPmZV0Z+GuWkYh27D 7Q== Received: from sc-exch01.marvell.com ([199.233.58.181]) by mx0b-0016f401.pphosted.com with ESMTP id 2wrcfptc42-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 08 Dec 2019 03:57:08 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Sun, 8 Dec 2019 03:57:06 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Sun, 8 Dec 2019 03:57:06 -0800 Received: from ajoseph83.caveonetworks.com.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id 886D43F703F; Sun, 8 Dec 2019 03:57:01 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Declan Doherty , Thomas Monjalon CC: Vamsi Attunuru , Jerin Jacob , Narayana Prasad , Kiran Kumar K , Nithin Dabilpuram , "Pavan Nikhilesh" , Ankur Dwivedi , Anoob Joseph , Archana Muniganti , Tejasree Kondoj , Lukasz Bartosik , Date: Sun, 8 Dec 2019 17:24:54 +0530 Message-ID: <1575806094-28391-16-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1575806094-28391-1-git-send-email-anoobj@marvell.com> References: <1575806094-28391-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-08_03:2019-12-05,2019-12-08 signatures=0 Subject: [dpdk-dev] [PATCH 15/15] crypto/octeontx2: sync inline tag type cfg with Rx adapter configuration X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Vamsi Attunuru Tag type configuration for the inline processed packets is set during ethdev configuration, it might conflict with tag type configuration done during Rx adapter configuration which would be setup later. This conflict is fixed as part of flow rule creation by updating tag type config of inline same as Rx adapter configured tag type. Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Archana Muniganti Signed-off-by: Tejasree Kondoj Signed-off-by: Vamsi Attunuru --- drivers/common/octeontx2/otx2_common.h | 2 ++ drivers/crypto/octeontx2/otx2_cryptodev.c | 2 ++ drivers/crypto/octeontx2/otx2_security.c | 28 ++++++++++++++++++++++++++++ drivers/crypto/octeontx2/otx2_security.h | 2 ++ drivers/net/octeontx2/otx2_flow.c | 26 ++++++++++++++++++++++++++ 5 files changed, 60 insertions(+) diff --git a/drivers/common/octeontx2/otx2_common.h b/drivers/common/octeontx2/otx2_common.h index 6456c4b..3e53e5b 100644 --- a/drivers/common/octeontx2/otx2_common.h +++ b/drivers/common/octeontx2/otx2_common.h @@ -81,12 +81,14 @@ typedef int (*otx2_sec_eth_ctx_create_t)(struct rte_eth_dev *eth_dev); typedef void (*otx2_sec_eth_ctx_destroy_t)(struct rte_eth_dev *eth_dev); typedef int (*otx2_sec_eth_init_t)(struct rte_eth_dev *eth_dev); typedef void (*otx2_sec_eth_fini_t)(struct rte_eth_dev *eth_dev); +typedef int (*otx2_sec_eth_update_tag_type_t)(struct rte_eth_dev *eth_dev); struct otx2_sec_eth_crypto_idev_ops { otx2_sec_eth_ctx_create_t ctx_create; otx2_sec_eth_ctx_destroy_t ctx_destroy; otx2_sec_eth_init_t init; otx2_sec_eth_fini_t fini; + otx2_sec_eth_update_tag_type_t update_tag_type; }; extern struct otx2_sec_eth_crypto_idev_ops otx2_sec_idev_ops; diff --git a/drivers/crypto/octeontx2/otx2_cryptodev.c b/drivers/crypto/octeontx2/otx2_cryptodev.c index 34feb82..b944a51 100644 --- a/drivers/crypto/octeontx2/otx2_cryptodev.c +++ b/drivers/crypto/octeontx2/otx2_cryptodev.c @@ -160,4 +160,6 @@ RTE_INIT(otx2_cpt_init_log) otx2_sec_idev_ops.ctx_destroy = otx2_sec_eth_ctx_destroy; otx2_sec_idev_ops.init = otx2_sec_eth_init; otx2_sec_idev_ops.fini = otx2_sec_eth_fini; + otx2_sec_idev_ops.update_tag_type = otx2_sec_eth_update_tag_type; + } diff --git a/drivers/crypto/octeontx2/otx2_security.c b/drivers/crypto/octeontx2/otx2_security.c index 7bd5625..25fdc4e 100644 --- a/drivers/crypto/octeontx2/otx2_security.c +++ b/drivers/crypto/octeontx2/otx2_security.c @@ -749,6 +749,34 @@ sec_eth_ipsec_cfg(struct rte_eth_dev *eth_dev, uint8_t tt) } int +otx2_sec_eth_update_tag_type(struct rte_eth_dev *eth_dev) +{ + struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev); + struct otx2_mbox *mbox = dev->mbox; + struct nix_aq_enq_rsp *rsp; + struct nix_aq_enq_req *aq; + int ret; + + aq = otx2_mbox_alloc_msg_nix_aq_enq(mbox); + aq->qidx = 0; /* Read RQ:0 context */ + aq->ctype = NIX_AQ_CTYPE_RQ; + aq->op = NIX_AQ_INSTOP_READ; + + ret = otx2_mbox_process_msg(mbox, (void *)&rsp); + if (ret < 0) { + otx2_err("Could not read RQ context"); + return ret; + } + + /* Update tag type */ + ret = sec_eth_ipsec_cfg(eth_dev, rsp->rq.sso_tt); + if (ret < 0) + otx2_err("Could not update sec eth tag type"); + + return ret; +} + +int otx2_sec_eth_init(struct rte_eth_dev *eth_dev) { const size_t sa_width = sizeof(struct otx2_ipsec_fp_in_sa); diff --git a/drivers/crypto/octeontx2/otx2_security.h b/drivers/crypto/octeontx2/otx2_security.h index e576f67..3ae6dc6 100644 --- a/drivers/crypto/octeontx2/otx2_security.h +++ b/drivers/crypto/octeontx2/otx2_security.h @@ -146,6 +146,8 @@ int otx2_sec_eth_ctx_create(struct rte_eth_dev *eth_dev); void otx2_sec_eth_ctx_destroy(struct rte_eth_dev *eth_dev); +int otx2_sec_eth_update_tag_type(struct rte_eth_dev *eth_dev); + int otx2_sec_eth_init(struct rte_eth_dev *eth_dev); void otx2_sec_eth_fini(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/octeontx2/otx2_flow.c b/drivers/net/octeontx2/otx2_flow.c index f1fb9f9..dea5337 100644 --- a/drivers/net/octeontx2/otx2_flow.c +++ b/drivers/net/octeontx2/otx2_flow.c @@ -299,6 +299,22 @@ flow_free_rss_action(struct rte_eth_dev *eth_dev, return 0; } +static int +flow_update_sec_tt(struct rte_eth_dev *eth_dev, + const struct rte_flow_action actions[]) +{ + int rc = 0; + + for (; actions->type != RTE_FLOW_ACTION_TYPE_END; actions++) { + if (actions->type == RTE_FLOW_ACTION_TYPE_SECURITY) { + if (otx2_sec_idev_ops.update_tag_type != NULL) + rc = otx2_sec_idev_ops.update_tag_type(eth_dev); + break; + } + } + + return rc; +} static int flow_parse_meta_items(__rte_unused struct otx2_parse_state *pst) @@ -491,6 +507,16 @@ otx2_flow_create(struct rte_eth_dev *dev, goto err_exit; } + if (hw->rx_offloads & DEV_RX_OFFLOAD_SECURITY) { + rc = flow_update_sec_tt(dev, actions); + if (rc != 0) { + rte_flow_error_set(error, EIO, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "Failed to update tt with sec act"); + goto err_exit; + } + } list = &hw->npc_flow.flow_list[flow->priority]; /* List in ascending order of mcam entries */