From patchwork Mon Mar 16 15:29:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harman Kalra X-Patchwork-Id: 66718 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id DBA0BA0559; Mon, 16 Mar 2020 16:30:11 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C58B51C069; Mon, 16 Mar 2020 16:30:01 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 663CB1C028 for ; Mon, 16 Mar 2020 16:29:59 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 02GFKT78029483 for ; Mon, 16 Mar 2020 08:29:58 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=pfpt0818; bh=SMGPXcmtytl8D1IbwK3rv2ieYf/1tElx0Ca9LU3KZ9E=; b=kG8zs1qffPek4oMDI0f9h4BJ8XLG5RVYji9FoDZFGUJsz+VIUjr9IoAfC3OicY/wwJLT TIFs36VxRI79aJl/gVLXRNbhv4GJQI3WaAvPTujb2VTlsFex7F33Q+oko5yZxsa/klVT GUQ6xCa32Cqvk4/cEqXZPaBWEc4qIc3Qs6WHFYZksSHabSxqguKnxdZPqq6M+ou0r5et zIERt9Ya3aTX+GvuWfC2Yoq2tDShJrJUycNA4WCofK2G2ITQKOft2pZZ5xePCxGKWdXP sMn8xfrYCIwEarfglDlVxV4vOtZL3Fl83mfD2mWpRAeodt46G2eCqsBnmn/Npwbvkl/M yg== Received: from sc-exch01.marvell.com ([199.233.58.181]) by mx0b-0016f401.pphosted.com with ESMTP id 2yrxsmf9dv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 16 Mar 2020 08:29:58 -0700 Received: from SC-EXCH04.marvell.com (10.93.176.84) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 16 Mar 2020 08:29:56 -0700 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.177) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Mon, 16 Mar 2020 08:29:55 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Uo3DFWJo5EhGce4AZI6T1eZ6wbdHHSVyC5T4xsb28uNjJALRFk5dzPRouNNgY2b8y9dV8vxUTPjoqoeAvBWDKJfGCTUXskE5pAVkPUK0fnA4UKAfzsh358hMnvIp2Qv/cLBKsg0jozGIOsnvzzOIW6R1IcFoZMNc8xMhPr2ImOmY4/ThssYjL0PymEWFHF/DV1rKQFKpTYbsakzoUPz/y2xZKHyigVov3nFxCb5ZSEkIBhZAIc7GxU6zmvIft/2HF/xsE2oMSxJT5Ioii51L8g45KYYHgWECY51xIG9w2BCVNUEa8ZCSS7mb5V0hK1q9G4uV9unBJkddBq48exPPxQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SMGPXcmtytl8D1IbwK3rv2ieYf/1tElx0Ca9LU3KZ9E=; b=RG7f7CprK4oXnUuhVPWQnxj92QVi2yNsIFSi+6P3puCAP2KtqzVBZ7xlhy3owzNQC5QJA5X4rKEwI+5RakYCHbEGdq5IPnUOdvYoHBvxpo5P0ONasvpCH/IfIFgaKo35hY4HrrGulonCkFo0O8IqNDONxDA7Tz6uJyn7pviBrU7KzhigtDAup6lpApuj2hFJbfZGCID19C1ZcOTy0kGIonhK8XHA4/pY9hAr3xKrosNwwUvvC5Tuf7IFRLNwBWQncwmBwl1NpzTF9+VvbrDYAufdxbrTkQFb3ruofQ85+2t4EgHtv0vCyNVkGCCs2jYvTflGJmL2O1t1xj3YFUYHug== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SMGPXcmtytl8D1IbwK3rv2ieYf/1tElx0Ca9LU3KZ9E=; b=VKck1k37VBoP7uDABMwWkYRBy7YuAp9gg9HBtEa0u/vJ+UYLosOO9T5M7bmLFQjiZVa/VBTMiSjhQsCW6fw3zNiiFPpAZSQHsdbqIJLr2jiwQaCpW0u2ABxW/s6tBSSClyjr3K4w/UebM5MUUVuI/k6o3rRnj/fgTvRfU3TqGyU= Received: from MN2PR18MB2848.namprd18.prod.outlook.com (2603:10b6:208:38::19) by MN2PR18MB3592.namprd18.prod.outlook.com (2603:10b6:208:26c::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2814.19; Mon, 16 Mar 2020 15:29:55 +0000 Received: from MN2PR18MB2848.namprd18.prod.outlook.com ([fe80::f829:3e55:94f6:4efb]) by MN2PR18MB2848.namprd18.prod.outlook.com ([fe80::f829:3e55:94f6:4efb%5]) with mapi id 15.20.2814.021; Mon, 16 Mar 2020 15:29:55 +0000 From: Harman Kalra To: CC: , , Harman Kalra Date: Mon, 16 Mar 2020 20:59:10 +0530 Message-ID: <1584372553-28710-2-git-send-email-hkalra@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1584372553-28710-1-git-send-email-hkalra@marvell.com> References: <1584372553-28710-1-git-send-email-hkalra@marvell.com> X-ClientProxiedBy: BM1PR0101CA0036.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:1a::22) To MN2PR18MB2848.namprd18.prod.outlook.com (2603:10b6:208:38::19) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from hkarlara-OptiPlex-3046.marvell.com (115.113.156.2) by BM1PR0101CA0036.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:1a::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.2814.13 via Frontend Transport; Mon, 16 Mar 2020 15:29:53 +0000 X-Mailer: git-send-email 2.7.4 X-Originating-IP: [115.113.156.2] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 282f945b-092a-478c-5778-08d7c9bee0be X-MS-TrafficTypeDiagnostic: MN2PR18MB3592: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:312; X-Forefront-PRVS: 03449D5DD1 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4636009)(366004)(39850400004)(136003)(376002)(346002)(396003)(199004)(66476007)(30864003)(956004)(478600001)(66556008)(186003)(86362001)(26005)(16526019)(36756003)(66946007)(6486002)(4326008)(6636002)(316002)(55236004)(7696005)(37006003)(52116002)(107886003)(2616005)(8676002)(8936002)(2906002)(81166006)(5660300002)(81156014)(34206002)(6666004); DIR:OUT; SFP:1101; SCL:1; SRVR:MN2PR18MB3592; H:MN2PR18MB2848.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; Received-SPF: None (protection.outlook.com: marvell.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 1qL5ajNPaVFXOriQPiiik7L68eiahS3Wcw6pS1RIuPXH5WiPPqmyIyOT4ADwv5McJ6p/7poeQCLnOt/FWWIpkIXLMha0ebkgWDTL08o40EcHHaz47xV1KDlPv7YZ3URhy4a1FEGIwk7g3aPcb47dQrztVeimzgejuXVNE2gy6k92AANGe/X7HpfSB+J2VOJipS1AFbBfijZk2ZfIhLMjAMw/CVC22xOVwPejwo66/WKt7qouaJICJoZuniEcOXX01jEh4QasTepqDMeqqgldwQ9hhxh0RsbjghaBh3vlaSPuVgCmCQCgmVM3FMsubkbW6dfXJ8OTXARVrmDbE2F0VyslxeIVtbDHfqMOMDfa2ys4L9VQzMhgzNJFLTvBZvP3wurgwJz5j2NIUvgNnha7wkamhgS2OsWQwG3c/VMj5rAVG5bJOA+hoaga8BA7yErq X-MS-Exchange-AntiSpam-MessageData: Qh+5L+3rnn9iAqkULqBPoZphgEeGc71UJ+FfnOJcvA92jhndxPb67O+dKSpcmfTFwwZ9/FsMHGTGCp46rCSQLbm6YElT73aA/sAiDc1epsJ/iNq3bCN+HPDMFfxlsWwjAIL1V+XI+CWxoZ8YiI5x7Q== X-MS-Exchange-CrossTenant-Network-Message-Id: 282f945b-092a-478c-5778-08d7c9bee0be X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Mar 2020 15:29:54.9113 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: MhRMr2wrZg197bKokYvN4fS9QiyzT+urnVRLJwkP75xxHQaipkIhYdydaOPqekml7tQnQxSZsO67kfBvBmCpvQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR18MB3592 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.645 definitions=2020-03-16_06:2020-03-12, 2020-03-16 signatures=0 Subject: [dpdk-dev] [PATCH 1/4] event/octeontx: add multi segment support to eventdev X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Adding support for multi segment to the eventdev PMD. Signed-off-by: Harman Kalra --- drivers/event/octeontx/ssovf_evdev.c | 33 +++++++--- drivers/event/octeontx/ssovf_evdev.h | 13 ++++ drivers/event/octeontx/ssovf_worker.c | 90 ++++++++++++++++++++++++--- drivers/event/octeontx/ssovf_worker.h | 76 +++++++++++++++++++--- 4 files changed, 189 insertions(+), 23 deletions(-) diff --git a/drivers/event/octeontx/ssovf_evdev.c b/drivers/event/octeontx/ssovf_evdev.c index f9e93244f..1024b7284 100644 --- a/drivers/event/octeontx/ssovf_evdev.c +++ b/drivers/event/octeontx/ssovf_evdev.c @@ -146,15 +146,31 @@ ssovf_fastpath_fns_set(struct rte_eventdev *dev) dev->enqueue_burst = ssows_enq_burst; dev->enqueue_new_burst = ssows_enq_new_burst; dev->enqueue_forward_burst = ssows_enq_fwd_burst; - dev->dequeue = ssows_deq; - dev->dequeue_burst = ssows_deq_burst; - dev->txa_enqueue = sso_event_tx_adapter_enqueue; - dev->txa_enqueue_same_dest = dev->txa_enqueue; - if (edev->is_timeout_deq) { - dev->dequeue = ssows_deq_timeout; - dev->dequeue_burst = ssows_deq_timeout_burst; + if (!!(edev->rx_offload_flags & OCCTX_RX_MULTI_SEG_F)) { + dev->dequeue = ssows_deq_mseg; + dev->dequeue_burst = ssows_deq_burst_mseg; + + if (edev->is_timeout_deq) { + dev->dequeue = ssows_deq_timeout_mseg; + dev->dequeue_burst = ssows_deq_timeout_burst_mseg; + } + } else { + dev->dequeue = ssows_deq; + dev->dequeue_burst = ssows_deq_burst; + + if (edev->is_timeout_deq) { + dev->dequeue = ssows_deq_timeout; + dev->dequeue_burst = ssows_deq_timeout_burst; + } } + + if (!!(edev->tx_offload_flags & OCCTX_TX_MULTI_SEG_F)) + dev->txa_enqueue = sso_event_tx_adapter_enqueue_mseg; + else + dev->txa_enqueue = sso_event_tx_adapter_enqueue; + + dev->txa_enqueue_same_dest = dev->txa_enqueue; } static void @@ -411,6 +427,7 @@ ssovf_eth_rx_adapter_queue_add(const struct rte_eventdev *dev, { int ret = 0; const struct octeontx_nic *nic = eth_dev->data->dev_private; + struct ssovf_evdev *edev = ssovf_pmd_priv(dev); pki_mod_qos_t pki_qos; RTE_SET_USED(dev); @@ -447,6 +464,8 @@ ssovf_eth_rx_adapter_queue_add(const struct rte_eventdev *dev, ssovf_log_err("failed to modify QOS, port=%d, q=%d", nic->port_id, queue_conf->ev.queue_id); + edev->rx_offload_flags = nic->rx_offload_flags; + edev->tx_offload_flags = nic->tx_offload_flags; return ret; } diff --git a/drivers/event/octeontx/ssovf_evdev.h b/drivers/event/octeontx/ssovf_evdev.h index 0e622152c..1c3ae8556 100644 --- a/drivers/event/octeontx/ssovf_evdev.h +++ b/drivers/event/octeontx/ssovf_evdev.h @@ -12,6 +12,8 @@ #include #include +#include "octeontx_rxtx.h" + #define EVENTDEV_NAME_OCTEONTX_PMD event_octeontx #define SSOVF_LOG(level, fmt, args...) \ @@ -132,6 +134,7 @@ enum ssovf_type { }; struct ssovf_evdev { + OFFLOAD_FLAGS; /*Sequence should not be changed */ uint8_t max_event_queues; uint8_t max_event_ports; uint8_t is_timeout_deq; @@ -175,6 +178,14 @@ uint16_t ssows_deq_timeout(void *port, struct rte_event *ev, uint64_t timeout_ticks); uint16_t ssows_deq_timeout_burst(void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); +uint16_t ssows_deq_mseg(void *port, struct rte_event *ev, + uint64_t timeout_ticks); +uint16_t ssows_deq_burst_mseg(void *port, struct rte_event ev[], + uint16_t nb_events, uint64_t timeout_ticks); +uint16_t ssows_deq_timeout_mseg(void *port, struct rte_event *ev, + uint64_t timeout_ticks); +uint16_t ssows_deq_timeout_burst_mseg(void *port, struct rte_event ev[], + uint16_t nb_events, uint64_t timeout_ticks); typedef void (*ssows_handle_event_t)(void *arg, struct rte_event ev); void ssows_flush_events(struct ssows *ws, uint8_t queue_id, @@ -182,6 +193,8 @@ void ssows_flush_events(struct ssows *ws, uint8_t queue_id, void ssows_reset(struct ssows *ws); uint16_t sso_event_tx_adapter_enqueue(void *port, struct rte_event ev[], uint16_t nb_events); +uint16_t sso_event_tx_adapter_enqueue_mseg(void *port, + struct rte_event ev[], uint16_t nb_events); int ssovf_info(struct ssovf_info *info); void *ssovf_bar(enum ssovf_type, uint8_t id, uint8_t bar); int test_eventdev_octeontx(void); diff --git a/drivers/event/octeontx/ssovf_worker.c b/drivers/event/octeontx/ssovf_worker.c index 169939d1a..951f66890 100644 --- a/drivers/event/octeontx/ssovf_worker.c +++ b/drivers/event/octeontx/ssovf_worker.c @@ -103,7 +103,7 @@ ssows_deq(void *port, struct rte_event *ev, uint64_t timeout_ticks) ssows_swtag_wait(ws); return 1; } else { - return ssows_get_work(ws, ev); + return ssows_get_work(ws, ev, OCCTX_RX_OFFLOAD_NONE); } } @@ -118,9 +118,9 @@ ssows_deq_timeout(void *port, struct rte_event *ev, uint64_t timeout_ticks) ws->swtag_req = 0; ssows_swtag_wait(ws); } else { - ret = ssows_get_work(ws, ev); + ret = ssows_get_work(ws, ev, OCCTX_RX_OFFLOAD_NONE); for (iter = 1; iter < timeout_ticks && (ret == 0); iter++) - ret = ssows_get_work(ws, ev); + ret = ssows_get_work(ws, ev, OCCTX_RX_OFFLOAD_NONE); } return ret; } @@ -143,6 +143,61 @@ ssows_deq_timeout_burst(void *port, struct rte_event ev[], uint16_t nb_events, return ssows_deq_timeout(port, ev, timeout_ticks); } +__rte_always_inline uint16_t __hot +ssows_deq_mseg(void *port, struct rte_event *ev, uint64_t timeout_ticks) +{ + struct ssows *ws = port; + + RTE_SET_USED(timeout_ticks); + + if (ws->swtag_req) { + ws->swtag_req = 0; + ssows_swtag_wait(ws); + return 1; + } else { + return ssows_get_work(ws, ev, OCCTX_RX_OFFLOAD_NONE | + OCCTX_RX_MULTI_SEG_F); + } +} + +__rte_always_inline uint16_t __hot +ssows_deq_timeout_mseg(void *port, struct rte_event *ev, uint64_t timeout_ticks) +{ + struct ssows *ws = port; + uint64_t iter; + uint16_t ret = 1; + + if (ws->swtag_req) { + ws->swtag_req = 0; + ssows_swtag_wait(ws); + } else { + ret = ssows_get_work(ws, ev, OCCTX_RX_OFFLOAD_NONE | + OCCTX_RX_MULTI_SEG_F); + for (iter = 1; iter < timeout_ticks && (ret == 0); iter++) + ret = ssows_get_work(ws, ev, OCCTX_RX_OFFLOAD_NONE | + OCCTX_RX_MULTI_SEG_F); + } + return ret; +} + +uint16_t __hot +ssows_deq_burst_mseg(void *port, struct rte_event ev[], uint16_t nb_events, + uint64_t timeout_ticks) +{ + RTE_SET_USED(nb_events); + + return ssows_deq_mseg(port, ev, timeout_ticks); +} + +uint16_t __hot +ssows_deq_timeout_burst_mseg(void *port, struct rte_event ev[], + uint16_t nb_events, uint64_t timeout_ticks) +{ + RTE_SET_USED(nb_events); + + return ssows_deq_timeout_mseg(port, ev, timeout_ticks); +} + __rte_always_inline uint16_t __hot ssows_enq(void *port, const struct rte_event *ev) { @@ -231,7 +286,9 @@ ssows_flush_events(struct ssows *ws, uint8_t queue_id, ev.event = sched_type_queue | (get_work0 & 0xffffffff); if (get_work1 && ev.event_type == RTE_EVENT_TYPE_ETHDEV) ev.mbuf = ssovf_octeontx_wqe_to_pkt(get_work1, - (ev.event >> 20) & 0x7F); + (ev.event >> 20) & 0x7F, + OCCTX_RX_OFFLOAD_NONE | + OCCTX_RX_MULTI_SEG_F); else ev.u64 = get_work1; @@ -262,9 +319,9 @@ ssows_reset(struct ssows *ws) } } -uint16_t -sso_event_tx_adapter_enqueue(void *port, - struct rte_event ev[], uint16_t nb_events) +static __rte_always_inline uint16_t +__sso_event_tx_adapter_enqueue(void *port, struct rte_event ev[], + uint16_t nb_events, const uint16_t flag) { uint16_t port_id; uint16_t queue_id; @@ -298,5 +355,22 @@ sso_event_tx_adapter_enqueue(void *port, ethdev = &rte_eth_devices[port_id]; txq = ethdev->data->tx_queues[queue_id]; - return __octeontx_xmit_pkts(txq, &m, 1, cmd, OCCTX_TX_OFFLOAD_NONE); + return __octeontx_xmit_pkts(txq, &m, 1, cmd, flag); +} + +uint16_t +sso_event_tx_adapter_enqueue(void *port, struct rte_event ev[], + uint16_t nb_events) +{ + return __sso_event_tx_adapter_enqueue(port, ev, nb_events, + OCCTX_TX_OFFLOAD_NONE); +} + +uint16_t +sso_event_tx_adapter_enqueue_mseg(void *port, struct rte_event ev[], + uint16_t nb_events) +{ + return __sso_event_tx_adapter_enqueue(port, ev, nb_events, + OCCTX_TX_OFFLOAD_NONE | + OCCTX_TX_MULTI_SEG_F); } diff --git a/drivers/event/octeontx/ssovf_worker.h b/drivers/event/octeontx/ssovf_worker.h index c4f886d63..61332304a 100644 --- a/drivers/event/octeontx/ssovf_worker.h +++ b/drivers/event/octeontx/ssovf_worker.h @@ -23,8 +23,45 @@ enum { /* SSO Operations */ +static __rte_always_inline void +ssovf_octeontx_wqe_xtract_mseg(octtx_wqe_t *wqe, + struct rte_mbuf *mbuf) +{ + octtx_pki_buflink_t *buflink; + rte_iova_t *iova_list; + uint8_t nb_segs; + uint64_t bytes_left = wqe->s.w1.len - wqe->s.w5.size; + + nb_segs = wqe->s.w0.bufs; + + buflink = (octtx_pki_buflink_t *)((uintptr_t)wqe->s.w3.addr - + sizeof(octtx_pki_buflink_t)); + + /*TODO: work with rearm data */ + + while (--nb_segs) { + iova_list = (rte_iova_t *)(uintptr_t)(buflink->w1.s.addr); + mbuf->next = (struct rte_mbuf *)(rte_iova_t *)(iova_list - 2) + - (OCTTX_PACKET_LATER_SKIP / 128); + mbuf = mbuf->next; + + mbuf->data_off = sizeof(octtx_pki_buflink_t); + + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, 1); + if (nb_segs == 1) + mbuf->data_len = bytes_left; + else + mbuf->data_len = buflink->w0.s.size; + + bytes_left = bytes_left - buflink->w0.s.size; + buflink = (octtx_pki_buflink_t *)(rte_iova_t *)(iova_list - 2); + + } +} + static __rte_always_inline struct rte_mbuf * -ssovf_octeontx_wqe_to_pkt(uint64_t work, uint16_t port_info) +ssovf_octeontx_wqe_to_pkt(uint64_t work, uint16_t port_info, + const uint16_t flag) { struct rte_mbuf *mbuf; octtx_wqe_t *wqe = (octtx_wqe_t *)(uintptr_t)work; @@ -35,10 +72,18 @@ ssovf_octeontx_wqe_to_pkt(uint64_t work, uint16_t port_info) mbuf->packet_type = ptype_table[wqe->s.w2.lcty][wqe->s.w2.lety][wqe->s.w2.lfty]; mbuf->data_off = RTE_PTR_DIFF(wqe->s.w3.addr, mbuf->buf_addr); - mbuf->pkt_len = wqe->s.w1.len; - mbuf->data_len = mbuf->pkt_len; - mbuf->nb_segs = 1; mbuf->ol_flags = 0; + mbuf->pkt_len = wqe->s.w1.len; + + if (!!(flag & OCCTX_RX_MULTI_SEG_F)) { + mbuf->nb_segs = wqe->s.w0.bufs; + mbuf->data_len = wqe->s.w5.size; + ssovf_octeontx_wqe_xtract_mseg(wqe, mbuf); + } else { + mbuf->nb_segs = 1; + mbuf->data_len = mbuf->pkt_len; + } + mbuf->port = rte_octeontx_pchan_map[port_info >> 4][port_info & 0xF]; rte_mbuf_refcnt_set(mbuf, 1); @@ -49,14 +94,29 @@ static __rte_always_inline void ssovf_octeontx_wqe_free(uint64_t work) { octtx_wqe_t *wqe = (octtx_wqe_t *)(uintptr_t)work; - struct rte_mbuf *mbuf; + uint8_t nb_segs = wqe->s.w0.bufs; + octtx_pki_buflink_t *buflink; + struct rte_mbuf *mbuf, *head; + rte_iova_t *iova_list; mbuf = (struct rte_mbuf *)((uintptr_t)wqe - OCTTX_PACKET_WQE_SKIP); - rte_pktmbuf_free(mbuf); + buflink = (octtx_pki_buflink_t *)((uintptr_t)wqe->s.w3.addr - + sizeof(octtx_pki_buflink_t)); + head = mbuf; + while (--nb_segs) { + iova_list = (rte_iova_t *)(uintptr_t)(buflink->w1.s.addr); + mbuf = (struct rte_mbuf *)(rte_iova_t *)(iova_list - 2) + - (OCTTX_PACKET_LATER_SKIP / 128); + + mbuf->next = NULL; + rte_pktmbuf_free(mbuf); + buflink = (octtx_pki_buflink_t *)(rte_iova_t *)(iova_list - 2); + } + rte_pktmbuf_free(head); } static __rte_always_inline uint16_t -ssows_get_work(struct ssows *ws, struct rte_event *ev) +ssows_get_work(struct ssows *ws, struct rte_event *ev, const uint16_t flag) { uint64_t get_work0, get_work1; uint64_t sched_type_queue; @@ -71,7 +131,7 @@ ssows_get_work(struct ssows *ws, struct rte_event *ev) if (get_work1 && ev->event_type == RTE_EVENT_TYPE_ETHDEV) { ev->mbuf = ssovf_octeontx_wqe_to_pkt(get_work1, - (ev->event >> 20) & 0x7F); + (ev->event >> 20) & 0x7F, flag); } else if (unlikely((get_work0 & 0xFFFFFFFF) == 0xFFFFFFFF)) { ssovf_octeontx_wqe_free(get_work1); return 0; From patchwork Mon Mar 16 15:29:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harman Kalra X-Patchwork-Id: 66719 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0B467A0559; Mon, 16 Mar 2020 16:30:26 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 234AB1C08C; Mon, 16 Mar 2020 16:30:04 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 1DCCB1C06C for ; Mon, 16 Mar 2020 16:30:01 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 02GFKT7A029483 for ; Mon, 16 Mar 2020 08:30:01 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=pfpt0818; bh=yGQesIQ0sK0yv4v4DBHhhcutYzM/qmVMGb+7N7KIvYQ=; b=sFGDE5kUfwNMjPCHYwl8AsdmVS9785hc5yJ1qU7kkSXpl9OHsLIbfz3rjnfd7FLU80ow arV++fF+SSQwHE5HH5xoH6cBXHDNNyQtvfau9jyMMMpUrdxRc9OYKZzWJ65Uvdkyu8jW PCvjVy86djMZCiotDaHbbOx+wcR+4YSPwzW7cPSZsaCTLA6Qpr+NlsmlZwaUhGtzUAC6 X2TAkQ0IyeuMy3GiDMWmAelOQXsHYPh4WUZerOk7HXi1AJb2zR8z4S/pFXFaM4ryzcLX SAyzsXepQdFyosfpoJJWP21egGKq13gejeXmuLml/SRrs9fuF5gemTEhmboSSgcXi9+E sQ== Received: from sc-exch02.marvell.com ([199.233.58.182]) by mx0b-0016f401.pphosted.com with ESMTP id 2yrxsmf9e6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 16 Mar 2020 08:30:01 -0700 Received: from SC-EXCH04.marvell.com (10.93.176.84) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 16 Mar 2020 08:29:59 -0700 Received: from NAM04-BN3-obe.outbound.protection.outlook.com (104.47.46.59) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Mon, 16 Mar 2020 08:29:58 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gk5qDQBqrs4aPQrkTwci3Y/i6whgwbbXL2kLCegUzxVHpLq6ZDbIMk8cGHo8f1PxjOd7CLiwmjmoQwtOBWSPilb65ff+RReTm9UmZQ1wTic8otjslWOHw6nkdCp7WW+Esk3M/faTiA2PV2IUAiAqfDBBMjruLoFBHHOCO2S3kWhwUTXTHR6EP0J/uCE3lpmzLQiaffBVg5FQRjAhE/XkKWb8hWH8tqdU7b0pLTWiAIKNYvFShujXBBBB3oCKqQIWoHi452ismhxWkrZkzGkeaEjWZmSGxz5kzFbnoClatXWqq10BGLQ96QAuAUZa4pc0j3uhQDwp+yVJ0E4+LMgmtw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yGQesIQ0sK0yv4v4DBHhhcutYzM/qmVMGb+7N7KIvYQ=; b=JzawPGwn6PL/O4HSq3HJGoXmGwfU8amszfp2MRLtnodK1xTHBxoZajN+o8K40OlyRbzIZ9s69viAgDnF0W61eBw2O6+do/ChBwiszfcBsGpmTuqr8Nv5jy3mI6e8nFoRpFkP+Z+jKEANJRGl3uIZfZKaPj2JkOUHZ1GMyrsDarXXfxDGoLbhIsmsN7PN5/jJ28xsL+KJzINvcAP4XrokPj7moLaTfctsDKc4hEkPG5PPX11WjIYR8KyBX9x/NYoixiP8A9SQqdkdyP5YMxaQY2NhqKylwqHT+aIU6r0IxHiE5/gK7j/et9kHDZaZKaS4GQOyEmO9Kp3kAekDZzhV9A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yGQesIQ0sK0yv4v4DBHhhcutYzM/qmVMGb+7N7KIvYQ=; b=MIzatwAEak/Y0o9YUSBLzgCiGv7EwqJjK0lfqMIeUOHQCvvv8fte08jHEvPTkChuJ4aHq9QZcv3EttBGwO7218o0w2Rw6ngl2YZH/5IYr9N1tKDKsu1df/22krRYDskHlhATWhB5rIiSxPNZ5WHoH1C1YYuKp5FAdnx5n9tnV9M= Received: from MN2PR18MB2848.namprd18.prod.outlook.com (2603:10b6:208:38::19) by MN2PR18MB3592.namprd18.prod.outlook.com (2603:10b6:208:26c::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2814.19; Mon, 16 Mar 2020 15:29:57 +0000 Received: from MN2PR18MB2848.namprd18.prod.outlook.com ([fe80::f829:3e55:94f6:4efb]) by MN2PR18MB2848.namprd18.prod.outlook.com ([fe80::f829:3e55:94f6:4efb%5]) with mapi id 15.20.2814.021; Mon, 16 Mar 2020 15:29:57 +0000 From: Harman Kalra To: CC: , , Harman Kalra Date: Mon, 16 Mar 2020 20:59:11 +0530 Message-ID: <1584372553-28710-3-git-send-email-hkalra@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1584372553-28710-1-git-send-email-hkalra@marvell.com> References: <1584372553-28710-1-git-send-email-hkalra@marvell.com> X-ClientProxiedBy: BM1PR0101CA0036.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:1a::22) To MN2PR18MB2848.namprd18.prod.outlook.com (2603:10b6:208:38::19) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from hkarlara-OptiPlex-3046.marvell.com (115.113.156.2) by BM1PR0101CA0036.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:1a::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.2814.13 via Frontend Transport; Mon, 16 Mar 2020 15:29:56 +0000 X-Mailer: git-send-email 2.7.4 X-Originating-IP: [115.113.156.2] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9ea64c91-b911-4a55-80d6-08d7c9bee24e X-MS-TrafficTypeDiagnostic: MN2PR18MB3592: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4502; X-Forefront-PRVS: 03449D5DD1 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4636009)(366004)(39850400004)(136003)(376002)(346002)(396003)(199004)(66476007)(30864003)(956004)(478600001)(66556008)(186003)(86362001)(26005)(16526019)(36756003)(66946007)(6486002)(4326008)(6636002)(316002)(55236004)(7696005)(37006003)(52116002)(107886003)(2616005)(8676002)(8936002)(2906002)(81166006)(5660300002)(81156014)(34206002)(6666004); DIR:OUT; SFP:1101; SCL:1; SRVR:MN2PR18MB3592; H:MN2PR18MB2848.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; Received-SPF: None (protection.outlook.com: marvell.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: kLYqCOLc6vGYaQaqZc+qAynPoXxz4TNO8KfIAFGkkisEF+sQYC/ygMPfw2F36NszjPE1GpkUE675LS6URmR+m6EMqY8F0HfbC8LVnUKirnJYhoRE0zDGlpvsnFtqYbnqfnApohMKj6Q2R76SX+DwaEhMD+pr4xWDLVYJD0rHgUC8qKfyUD9sfn/TPFARPBhB4GR+upbMO4rMIxhA7C77LS98IN96svP6xcbG281ESCcuxmTI2F81qVGwmVYjh9kIDAFL4zZih/54MhmEJwQd+Y3lHNbfnDlet5AAgyshFglKmE++lwD7ON3YhA3rB8k7jpLhO4vQTxb2wYXSkMOb8cMpEFC/sfQo0iPhfA90LPHL4G8BhoQ44IP+ZNJFimrIB3fjisycKwsWwoUFDcKJKcaYFcHHmDtYmFqJNz6AJqXEHgmTwmKukAsaRcxdJdat X-MS-Exchange-AntiSpam-MessageData: LJ2x8MfvMC+ytWEtV69uWQDcoud4QUqb9kthj+hAK2bRzEAzgkCCRDcCLhMOhtJKqSA4PbrMnbpRblsXHWXe6AMLcmb0ymASI10GdzmFfaqY6GIXycBAFg62L65aehd5sJJdBzF5jJeUs/pyWG87XQ== X-MS-Exchange-CrossTenant-Network-Message-Id: 9ea64c91-b911-4a55-80d6-08d7c9bee24e X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Mar 2020 15:29:57.5318 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: m2QzFqOP6IpNV3f6P8RRxaeyImucICO0cUNxKH9zrTH7VhUI9V//bBiNMZ5CpvISUd2TMsVqDVKGp7zTp01gGg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR18MB3592 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.645 definitions=2020-03-16_06:2020-03-12, 2020-03-16 signatures=0 Subject: [dpdk-dev] [PATCH 2/4] event/octeontx: add framework for Rx/Tx offloads X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Adding macro based framework to hook dequeue/enqueue function pointers to the appropriate function based on rx/tx offloads. Signed-off-by: Harman Kalra --- drivers/event/octeontx/ssovf_evdev.c | 36 ---- drivers/event/octeontx/ssovf_evdev.h | 24 +-- drivers/event/octeontx/ssovf_worker.c | 259 ++++++++++++++------------ drivers/net/octeontx/octeontx_rxtx.h | 7 + 4 files changed, 150 insertions(+), 176 deletions(-) diff --git a/drivers/event/octeontx/ssovf_evdev.c b/drivers/event/octeontx/ssovf_evdev.c index 1024b7284..5d074bcbc 100644 --- a/drivers/event/octeontx/ssovf_evdev.c +++ b/drivers/event/octeontx/ssovf_evdev.c @@ -137,42 +137,6 @@ ssovf_mbox_timeout_ticks(uint64_t ns, uint64_t *tmo_ticks) return 0; } -static void -ssovf_fastpath_fns_set(struct rte_eventdev *dev) -{ - struct ssovf_evdev *edev = ssovf_pmd_priv(dev); - - dev->enqueue = ssows_enq; - dev->enqueue_burst = ssows_enq_burst; - dev->enqueue_new_burst = ssows_enq_new_burst; - dev->enqueue_forward_burst = ssows_enq_fwd_burst; - - if (!!(edev->rx_offload_flags & OCCTX_RX_MULTI_SEG_F)) { - dev->dequeue = ssows_deq_mseg; - dev->dequeue_burst = ssows_deq_burst_mseg; - - if (edev->is_timeout_deq) { - dev->dequeue = ssows_deq_timeout_mseg; - dev->dequeue_burst = ssows_deq_timeout_burst_mseg; - } - } else { - dev->dequeue = ssows_deq; - dev->dequeue_burst = ssows_deq_burst; - - if (edev->is_timeout_deq) { - dev->dequeue = ssows_deq_timeout; - dev->dequeue_burst = ssows_deq_timeout_burst; - } - } - - if (!!(edev->tx_offload_flags & OCCTX_TX_MULTI_SEG_F)) - dev->txa_enqueue = sso_event_tx_adapter_enqueue_mseg; - else - dev->txa_enqueue = sso_event_tx_adapter_enqueue; - - dev->txa_enqueue_same_dest = dev->txa_enqueue; -} - static void ssovf_info_get(struct rte_eventdev *dev, struct rte_event_dev_info *dev_info) { diff --git a/drivers/event/octeontx/ssovf_evdev.h b/drivers/event/octeontx/ssovf_evdev.h index 1c3ae8556..1f5066c9a 100644 --- a/drivers/event/octeontx/ssovf_evdev.h +++ b/drivers/event/octeontx/ssovf_evdev.h @@ -14,6 +14,9 @@ #include "octeontx_rxtx.h" +#define SSO_RX_ADPTR_ENQ_FASTPATH_FUNC OCCTX_RX_FASTPATH_MODES +#define SSO_TX_ADPTR_ENQ_FASTPATH_FUNC OCCTX_TX_FASTPATH_MODES + #define EVENTDEV_NAME_OCTEONTX_PMD event_octeontx #define SSOVF_LOG(level, fmt, args...) \ @@ -171,32 +174,13 @@ uint16_t ssows_enq_new_burst(void *port, const struct rte_event ev[], uint16_t nb_events); uint16_t ssows_enq_fwd_burst(void *port, const struct rte_event ev[], uint16_t nb_events); -uint16_t ssows_deq(void *port, struct rte_event *ev, uint64_t timeout_ticks); -uint16_t ssows_deq_burst(void *port, struct rte_event ev[], - uint16_t nb_events, uint64_t timeout_ticks); -uint16_t ssows_deq_timeout(void *port, struct rte_event *ev, - uint64_t timeout_ticks); -uint16_t ssows_deq_timeout_burst(void *port, struct rte_event ev[], - uint16_t nb_events, uint64_t timeout_ticks); -uint16_t ssows_deq_mseg(void *port, struct rte_event *ev, - uint64_t timeout_ticks); -uint16_t ssows_deq_burst_mseg(void *port, struct rte_event ev[], - uint16_t nb_events, uint64_t timeout_ticks); -uint16_t ssows_deq_timeout_mseg(void *port, struct rte_event *ev, - uint64_t timeout_ticks); -uint16_t ssows_deq_timeout_burst_mseg(void *port, struct rte_event ev[], - uint16_t nb_events, uint64_t timeout_ticks); - typedef void (*ssows_handle_event_t)(void *arg, struct rte_event ev); void ssows_flush_events(struct ssows *ws, uint8_t queue_id, ssows_handle_event_t fn, void *arg); void ssows_reset(struct ssows *ws); -uint16_t sso_event_tx_adapter_enqueue(void *port, - struct rte_event ev[], uint16_t nb_events); -uint16_t sso_event_tx_adapter_enqueue_mseg(void *port, - struct rte_event ev[], uint16_t nb_events); int ssovf_info(struct ssovf_info *info); void *ssovf_bar(enum ssovf_type, uint8_t id, uint8_t bar); int test_eventdev_octeontx(void); +void ssovf_fastpath_fns_set(struct rte_eventdev *dev); #endif /* __SSOVF_EVDEV_H__ */ diff --git a/drivers/event/octeontx/ssovf_worker.c b/drivers/event/octeontx/ssovf_worker.c index 951f66890..5d8e213ce 100644 --- a/drivers/event/octeontx/ssovf_worker.c +++ b/drivers/event/octeontx/ssovf_worker.c @@ -91,112 +91,62 @@ ssows_release_event(struct ssows *ws) ssows_swtag_untag(ws); } -__rte_always_inline uint16_t __hot -ssows_deq(void *port, struct rte_event *ev, uint64_t timeout_ticks) -{ - struct ssows *ws = port; - - RTE_SET_USED(timeout_ticks); - - if (ws->swtag_req) { - ws->swtag_req = 0; - ssows_swtag_wait(ws); - return 1; - } else { - return ssows_get_work(ws, ev, OCCTX_RX_OFFLOAD_NONE); - } -} - -__rte_always_inline uint16_t __hot -ssows_deq_timeout(void *port, struct rte_event *ev, uint64_t timeout_ticks) -{ - struct ssows *ws = port; - uint64_t iter; - uint16_t ret = 1; - - if (ws->swtag_req) { - ws->swtag_req = 0; - ssows_swtag_wait(ws); - } else { - ret = ssows_get_work(ws, ev, OCCTX_RX_OFFLOAD_NONE); - for (iter = 1; iter < timeout_ticks && (ret == 0); iter++) - ret = ssows_get_work(ws, ev, OCCTX_RX_OFFLOAD_NONE); - } - return ret; +#define R(name, f0, flags) \ +static uint16_t __rte_noinline __hot \ +ssows_deq_ ##name(void *port, struct rte_event *ev, uint64_t timeout_ticks) \ +{ \ + struct ssows *ws = port; \ + \ + RTE_SET_USED(timeout_ticks); \ + \ + if (ws->swtag_req) { \ + ws->swtag_req = 0; \ + ssows_swtag_wait(ws); \ + return 1; \ + } else { \ + return ssows_get_work(ws, ev, flags); \ + } \ +} \ + \ +static uint16_t __hot \ +ssows_deq_burst_ ##name(void *port, struct rte_event ev[], \ + uint16_t nb_events, uint64_t timeout_ticks) \ +{ \ + RTE_SET_USED(nb_events); \ + \ + return ssows_deq_ ##name(port, ev, timeout_ticks); \ +} \ + \ +static uint16_t __hot \ +ssows_deq_timeout_ ##name(void *port, struct rte_event *ev, \ + uint64_t timeout_ticks) \ +{ \ + struct ssows *ws = port; \ + uint64_t iter; \ + uint16_t ret = 1; \ + \ + if (ws->swtag_req) { \ + ws->swtag_req = 0; \ + ssows_swtag_wait(ws); \ + } else { \ + ret = ssows_get_work(ws, ev, flags); \ + for (iter = 1; iter < timeout_ticks && (ret == 0); iter++) \ + ret = ssows_get_work(ws, ev, flags); \ + } \ + return ret; \ +} \ + \ +static uint16_t __hot \ +ssows_deq_timeout_burst_ ##name(void *port, struct rte_event ev[], \ + uint16_t nb_events, uint64_t timeout_ticks) \ +{ \ + RTE_SET_USED(nb_events); \ + \ + return ssows_deq_timeout_ ##name(port, ev, timeout_ticks); \ } -uint16_t __hot -ssows_deq_burst(void *port, struct rte_event ev[], uint16_t nb_events, - uint64_t timeout_ticks) -{ - RTE_SET_USED(nb_events); - - return ssows_deq(port, ev, timeout_ticks); -} - -uint16_t __hot -ssows_deq_timeout_burst(void *port, struct rte_event ev[], uint16_t nb_events, - uint64_t timeout_ticks) -{ - RTE_SET_USED(nb_events); - - return ssows_deq_timeout(port, ev, timeout_ticks); -} - -__rte_always_inline uint16_t __hot -ssows_deq_mseg(void *port, struct rte_event *ev, uint64_t timeout_ticks) -{ - struct ssows *ws = port; - - RTE_SET_USED(timeout_ticks); - - if (ws->swtag_req) { - ws->swtag_req = 0; - ssows_swtag_wait(ws); - return 1; - } else { - return ssows_get_work(ws, ev, OCCTX_RX_OFFLOAD_NONE | - OCCTX_RX_MULTI_SEG_F); - } -} - -__rte_always_inline uint16_t __hot -ssows_deq_timeout_mseg(void *port, struct rte_event *ev, uint64_t timeout_ticks) -{ - struct ssows *ws = port; - uint64_t iter; - uint16_t ret = 1; - - if (ws->swtag_req) { - ws->swtag_req = 0; - ssows_swtag_wait(ws); - } else { - ret = ssows_get_work(ws, ev, OCCTX_RX_OFFLOAD_NONE | - OCCTX_RX_MULTI_SEG_F); - for (iter = 1; iter < timeout_ticks && (ret == 0); iter++) - ret = ssows_get_work(ws, ev, OCCTX_RX_OFFLOAD_NONE | - OCCTX_RX_MULTI_SEG_F); - } - return ret; -} - -uint16_t __hot -ssows_deq_burst_mseg(void *port, struct rte_event ev[], uint16_t nb_events, - uint64_t timeout_ticks) -{ - RTE_SET_USED(nb_events); - - return ssows_deq_mseg(port, ev, timeout_ticks); -} - -uint16_t __hot -ssows_deq_timeout_burst_mseg(void *port, struct rte_event ev[], - uint16_t nb_events, uint64_t timeout_ticks) -{ - RTE_SET_USED(nb_events); - - return ssows_deq_timeout_mseg(port, ev, timeout_ticks); -} +SSO_RX_ADPTR_ENQ_FASTPATH_FUNC +#undef R __rte_always_inline uint16_t __hot ssows_enq(void *port, const struct rte_event *ev) @@ -321,7 +271,8 @@ ssows_reset(struct ssows *ws) static __rte_always_inline uint16_t __sso_event_tx_adapter_enqueue(void *port, struct rte_event ev[], - uint16_t nb_events, const uint16_t flag) + uint16_t nb_events, uint64_t *cmd, + const uint16_t flag) { uint16_t port_id; uint16_t queue_id; @@ -329,9 +280,7 @@ __sso_event_tx_adapter_enqueue(void *port, struct rte_event ev[], struct rte_eth_dev *ethdev; struct ssows *ws = port; struct octeontx_txq *txq; - uint64_t cmd[4]; - RTE_SET_USED(nb_events); switch (ev->sched_type) { case SSO_SYNC_ORDERED: ssows_swtag_norm(ws, ev->event, SSO_SYNC_ATOMIC); @@ -355,22 +304,92 @@ __sso_event_tx_adapter_enqueue(void *port, struct rte_event ev[], ethdev = &rte_eth_devices[port_id]; txq = ethdev->data->tx_queues[queue_id]; - return __octeontx_xmit_pkts(txq, &m, 1, cmd, flag); + return __octeontx_xmit_pkts(txq, &m, nb_events, cmd, flag); } -uint16_t -sso_event_tx_adapter_enqueue(void *port, struct rte_event ev[], - uint16_t nb_events) -{ - return __sso_event_tx_adapter_enqueue(port, ev, nb_events, - OCCTX_TX_OFFLOAD_NONE); +#define T(name, f3, f2, f1, f0, sz, flags) \ +static uint16_t __rte_noinline __hot \ +sso_event_tx_adapter_enqueue_ ## name(void *port, struct rte_event ev[], \ + uint16_t nb_events) \ +{ \ + uint64_t cmd[sz]; \ + return __sso_event_tx_adapter_enqueue(port, ev, nb_events, cmd, \ + flags); \ } -uint16_t -sso_event_tx_adapter_enqueue_mseg(void *port, struct rte_event ev[], - uint16_t nb_events) +SSO_TX_ADPTR_ENQ_FASTPATH_FUNC +#undef T + +void +ssovf_fastpath_fns_set(struct rte_eventdev *dev) { - return __sso_event_tx_adapter_enqueue(port, ev, nb_events, - OCCTX_TX_OFFLOAD_NONE | - OCCTX_TX_MULTI_SEG_F); + struct ssovf_evdev *edev = ssovf_pmd_priv(dev); + + dev->enqueue = ssows_enq; + dev->enqueue_burst = ssows_enq_burst; + dev->enqueue_new_burst = ssows_enq_new_burst; + dev->enqueue_forward_burst = ssows_enq_fwd_burst; + + const event_tx_adapter_enqueue ssow_txa_enqueue[2][2][2][2] = { +#define T(name, f3, f2, f1, f0, sz, flags) \ + [f3][f2][f1][f0] = sso_event_tx_adapter_enqueue_ ##name, + +SSO_TX_ADPTR_ENQ_FASTPATH_FUNC +#undef T + }; + + dev->txa_enqueue = ssow_txa_enqueue + [!!(edev->tx_offload_flags & OCCTX_TX_OFFLOAD_MBUF_NOFF_F)] + [0] + [0] + [!!(edev->tx_offload_flags & OCCTX_TX_MULTI_SEG_F)]; + + dev->txa_enqueue_same_dest = dev->txa_enqueue; + + /* Assigning dequeue func pointers */ + const event_dequeue_t ssow_deq[2] = { +#define R(name, f0, flags) \ + [f0] = ssows_deq_ ##name, + +SSO_RX_ADPTR_ENQ_FASTPATH_FUNC +#undef R + }; + + dev->dequeue = ssow_deq + [!!(edev->rx_offload_flags & OCCTX_RX_MULTI_SEG_F)]; + + const event_dequeue_burst_t ssow_deq_burst[2] = { +#define R(name, f0, flags) \ + [f0] = ssows_deq_burst_ ##name, + +SSO_RX_ADPTR_ENQ_FASTPATH_FUNC +#undef R + }; + + dev->dequeue_burst = ssow_deq_burst + [!!(edev->rx_offload_flags & OCCTX_RX_MULTI_SEG_F)]; + + if (edev->is_timeout_deq) { + const event_dequeue_t ssow_deq_timeout[2] = { +#define R(name, f0, flags) \ + [f0] = ssows_deq_timeout_ ##name, + +SSO_RX_ADPTR_ENQ_FASTPATH_FUNC +#undef R + }; + + dev->dequeue = ssow_deq_timeout + [!!(edev->rx_offload_flags & OCCTX_RX_MULTI_SEG_F)]; + + const event_dequeue_burst_t ssow_deq_timeout_burst[2] = { +#define R(name, f0, flags) \ + [f0] = ssows_deq_timeout_burst_ ##name, + +SSO_RX_ADPTR_ENQ_FASTPATH_FUNC +#undef R + }; + + dev->dequeue_burst = ssow_deq_timeout_burst + [!!(edev->rx_offload_flags & OCCTX_RX_MULTI_SEG_F)]; + } } diff --git a/drivers/net/octeontx/octeontx_rxtx.h b/drivers/net/octeontx/octeontx_rxtx.h index acc1f5cb8..e7656f333 100644 --- a/drivers/net/octeontx/octeontx_rxtx.h +++ b/drivers/net/octeontx/octeontx_rxtx.h @@ -488,4 +488,11 @@ T(noff_ol3ol4csum_l3l4csum_mseg, 1, 1, 1, 1, 14, \ NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F | \ MULT_F) +/* RX offload macros */ +#define MULT_RX_F OCCTX_RX_MULTI_SEG_F +/* [MULTI_SEG] */ +#define OCCTX_RX_FASTPATH_MODES \ +R(no_offload, 0, OCCTX_RX_OFFLOAD_NONE) \ +R(mseg, 1, MULT_RX_F) \ + #endif /* __OCTEONTX_RXTX_H__ */ From patchwork Mon Mar 16 15:29:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harman Kalra X-Patchwork-Id: 66720 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4735CA0559; Mon, 16 Mar 2020 16:30:37 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 81BD51C07B; Mon, 16 Mar 2020 16:30:06 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 63B881C0AD for ; Mon, 16 Mar 2020 16:30:05 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 02GFKT7C029483 for ; Mon, 16 Mar 2020 08:30:04 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=pfpt0818; bh=Li4bn9v2MrbFt75MQxX9pvhoZ2qDe/DeGrEt/eLkyE4=; b=vhbo2Z/gSdFafLbN3KerPP17sTb2d7V9/JvIEMtcOn7SVqjECm4Uu8kDnINYBd4ptaFs V3URk0blbD6WIIRGuW0FB9xn1hxJAQ667ZfVVPGeV1e279+MP1e0mJ17M7kGsgjsiPjV JdK6sB7c1m3N8gdd304iuHIK6BB0lvAadLpWYJSdWI8VIfxDv3u5sOA7n64PJGjmf6+H xwMymoEBoojsAbDP08/1kr1nfyzvGIHqNKJ9im+NljQ76S+qQe9I6AC9pYnrt7KvFBH4 BY/taE0qlJdBOK1jdKtaszWuZ4sdwdsxQ3YXxgeQldaQOpRULJtr+CJCzZqmMeueSJy8 /Q== Received: from sc-exch01.marvell.com ([199.233.58.181]) by mx0b-0016f401.pphosted.com with ESMTP id 2yrxsmf9em-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 16 Mar 2020 08:30:04 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 16 Mar 2020 08:30:03 -0700 Received: from SC-EXCH03.marvell.com (10.93.176.83) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 16 Mar 2020 08:30:02 -0700 Received: from NAM04-BN3-obe.outbound.protection.outlook.com (104.47.46.50) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Mon, 16 Mar 2020 08:30:01 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=b4xe4Gw5wx3bxXGdPXz6MBAkjidKrtVwHY4p1jYj3k9VRoLabESAIO4cqpJz0k34tdTmU+BeUp9CVN011VkfXSEvIIgnGDe7i3rkaUKguyCJndSJbgkK3gMxzL7AR7fSYJDo0MUhGWncfdh65IGmhpYxRYy7KJUr1SS3LPiPq4v0roTJ3L+s2Uk5fBwa/ZXA9ILUWMXCMn3XLxK5FPX4SQcPtoxZuo2jsnUY9R/i5ErF8iOHmyvE62h88ny8MwDQVaa6v0eeZ1Qms4R+jU2LbeVZtf7E+joj3MTSb/rnDYremmDpHeQMBLMbemQo6wsT0FnaVLaRkB0FlaAVxPkxbQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Li4bn9v2MrbFt75MQxX9pvhoZ2qDe/DeGrEt/eLkyE4=; b=TL1EE45bWwoMzZdJ7nfKUZMcUjty8WXa0o2GJk0rMm9HaBWAa5o+DkquDygnO2auItKqrdqnHTeA6dxgFV09PSZLAdEaiAQamtz9Jv5fNnCw4jHN9J5FP03yDX8lp7B9OfetQ4p35hEg6eTqLh/CLJCbKp7WB513MSQmwkHWaKxryzcHrRc5SPAgYztcEehoH/l8J8mG1g6FzdJJUHe/V49Kt0GvoWxp6ugooiIw/JRDDJzyG8LAf92crHcHgSLy836jxOYvoVYM3d06NRRqeKLgJxOHc2RY+ehXBGuM8h5YFggSTTIXmUL+AIJw56GiUO6+cn6DtGtg/5fXkxNFfg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Li4bn9v2MrbFt75MQxX9pvhoZ2qDe/DeGrEt/eLkyE4=; b=fXO553irKf+Qi6WJtiKChs/WbZfhpeybzcjgn4z2BhWZ2seUutgDp7yXAb1jGXnHV8c+r6IBtiMJ8N+H0X7IKxbOKwnE0l6JHtFuyrcc2KGJ4H3tkLYPmG8Gr7s+Ww4OPjqy8I3fC2g5ITbshVlwHlYaPatZ09O5YOA5/k9XbY0= Received: from MN2PR18MB2848.namprd18.prod.outlook.com (2603:10b6:208:38::19) by MN2PR18MB3592.namprd18.prod.outlook.com (2603:10b6:208:26c::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2814.19; Mon, 16 Mar 2020 15:30:00 +0000 Received: from MN2PR18MB2848.namprd18.prod.outlook.com ([fe80::f829:3e55:94f6:4efb]) by MN2PR18MB2848.namprd18.prod.outlook.com ([fe80::f829:3e55:94f6:4efb%5]) with mapi id 15.20.2814.021; Mon, 16 Mar 2020 15:30:00 +0000 From: Harman Kalra To: CC: , Date: Mon, 16 Mar 2020 20:59:12 +0530 Message-ID: <1584372553-28710-4-git-send-email-hkalra@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1584372553-28710-1-git-send-email-hkalra@marvell.com> References: <1584372553-28710-1-git-send-email-hkalra@marvell.com> X-ClientProxiedBy: BM1PR0101CA0036.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:1a::22) To MN2PR18MB2848.namprd18.prod.outlook.com (2603:10b6:208:38::19) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from hkarlara-OptiPlex-3046.marvell.com (115.113.156.2) by BM1PR0101CA0036.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:1a::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.2814.13 via Frontend Transport; Mon, 16 Mar 2020 15:29:58 +0000 X-Mailer: git-send-email 2.7.4 X-Originating-IP: [115.113.156.2] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a13d6021-17b1-4b1a-deac-08d7c9bee3d8 X-MS-TrafficTypeDiagnostic: MN2PR18MB3592: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:220; X-Forefront-PRVS: 03449D5DD1 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4636009)(366004)(39850400004)(136003)(376002)(346002)(396003)(199004)(66476007)(956004)(478600001)(66556008)(186003)(86362001)(26005)(16526019)(36756003)(66946007)(6486002)(4326008)(6636002)(316002)(55236004)(7696005)(37006003)(52116002)(107886003)(2616005)(8676002)(8936002)(2906002)(81166006)(5660300002)(81156014)(34206002)(6666004); DIR:OUT; SFP:1101; SCL:1; SRVR:MN2PR18MB3592; H:MN2PR18MB2848.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; Received-SPF: None (protection.outlook.com: marvell.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: HLUAtquGREfkJ0xYfYKe686V7XAKducUBks6VGjUn6QUbVdgdaqfGpB1a7ABC7A+YREgSewcD4sbnOEyvZmGCGJOeUICgwwgLbsN5RB+ZVfmoql7UivP11N4c8dDNlFvXRvOHjGibWFTLghV10LuoVFI9BQwJcoKmuPvgJW4P98IsQQOt86BeHWunwz3Gph3RgU6CCnIOBkMsqGg9oHoG4viGkKDSq6zZZnR6WdkEmEKxM5UoDXfoM9siNv2MywK44JPa9FZT8wWwlxEjEUPpVQL3cYowsIurDKWcLho7zYD2hiKkldODRXktKzFF/oY4JyJJ7wHEcrYkXWL1DSQi+3ERieE4C1RWP0E1V/vO/qM8xU+cexgLVDsTrRTyVvAb/e3Ztyac8jpLgzEo0WRgMzeaSDWZhN7NEE4hbO2epzaBrEhMbWLyOfHKYLrR055 X-MS-Exchange-AntiSpam-MessageData: 24OSTmXDJfCrgpmi2hBCjhtZYOLSrmtmNJz25w1VtkFEGXqLo1+9wjofOrDgOgbW4gD587fi12+xr/Ef+BWkfGNz763QPfaqdKOzwZ3M5rdxvPfISGg38XRQTRHrdfsTP5Hb+jFTHRUt59d9eFokIw== X-MS-Exchange-CrossTenant-Network-Message-Id: a13d6021-17b1-4b1a-deac-08d7c9bee3d8 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Mar 2020 15:30:00.5681 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: s+baOmCURwwxAgiox+sjU1tXpyECDyNvnJ7QnHbDT7QACMUb8f+GGroJx9Pnxk0c/UWZ6ykjM/agKNJ7TiNetA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR18MB3592 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.645 definitions=2020-03-16_06:2020-03-12, 2020-03-16 signatures=0 Subject: [dpdk-dev] [PATCH 3/4] event/octeontx: add VLAN filter offload support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Vamsi Attunuru Adding rx burst function pointer hooks for vlan filter offload in event PMD. Signed-off-by: Vamsi Attunuru Acked-by: Harman Kalra --- drivers/event/octeontx/ssovf_worker.c | 38 +++++++++++++++------------ drivers/event/octeontx/ssovf_worker.h | 9 +++++++ drivers/net/octeontx/octeontx_rxtx.h | 11 +++++--- 3 files changed, 37 insertions(+), 21 deletions(-) diff --git a/drivers/event/octeontx/ssovf_worker.c b/drivers/event/octeontx/ssovf_worker.c index 5d8e213ce..093f6f000 100644 --- a/drivers/event/octeontx/ssovf_worker.c +++ b/drivers/event/octeontx/ssovf_worker.c @@ -91,7 +91,7 @@ ssows_release_event(struct ssows *ws) ssows_swtag_untag(ws); } -#define R(name, f0, flags) \ +#define R(name, f1, f0, flags) \ static uint16_t __rte_noinline __hot \ ssows_deq_ ##name(void *port, struct rte_event *ev, uint64_t timeout_ticks) \ { \ @@ -347,49 +347,53 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC dev->txa_enqueue_same_dest = dev->txa_enqueue; /* Assigning dequeue func pointers */ - const event_dequeue_t ssow_deq[2] = { -#define R(name, f0, flags) \ - [f0] = ssows_deq_ ##name, + const event_dequeue_t ssow_deq[2][2] = { +#define R(name, f1, f0, flags) \ + [f1][f0] = ssows_deq_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; dev->dequeue = ssow_deq + [!!(edev->rx_offload_flags & OCCTX_RX_VLAN_FLTR_F)] [!!(edev->rx_offload_flags & OCCTX_RX_MULTI_SEG_F)]; - const event_dequeue_burst_t ssow_deq_burst[2] = { -#define R(name, f0, flags) \ - [f0] = ssows_deq_burst_ ##name, + const event_dequeue_burst_t ssow_deq_burst[2][2] = { +#define R(name, f1, f0, flags) \ + [f1][f0] = ssows_deq_burst_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; dev->dequeue_burst = ssow_deq_burst + [!!(edev->rx_offload_flags & OCCTX_RX_VLAN_FLTR_F)] [!!(edev->rx_offload_flags & OCCTX_RX_MULTI_SEG_F)]; if (edev->is_timeout_deq) { - const event_dequeue_t ssow_deq_timeout[2] = { -#define R(name, f0, flags) \ - [f0] = ssows_deq_timeout_ ##name, + const event_dequeue_t ssow_deq_timeout[2][2] = { +#define R(name, f1, f0, flags) \ + [f1][f0] = ssows_deq_timeout_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; - dev->dequeue = ssow_deq_timeout - [!!(edev->rx_offload_flags & OCCTX_RX_MULTI_SEG_F)]; + dev->dequeue = ssow_deq_timeout + [!!(edev->rx_offload_flags & OCCTX_RX_VLAN_FLTR_F)] + [!!(edev->rx_offload_flags & OCCTX_RX_MULTI_SEG_F)]; - const event_dequeue_burst_t ssow_deq_timeout_burst[2] = { -#define R(name, f0, flags) \ - [f0] = ssows_deq_timeout_burst_ ##name, + const event_dequeue_burst_t ssow_deq_timeout_burst[2][2] = { +#define R(name, f1, f0, flags) \ + [f1][f0] = ssows_deq_timeout_burst_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; - dev->dequeue_burst = ssow_deq_timeout_burst - [!!(edev->rx_offload_flags & OCCTX_RX_MULTI_SEG_F)]; + dev->dequeue_burst = ssow_deq_timeout_burst + [!!(edev->rx_offload_flags & OCCTX_RX_VLAN_FLTR_F)] + [!!(edev->rx_offload_flags & OCCTX_RX_MULTI_SEG_F)]; } } diff --git a/drivers/event/octeontx/ssovf_worker.h b/drivers/event/octeontx/ssovf_worker.h index 61332304a..e23bd057e 100644 --- a/drivers/event/octeontx/ssovf_worker.h +++ b/drivers/event/octeontx/ssovf_worker.h @@ -84,6 +84,15 @@ ssovf_octeontx_wqe_to_pkt(uint64_t work, uint16_t port_info, mbuf->data_len = mbuf->pkt_len; } + if (!!(flag & OCCTX_RX_VLAN_FLTR_F)) { + if (likely(wqe->s.w2.vv)) { + mbuf->ol_flags |= PKT_RX_VLAN; + mbuf->vlan_tci = + ntohs(*((uint16_t *)((char *)mbuf->buf_addr + + mbuf->data_off + wqe->s.w4.vlptr + 2))); + } + } + mbuf->port = rte_octeontx_pchan_map[port_info >> 4][port_info & 0xF]; rte_mbuf_refcnt_set(mbuf, 1); diff --git a/drivers/net/octeontx/octeontx_rxtx.h b/drivers/net/octeontx/octeontx_rxtx.h index e7656f333..515c4699a 100644 --- a/drivers/net/octeontx/octeontx_rxtx.h +++ b/drivers/net/octeontx/octeontx_rxtx.h @@ -489,10 +489,13 @@ T(noff_ol3ol4csum_l3l4csum_mseg, 1, 1, 1, 1, 14, \ MULT_F) /* RX offload macros */ +#define VLAN_FLTR_F OCCTX_RX_VLAN_FLTR_F #define MULT_RX_F OCCTX_RX_MULTI_SEG_F -/* [MULTI_SEG] */ -#define OCCTX_RX_FASTPATH_MODES \ -R(no_offload, 0, OCCTX_RX_OFFLOAD_NONE) \ -R(mseg, 1, MULT_RX_F) \ +/* [VLAN_FLTR][MULTI_SEG] */ +#define OCCTX_RX_FASTPATH_MODES \ +R(no_offload, 0, 0, OCCTX_RX_OFFLOAD_NONE) \ +R(mseg, 0, 1, MULT_RX_F) \ +R(vlan, 1, 0, VLAN_FLTR_F) \ +R(vlan_mseg, 1, 1, VLAN_FLTR_F | MULT_RX_F) #endif /* __OCTEONTX_RXTX_H__ */ From patchwork Mon Mar 16 15:29:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harman Kalra X-Patchwork-Id: 66721 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 810A9A0559; Mon, 16 Mar 2020 16:30:50 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 67FFD1C0B8; Mon, 16 Mar 2020 16:30:09 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 8FEF51C0B1 for ; Mon, 16 Mar 2020 16:30:08 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 02GFKjZ7029578 for ; Mon, 16 Mar 2020 08:30:08 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=pfpt0818; bh=2USWSVv4HXiL+XTOomay3XWUO5lHPYP/DkuvyUjf7zk=; b=vVKxlutLMyeDctCjtJuHLOIcV7ZijwB43+FVBLAqgLnCITgHS3vyV+A/ikpunsjlSb29 fNNs+Ryp/0NkoC/7IWWDUUE0xejc8UA0KZZgW/v8/oIdvG61qQhKKb/LOtVno3FPO1bS kB+dJMnCWpj5CldTZCHgJF0c5n3UmdCzCfxoCxfjlaLNYtjnT2rN9QeqC7Gvi/ClbWc0 QSTARLyqVKmcRI4D7dffeMEPSXnT8U8MrgRuMtswMwD5ilOGvylCevU5qZgfXHouHXqS Wf3ZVAufy3wtilZ0FlhLtheKQ1+m2KjWc2smCUNY7ZDR8+/uN4K2JJS/RsOSNnTdPwYG BA== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0b-0016f401.pphosted.com with ESMTP id 2yrxsmf9ev-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 16 Mar 2020 08:30:07 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 16 Mar 2020 08:30:05 -0700 Received: from SC-EXCH01.marvell.com (10.93.176.81) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 16 Mar 2020 08:30:05 -0700 Received: from NAM04-SN1-obe.outbound.protection.outlook.com (104.47.44.56) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Mon, 16 Mar 2020 08:30:04 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RJaShskRaPn7mPs1LiOUMkY/dsKx42GVSbXw1a1Ffur9BsxVCm76ES4F0U0kgrY1hhWMirNhMerxkBAyMvh1JOaljwTu33WPONPQV5UYu/CFUd77MAMA8T18EVREPNSnqnhshaXvDiiPYS3b0bUf32NJ2ijF/shDsJtzDtHe85KcQVrkETMJOaSc8bWGUQOZygyD7a01Uk/xZwpP1TY6jobaKejwvZdLp4rjrTixc0AS6DrS4nU6a+SAHg7Ky+33O8/+Ti7wI67WqRYsEt7jhACjP7PJ7+dlHq7JBQ4O2xkWl/kB/AXEsgus61dmBewfBoCUJfLDU3pfsJ6rg8bIkA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2USWSVv4HXiL+XTOomay3XWUO5lHPYP/DkuvyUjf7zk=; b=CcNDRhdY4z6memyeVaQbPMocDF52NoslbOX2YUXJswDSKMJdFzuPe48Wdrdg0JgLM7F/LUYQ4BTFImWvEXPf0ropFx9j7v1JKy3wY3tFRTTyK74pd9yozKFfwgL6xf8gkDnyKI4F4do3euuSIM4LOmdHCs2yHrYgz5mwJjII9L/xhIg3VwhNCpwweuHFMPPO4HpxSmyhsAQB85Pt/alkEk+gtuEi5k2vOVDEK4sZzJdLksustui3t4AQ+XJo6KOUTbdgXwEDwEVga6TcsQe8fkDKhhsIdHWNld6UAfoPPmFoIGOHAh1FPdd2fFl5s2ZlCHsES9qsAvfcB8dDAGH1Aw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2USWSVv4HXiL+XTOomay3XWUO5lHPYP/DkuvyUjf7zk=; b=RrelKAmUGfYlXjBlxkNxVwgCjdBLXfMFW9JF9xJl9ZO3tHp0DDwxA8m2uO1ulUtyX1upmFh0ZkEmxWUhVgU4LzgHubGPUDyAFCdX14rAytYn6/JNplcl0S3+TZ5jZOWSMu1SVciuRcO0CPfc7AjBPrTfAHESDpenujo3Ezz+J34= Received: from MN2PR18MB2848.namprd18.prod.outlook.com (2603:10b6:208:38::19) by MN2PR18MB2640.namprd18.prod.outlook.com (2603:10b6:208:ad::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2814.18; Mon, 16 Mar 2020 15:30:03 +0000 Received: from MN2PR18MB2848.namprd18.prod.outlook.com ([fe80::f829:3e55:94f6:4efb]) by MN2PR18MB2848.namprd18.prod.outlook.com ([fe80::f829:3e55:94f6:4efb%5]) with mapi id 15.20.2814.021; Mon, 16 Mar 2020 15:30:03 +0000 From: Harman Kalra To: CC: , , Harman Kalra Date: Mon, 16 Mar 2020 20:59:13 +0530 Message-ID: <1584372553-28710-5-git-send-email-hkalra@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1584372553-28710-1-git-send-email-hkalra@marvell.com> References: <1584372553-28710-1-git-send-email-hkalra@marvell.com> X-ClientProxiedBy: BM1PR0101CA0036.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:1a::22) To MN2PR18MB2848.namprd18.prod.outlook.com (2603:10b6:208:38::19) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from hkarlara-OptiPlex-3046.marvell.com (115.113.156.2) by BM1PR0101CA0036.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:1a::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.2814.13 via Frontend Transport; Mon, 16 Mar 2020 15:30:01 +0000 X-Mailer: git-send-email 2.7.4 X-Originating-IP: [115.113.156.2] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b846ffc9-e237-4a64-c595-08d7c9bee5ba X-MS-TrafficTypeDiagnostic: MN2PR18MB2640: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:826; X-Forefront-PRVS: 03449D5DD1 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4636009)(136003)(366004)(396003)(39850400004)(376002)(346002)(199004)(34206002)(107886003)(4326008)(55236004)(7696005)(52116002)(66946007)(36756003)(66476007)(6666004)(66556008)(2616005)(30864003)(956004)(81166006)(6486002)(37006003)(478600001)(2906002)(86362001)(81156014)(16526019)(186003)(26005)(316002)(6636002)(5660300002)(8676002)(8936002); DIR:OUT; SFP:1101; SCL:1; SRVR:MN2PR18MB2640; H:MN2PR18MB2848.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; Received-SPF: None (protection.outlook.com: marvell.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: dMMykh2V2ZlsCdM6d9dolL1MQnMzBK6MUB3nvbJXCo0ci2iBPJTGla3rYJrVfkjo/h1IoMd2q14fT6lyjMTOUex9/m/KsRrq5AgTpLNi4kV7reiejJl/PvTYYlea4KmejeHTG+d1OMuWHaRuHpoKPV8gF4Opnq2IIDwrvbB8LE++dH09MlT6XuRO7B2dbGIeGaIyAfpgfwXYc6mynTeOH/XOeZB8fCUP0+/YSk7iB8o0Tgw9YXSor9DBGwZG+rehLxQuWz645oz88Er1mzVX2AsRHrT9k/bjKNmxUI+W5JqM+vZ1fiSASTuJGr5JYJOehi5Ipk5IGYcdOQOyh2UExgfzhsasd5fbpHmn2NMqN9/fOnzUSigm1xTVVrs6w+W7GzKwc7/YSVIbjTXCHHLDQFZdTL4B3cLMdnNbuhNGsSqlZb2QxPY5wYe7ibkT1NdG X-MS-Exchange-AntiSpam-MessageData: Pi/4PvtBbTD7fitR6z3aY2utHZHwesPnYI54EcRlcPVrp6riTTroTFOBO5rN5+XvDabU2H3nXEQhHGqUskzfZ0tbLHKRLpaxFbsHS77gBVOybdBX+nf2fa1GXRDg6KhQunrz14wYbS5dsAAIvhYygg== X-MS-Exchange-CrossTenant-Network-Message-Id: b846ffc9-e237-4a64-c595-08d7c9bee5ba X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Mar 2020 15:30:03.2595 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: smPYhme8kl312RaJHH+4DsaxqcbxqNsxaH32RZaa4UQwXQWPFB47cdC5nljORKGd7sn49J3FDT9HYPnQ2IIq8A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR18MB2640 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.645 definitions=2020-03-16_06:2020-03-12, 2020-03-16 signatures=0 Subject: [dpdk-dev] [PATCH 4/4] event/octeontx: support Rx Tx checksum offload X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Adding support for rx checksum offload. In case of wrong checksum received (inner/outer l3/l4) it reports the corresponding layer which has bad checksum. It also adds rx burst function pointer hook for rx checksum offload to event PMD. Signed-off-by: Harman Kalra --- drivers/event/octeontx/ssovf_evdev.c | 1 + drivers/event/octeontx/ssovf_evdev.h | 2 + drivers/event/octeontx/ssovf_worker.c | 127 ++++++++++++++++++++++---- drivers/event/octeontx/ssovf_worker.h | 44 ++++++++- drivers/net/octeontx/octeontx_rxtx.h | 17 +++- 5 files changed, 168 insertions(+), 23 deletions(-) diff --git a/drivers/event/octeontx/ssovf_evdev.c b/drivers/event/octeontx/ssovf_evdev.c index 5d074bcbc..1b1a5d939 100644 --- a/drivers/event/octeontx/ssovf_evdev.c +++ b/drivers/event/octeontx/ssovf_evdev.c @@ -272,6 +272,7 @@ ssovf_port_setup(struct rte_eventdev *dev, uint8_t port_id, reg_off |= 1 << 16; /* Wait */ ws->getwork = ws->base + reg_off; ws->port = port_id; + ws->lookup_mem = octeontx_fastpath_lookup_mem_get(); for (q = 0; q < edev->nb_event_queues; q++) { ws->grps[q] = ssovf_bar(OCTEONTX_SSO_GROUP, q, 2); diff --git a/drivers/event/octeontx/ssovf_evdev.h b/drivers/event/octeontx/ssovf_evdev.h index 1f5066c9a..aa5acf246 100644 --- a/drivers/event/octeontx/ssovf_evdev.h +++ b/drivers/event/octeontx/ssovf_evdev.h @@ -157,6 +157,7 @@ struct ssows { uint8_t *getwork; uint8_t *grps[SSO_MAX_VHGRP]; uint8_t port; + void *lookup_mem; } __rte_cache_aligned; static inline struct ssovf_evdev * @@ -182,5 +183,6 @@ int ssovf_info(struct ssovf_info *info); void *ssovf_bar(enum ssovf_type, uint8_t id, uint8_t bar); int test_eventdev_octeontx(void); void ssovf_fastpath_fns_set(struct rte_eventdev *dev); +void *octeontx_fastpath_lookup_mem_get(void); #endif /* __SSOVF_EVDEV_H__ */ diff --git a/drivers/event/octeontx/ssovf_worker.c b/drivers/event/octeontx/ssovf_worker.c index 093f6f000..47995ad9f 100644 --- a/drivers/event/octeontx/ssovf_worker.c +++ b/drivers/event/octeontx/ssovf_worker.c @@ -91,7 +91,7 @@ ssows_release_event(struct ssows *ws) ssows_swtag_untag(ws); } -#define R(name, f1, f0, flags) \ +#define R(name, f2, f1, f0, flags) \ static uint16_t __rte_noinline __hot \ ssows_deq_ ##name(void *port, struct rte_event *ev, uint64_t timeout_ticks) \ { \ @@ -238,7 +238,8 @@ ssows_flush_events(struct ssows *ws, uint8_t queue_id, ev.mbuf = ssovf_octeontx_wqe_to_pkt(get_work1, (ev.event >> 20) & 0x7F, OCCTX_RX_OFFLOAD_NONE | - OCCTX_RX_MULTI_SEG_F); + OCCTX_RX_MULTI_SEG_F, + ws->lookup_mem); else ev.u64 = get_work1; @@ -340,16 +341,16 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC dev->txa_enqueue = ssow_txa_enqueue [!!(edev->tx_offload_flags & OCCTX_TX_OFFLOAD_MBUF_NOFF_F)] - [0] - [0] + [!!(edev->tx_offload_flags & OCCTX_TX_OFFLOAD_OL3_OL4_CSUM_F)] + [!!(edev->tx_offload_flags & OCCTX_TX_OFFLOAD_L3_L4_CSUM_F)] [!!(edev->tx_offload_flags & OCCTX_TX_MULTI_SEG_F)]; dev->txa_enqueue_same_dest = dev->txa_enqueue; /* Assigning dequeue func pointers */ - const event_dequeue_t ssow_deq[2][2] = { -#define R(name, f1, f0, flags) \ - [f1][f0] = ssows_deq_ ##name, + const event_dequeue_t ssow_deq[2][2][2] = { +#define R(name, f2, f1, f0, flags) \ + [f2][f1][f0] = ssows_deq_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R @@ -357,11 +358,12 @@ SSO_RX_ADPTR_ENQ_FASTPATH_FUNC dev->dequeue = ssow_deq [!!(edev->rx_offload_flags & OCCTX_RX_VLAN_FLTR_F)] + [!!(edev->rx_offload_flags & OCCTX_RX_OFFLOAD_CSUM_F)] [!!(edev->rx_offload_flags & OCCTX_RX_MULTI_SEG_F)]; - const event_dequeue_burst_t ssow_deq_burst[2][2] = { -#define R(name, f1, f0, flags) \ - [f1][f0] = ssows_deq_burst_ ##name, + const event_dequeue_burst_t ssow_deq_burst[2][2][2] = { +#define R(name, f2, f1, f0, flags) \ + [f2][f1][f0] = ssows_deq_burst_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R @@ -369,12 +371,13 @@ SSO_RX_ADPTR_ENQ_FASTPATH_FUNC dev->dequeue_burst = ssow_deq_burst [!!(edev->rx_offload_flags & OCCTX_RX_VLAN_FLTR_F)] + [!!(edev->rx_offload_flags & OCCTX_RX_OFFLOAD_CSUM_F)] [!!(edev->rx_offload_flags & OCCTX_RX_MULTI_SEG_F)]; if (edev->is_timeout_deq) { - const event_dequeue_t ssow_deq_timeout[2][2] = { -#define R(name, f1, f0, flags) \ - [f1][f0] = ssows_deq_timeout_ ##name, + const event_dequeue_t ssow_deq_timeout[2][2][2] = { +#define R(name, f2, f1, f0, flags) \ + [f2][f1][f0] = ssows_deq_timeout_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R @@ -382,11 +385,12 @@ SSO_RX_ADPTR_ENQ_FASTPATH_FUNC dev->dequeue = ssow_deq_timeout [!!(edev->rx_offload_flags & OCCTX_RX_VLAN_FLTR_F)] + [!!(edev->rx_offload_flags & OCCTX_RX_OFFLOAD_CSUM_F)] [!!(edev->rx_offload_flags & OCCTX_RX_MULTI_SEG_F)]; - const event_dequeue_burst_t ssow_deq_timeout_burst[2][2] = { -#define R(name, f1, f0, flags) \ - [f1][f0] = ssows_deq_timeout_burst_ ##name, + const event_dequeue_burst_t ssow_deq_timeout_burst[2][2][2] = { +#define R(name, f2, f1, f0, flags) \ + [f2][f1][f0] = ssows_deq_timeout_burst_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R @@ -394,6 +398,97 @@ SSO_RX_ADPTR_ENQ_FASTPATH_FUNC dev->dequeue_burst = ssow_deq_timeout_burst [!!(edev->rx_offload_flags & OCCTX_RX_VLAN_FLTR_F)] + [!!(edev->rx_offload_flags & OCCTX_RX_OFFLOAD_CSUM_F)] [!!(edev->rx_offload_flags & OCCTX_RX_MULTI_SEG_F)]; } } + +static void +octeontx_create_rx_ol_flags_array(void *mem) +{ + uint16_t idx, errcode, errlev; + uint32_t val, *ol_flags; + + /* Skip ptype array memory */ + ol_flags = (uint32_t *)mem; + + for (idx = 0; idx < BIT(ERRCODE_ERRLEN_WIDTH); idx++) { + errcode = idx & 0xff; + errlev = (idx & 0x700) >> 8; + + val = PKT_RX_IP_CKSUM_UNKNOWN; + val |= PKT_RX_L4_CKSUM_UNKNOWN; + val |= PKT_RX_OUTER_L4_CKSUM_UNKNOWN; + + switch (errlev) { + case OCCTX_ERRLEV_RE: + if (errcode) { + val |= PKT_RX_IP_CKSUM_BAD; + val |= PKT_RX_L4_CKSUM_BAD; + } else { + val |= PKT_RX_IP_CKSUM_GOOD; + val |= PKT_RX_L4_CKSUM_GOOD; + } + break; + case OCCTX_ERRLEV_LC: + if (errcode == OCCTX_EC_IP4_CSUM) { + val |= PKT_RX_IP_CKSUM_BAD; + val |= PKT_RX_EIP_CKSUM_BAD; + } else { + val |= PKT_RX_IP_CKSUM_GOOD; + } + break; + case OCCTX_ERRLEV_LD: + /* Check if parsed packet is neither IPv4 or IPV6 */ + if (errcode == OCCTX_EC_IP4_NOT) + break; + val |= PKT_RX_IP_CKSUM_GOOD; + if (errcode == OCCTX_EC_L4_CSUM) + val |= PKT_RX_OUTER_L4_CKSUM_BAD; + else + val |= PKT_RX_L4_CKSUM_GOOD; + break; + case OCCTX_ERRLEV_LE: + if (errcode == OCCTX_EC_IP4_CSUM) + val |= PKT_RX_IP_CKSUM_BAD; + else + val |= PKT_RX_IP_CKSUM_GOOD; + break; + case OCCTX_ERRLEV_LF: + /* Check if parsed packet is neither IPv4 or IPV6 */ + if (errcode == OCCTX_EC_IP4_NOT) + break; + val |= PKT_RX_IP_CKSUM_GOOD; + if (errcode == OCCTX_EC_L4_CSUM) + val |= PKT_RX_L4_CKSUM_BAD; + else + val |= PKT_RX_L4_CKSUM_GOOD; + break; + } + + ol_flags[idx] = val; + } +} + +void * +octeontx_fastpath_lookup_mem_get(void) +{ + const char name[] = OCCTX_FASTPATH_LOOKUP_MEM; + const struct rte_memzone *mz; + void *mem; + + mz = rte_memzone_lookup(name); + if (mz != NULL) + return mz->addr; + + /* Request for the first time */ + mz = rte_memzone_reserve_aligned(name, LOOKUP_ARRAY_SZ, + SOCKET_ID_ANY, 0, OCCTX_ALIGN); + if (mz != NULL) { + mem = mz->addr; + /* Form the rx ol_flags based on errcode */ + octeontx_create_rx_ol_flags_array(mem); + return mem; + } + return NULL; +} diff --git a/drivers/event/octeontx/ssovf_worker.h b/drivers/event/octeontx/ssovf_worker.h index e23bd057e..6b8be6e35 100644 --- a/drivers/event/octeontx/ssovf_worker.h +++ b/drivers/event/octeontx/ssovf_worker.h @@ -10,6 +10,34 @@ #include "ssovf_evdev.h" #include "octeontx_rxtx.h" +/* Alignment */ +#define OCCTX_ALIGN 128 + +/* Fastpath lookup */ +#define OCCTX_FASTPATH_LOOKUP_MEM "octeontx_fastpath_lookup_mem" + +/* WQE's ERRCODE + ERRLEV (11 bits) */ +#define ERRCODE_ERRLEN_WIDTH 11 +#define ERR_ARRAY_SZ ((BIT(ERRCODE_ERRLEN_WIDTH)) *\ + sizeof(uint32_t)) + +#define LOOKUP_ARRAY_SZ (ERR_ARRAY_SZ) + +#define OCCTX_EC_IP4_NOT 0x41 +#define OCCTX_EC_IP4_CSUM 0x42 +#define OCCTX_EC_L4_CSUM 0x62 + +enum OCCTX_ERRLEV_E { + OCCTX_ERRLEV_RE = 0, + OCCTX_ERRLEV_LA = 1, + OCCTX_ERRLEV_LB = 2, + OCCTX_ERRLEV_LC = 3, + OCCTX_ERRLEV_LD = 4, + OCCTX_ERRLEV_LE = 5, + OCCTX_ERRLEV_LF = 6, + OCCTX_ERRLEV_LG = 7, +}; + enum { SSO_SYNC_ORDERED, SSO_SYNC_ATOMIC, @@ -23,6 +51,14 @@ enum { /* SSO Operations */ +static __rte_always_inline uint32_t +ssovf_octeontx_rx_olflags_get(const void * const lookup_mem, const uint64_t in) +{ + const uint32_t * const ol_flags = (const uint32_t *)lookup_mem; + + return ol_flags[(in & 0x7ff)]; +} + static __rte_always_inline void ssovf_octeontx_wqe_xtract_mseg(octtx_wqe_t *wqe, struct rte_mbuf *mbuf) @@ -61,7 +97,7 @@ ssovf_octeontx_wqe_xtract_mseg(octtx_wqe_t *wqe, static __rte_always_inline struct rte_mbuf * ssovf_octeontx_wqe_to_pkt(uint64_t work, uint16_t port_info, - const uint16_t flag) + const uint16_t flag, const void *lookup_mem) { struct rte_mbuf *mbuf; octtx_wqe_t *wqe = (octtx_wqe_t *)(uintptr_t)work; @@ -75,6 +111,10 @@ ssovf_octeontx_wqe_to_pkt(uint64_t work, uint16_t port_info, mbuf->ol_flags = 0; mbuf->pkt_len = wqe->s.w1.len; + if (!!(flag & OCCTX_RX_OFFLOAD_CSUM_F)) + mbuf->ol_flags = ssovf_octeontx_rx_olflags_get(lookup_mem, + wqe->w[2]); + if (!!(flag & OCCTX_RX_MULTI_SEG_F)) { mbuf->nb_segs = wqe->s.w0.bufs; mbuf->data_len = wqe->s.w5.size; @@ -140,7 +180,7 @@ ssows_get_work(struct ssows *ws, struct rte_event *ev, const uint16_t flag) if (get_work1 && ev->event_type == RTE_EVENT_TYPE_ETHDEV) { ev->mbuf = ssovf_octeontx_wqe_to_pkt(get_work1, - (ev->event >> 20) & 0x7F, flag); + (ev->event >> 20) & 0x7F, flag, ws->lookup_mem); } else if (unlikely((get_work0 & 0xFFFFFFFF) == 0xFFFFFFFF)) { ssovf_octeontx_wqe_free(get_work1); return 0; diff --git a/drivers/net/octeontx/octeontx_rxtx.h b/drivers/net/octeontx/octeontx_rxtx.h index 515c4699a..6f850de96 100644 --- a/drivers/net/octeontx/octeontx_rxtx.h +++ b/drivers/net/octeontx/octeontx_rxtx.h @@ -490,12 +490,19 @@ T(noff_ol3ol4csum_l3l4csum_mseg, 1, 1, 1, 1, 14, \ /* RX offload macros */ #define VLAN_FLTR_F OCCTX_RX_VLAN_FLTR_F +#define CSUM_F OCCTX_RX_OFFLOAD_CSUM_F #define MULT_RX_F OCCTX_RX_MULTI_SEG_F -/* [VLAN_FLTR][MULTI_SEG] */ + +/* [VLAN_FLTR] [CSUM_F] [MULTI_SEG] */ #define OCCTX_RX_FASTPATH_MODES \ -R(no_offload, 0, 0, OCCTX_RX_OFFLOAD_NONE) \ -R(mseg, 0, 1, MULT_RX_F) \ -R(vlan, 1, 0, VLAN_FLTR_F) \ -R(vlan_mseg, 1, 1, VLAN_FLTR_F | MULT_RX_F) +R(no_offload, 0, 0, 0, OCCTX_RX_OFFLOAD_NONE) \ +R(mseg, 0, 0, 1, MULT_RX_F) \ +R(csum, 0, 1, 0, CSUM_F) \ +R(csum_mseg, 0, 1, 1, CSUM_F | MULT_RX_F) \ +R(vlan, 1, 0, 0, VLAN_FLTR_F) \ +R(vlan_mseg, 1, 0, 1, VLAN_FLTR_F | MULT_RX_F) \ +R(vlan_csum, 1, 1, 0, VLAN_FLTR_F | CSUM_F) \ +R(vlan_csum_mseg, 1, 1, 1, CSUM_F | VLAN_FLTR_F | \ + MULT_RX_F) #endif /* __OCTEONTX_RXTX_H__ */