From patchwork Thu Oct 12 12:29:59 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrien Mazarguil X-Patchwork-Id: 30284 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A79A81B2DE; Thu, 12 Oct 2017 14:30:40 +0200 (CEST) Received: from mail-wm0-f49.google.com (mail-wm0-f49.google.com [74.125.82.49]) by dpdk.org (Postfix) with ESMTP id 649DE1B293 for ; Thu, 12 Oct 2017 14:30:34 +0200 (CEST) Received: by mail-wm0-f49.google.com with SMTP id u138so12713351wmu.5 for ; Thu, 12 Oct 2017 05:30:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=KnTPa873KHzefN23s/QlM1gun6sAe6jUdDbUCPwHluQ=; b=TLTDjGHBkCPxnRReRAUshplsbJvMB0ZoyvPJKealZix/htn/aOtwvD3fh11pWn4rjI hLDI2la7D/wHvpJseaHNvgNF6C3H7BwcfN7ySkT8zbHFoGaTqCnUqRCcjU0b08xFcKkZ yv+4UpDa3sZYGLnXMT+GOzeXvfGLgSGY5WTQ/0K2OZyL2vcH/MB+MROIRtUZVO8ZHBxv dJHlaaGZQL4PlX/hBIPMHZnYTrtBTekJgDS4RIRyLjXGxeZgQLV0n1HQRIsRAOU/o91K cjOH7B4ZZgVPXgVT6aey5PUjIXgqjABWVeicXMls3W8WPixyjOoj3KFdLiPOUKNkbCXu VwTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=KnTPa873KHzefN23s/QlM1gun6sAe6jUdDbUCPwHluQ=; b=ciGo4BiDLb/LeXZ2VSOBpVq9g+pbJXIqwbnv6X8WFLSDhp0IceFIzgrHhwNV06BW6W YJjxlC17igozl2DjPvxnDKJfqX1f/dpmqOwrQqgazxBCFTL/Q8ybmhCKKkAhVILsCDlF dQh2wOmNIOBmaCaBmhf45aOSAKzPg0Ou25sKvyLGyRDzDqXApiMCjjN/BM+V+V/VutXX D56M8KKosqD+DCHR0aRVWggBtqygJ+X6+y2hBIle0gmb+9d506T/jBC4yB691wKT/zZh hzk7LcjRiPL1T+z4HfIC/2alC6/Re2ql4VhdD++xdnwV4yMJTzWDEE+/YdrTl4LvqopE p8Zg== X-Gm-Message-State: AMCzsaUmg4ksGynaDDwyb8FB2vd3KBuHBHJaM+PlroiFCUQ7Fufw/ZXA l6atV06SXU2/uP44sfO29ikZtw== X-Google-Smtp-Source: AOwi7QD+CwJ0N0QuAs3+Nat64q6xhscaNlOAnXvRDIVt2POOfkzRAub0+ge0/RdiUPi9duBA+Xdtwg== X-Received: by 10.223.163.71 with SMTP id d7mr1851792wrb.209.1507811433969; Thu, 12 Oct 2017 05:30:33 -0700 (PDT) Received: from 6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id x15sm114386wma.32.2017.10.12.05.30.32 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 12 Oct 2017 05:30:32 -0700 (PDT) From: Adrien Mazarguil To: Ferruh Yigit Cc: dev@dpdk.org, Matan Azrad , Ophir Munk , Moti Haimovsky , Vasily Philipov Date: Thu, 12 Oct 2017 14:29:59 +0200 Message-Id: <2d94860bbeb2c4886de4eb6d2aa750cfec9eb7a8.1507810956.git.adrien.mazarguil@6wind.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v6 4/5] net/mlx4: restore Rx offloads X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Moti Haimovsky This patch adds hardware offloading support for IPV4, UDP and TCP checksum verification, including inner/outer checksums on supported tunnel types. It also restores packet type recognition support. Signed-off-by: Vasily Philipov Signed-off-by: Moti Haimovsky Acked-by: Adrien Mazarguil --- doc/guides/nics/features/mlx4.ini | 1 + drivers/net/mlx4/mlx4_ethdev.c | 6 +- drivers/net/mlx4/mlx4_prm.h | 29 ++++++++ drivers/net/mlx4/mlx4_rxq.c | 5 ++ drivers/net/mlx4/mlx4_rxtx.c | 118 ++++++++++++++++++++++++++++++++- drivers/net/mlx4/mlx4_rxtx.h | 2 + 6 files changed, 158 insertions(+), 3 deletions(-) diff --git a/doc/guides/nics/features/mlx4.ini b/doc/guides/nics/features/mlx4.ini index 366e051..f6efd21 100644 --- a/doc/guides/nics/features/mlx4.ini +++ b/doc/guides/nics/features/mlx4.ini @@ -24,6 +24,7 @@ L3 checksum offload = Y L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y +Packet type parsing = Y Basic stats = Y Stats per queue = Y Other kdrv = Y diff --git a/drivers/net/mlx4/mlx4_ethdev.c b/drivers/net/mlx4/mlx4_ethdev.c index a8c0ee2..ca2170e 100644 --- a/drivers/net/mlx4/mlx4_ethdev.c +++ b/drivers/net/mlx4/mlx4_ethdev.c @@ -767,10 +767,14 @@ mlx4_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *info) info->max_mac_addrs = RTE_DIM(priv->mac); info->rx_offload_capa = 0; info->tx_offload_capa = 0; - if (priv->hw_csum) + if (priv->hw_csum) { info->tx_offload_capa |= (DEV_TX_OFFLOAD_IPV4_CKSUM | DEV_TX_OFFLOAD_UDP_CKSUM | DEV_TX_OFFLOAD_TCP_CKSUM); + info->rx_offload_capa |= (DEV_RX_OFFLOAD_IPV4_CKSUM | + DEV_RX_OFFLOAD_UDP_CKSUM | + DEV_RX_OFFLOAD_TCP_CKSUM); + } if (priv->hw_csum_l2tun) info->tx_offload_capa |= DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM; if (mlx4_get_ifname(priv, &ifname) == 0) diff --git a/drivers/net/mlx4/mlx4_prm.h b/drivers/net/mlx4/mlx4_prm.h index df5a6b4..3a77502 100644 --- a/drivers/net/mlx4/mlx4_prm.h +++ b/drivers/net/mlx4/mlx4_prm.h @@ -70,6 +70,14 @@ #define MLX4_SIZE_TO_TXBBS(size) \ (RTE_ALIGN((size), (MLX4_TXBB_SIZE)) >> (MLX4_TXBB_SHIFT)) +/* CQE checksum flags. */ +enum { + MLX4_CQE_L2_TUNNEL_IPV4 = (int)(1u << 25), + MLX4_CQE_L2_TUNNEL_L4_CSUM = (int)(1u << 26), + MLX4_CQE_L2_TUNNEL = (int)(1u << 27), + MLX4_CQE_L2_TUNNEL_IPOK = (int)(1u << 31), +}; + /* Send queue information. */ struct mlx4_sq { uint8_t *buf; /**< SQ buffer. */ @@ -119,4 +127,25 @@ mlx4_get_cqe(struct mlx4_cq *cq, uint32_t index) (cq->cqe_64 << 5)); } +/** + * Transpose a flag in a value. + * + * @param val + * Input value. + * @param from + * Flag to retrieve from input value. + * @param to + * Flag to set in output value. + * + * @return + * Output value with transposed flag enabled if present on input. + */ +static inline uint64_t +mlx4_transpose(uint64_t val, uint64_t from, uint64_t to) +{ + return (from >= to ? + (val & from) / (from / to) : + (val & from) * (to / from)); +} + #endif /* MLX4_PRM_H_ */ diff --git a/drivers/net/mlx4/mlx4_rxq.c b/drivers/net/mlx4/mlx4_rxq.c index 39c83bc..7ce5b26 100644 --- a/drivers/net/mlx4/mlx4_rxq.c +++ b/drivers/net/mlx4/mlx4_rxq.c @@ -464,6 +464,11 @@ mlx4_rx_queue_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, .sges_n = 0, .elts_n = rte_log2_u32(desc), .elts = elts, + /* Toggle Rx checksum offload if hardware supports it. */ + .csum = (priv->hw_csum && + dev->data->dev_conf.rxmode.hw_ip_checksum), + .csum_l2tun = (priv->hw_csum_l2tun && + dev->data->dev_conf.rxmode.hw_ip_checksum), .stats.idx = idx, .socket = socket, }; diff --git a/drivers/net/mlx4/mlx4_rxtx.c b/drivers/net/mlx4/mlx4_rxtx.c index fe7d5d0..87c5261 100644 --- a/drivers/net/mlx4/mlx4_rxtx.c +++ b/drivers/net/mlx4/mlx4_rxtx.c @@ -557,6 +557,107 @@ mlx4_tx_burst(void *dpdk_txq, struct rte_mbuf **pkts, uint16_t pkts_n) } /** + * Translate Rx completion flags to packet type. + * + * @param flags + * Rx completion flags returned by mlx4_cqe_flags(). + * + * @return + * Packet type in mbuf format. + */ +static inline uint32_t +rxq_cq_to_pkt_type(uint32_t flags) +{ + uint32_t pkt_type; + + if (flags & MLX4_CQE_L2_TUNNEL) + pkt_type = + mlx4_transpose(flags, + MLX4_CQE_L2_TUNNEL_IPV4, + RTE_PTYPE_L3_IPV4_EXT_UNKNOWN) | + mlx4_transpose(flags, + MLX4_CQE_STATUS_IPV4_PKT, + RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN); + else + pkt_type = mlx4_transpose(flags, + MLX4_CQE_STATUS_IPV4_PKT, + RTE_PTYPE_L3_IPV4_EXT_UNKNOWN); + return pkt_type; +} + +/** + * Translate Rx completion flags to offload flags. + * + * @param flags + * Rx completion flags returned by mlx4_cqe_flags(). + * @param csum + * Whether Rx checksums are enabled. + * @param csum_l2tun + * Whether Rx L2 tunnel checksums are enabled. + * + * @return + * Offload flags (ol_flags) in mbuf format. + */ +static inline uint32_t +rxq_cq_to_ol_flags(uint32_t flags, int csum, int csum_l2tun) +{ + uint32_t ol_flags = 0; + + if (csum) + ol_flags |= + mlx4_transpose(flags, + MLX4_CQE_STATUS_IP_HDR_CSUM_OK, + PKT_RX_IP_CKSUM_GOOD) | + mlx4_transpose(flags, + MLX4_CQE_STATUS_TCP_UDP_CSUM_OK, + PKT_RX_L4_CKSUM_GOOD); + if ((flags & MLX4_CQE_L2_TUNNEL) && csum_l2tun) + ol_flags |= + mlx4_transpose(flags, + MLX4_CQE_L2_TUNNEL_IPOK, + PKT_RX_IP_CKSUM_GOOD) | + mlx4_transpose(flags, + MLX4_CQE_L2_TUNNEL_L4_CSUM, + PKT_RX_L4_CKSUM_GOOD); + return ol_flags; +} + +/** + * Extract checksum information from CQE flags. + * + * @param cqe + * Pointer to CQE structure. + * @param csum + * Whether Rx checksums are enabled. + * @param csum_l2tun + * Whether Rx L2 tunnel checksums are enabled. + * + * @return + * CQE checksum information. + */ +static inline uint32_t +mlx4_cqe_flags(struct mlx4_cqe *cqe, int csum, int csum_l2tun) +{ + uint32_t flags = 0; + + /* + * The relevant bits are in different locations on their + * CQE fields therefore we can join them in one 32bit + * variable. + */ + if (csum) + flags = (rte_be_to_cpu_32(cqe->status) & + MLX4_CQE_STATUS_IPV4_CSUM_OK); + if (csum_l2tun) + flags |= (rte_be_to_cpu_32(cqe->vlan_my_qpn) & + (MLX4_CQE_L2_TUNNEL | + MLX4_CQE_L2_TUNNEL_IPOK | + MLX4_CQE_L2_TUNNEL_L4_CSUM | + MLX4_CQE_L2_TUNNEL_IPV4)); + return flags; +} + +/** * Poll one CQE from CQ. * * @param rxq @@ -664,8 +765,21 @@ mlx4_rx_burst(void *dpdk_rxq, struct rte_mbuf **pkts, uint16_t pkts_n) goto skip; } pkt = seg; - pkt->packet_type = 0; - pkt->ol_flags = 0; + if (rxq->csum | rxq->csum_l2tun) { + uint32_t flags = + mlx4_cqe_flags(cqe, + rxq->csum, + rxq->csum_l2tun); + + pkt->ol_flags = + rxq_cq_to_ol_flags(flags, + rxq->csum, + rxq->csum_l2tun); + pkt->packet_type = rxq_cq_to_pkt_type(flags); + } else { + pkt->packet_type = 0; + pkt->ol_flags = 0; + } pkt->pkt_len = len; } rep->nb_segs = 1; diff --git a/drivers/net/mlx4/mlx4_rxtx.h b/drivers/net/mlx4/mlx4_rxtx.h index 6c88efb..51af69c 100644 --- a/drivers/net/mlx4/mlx4_rxtx.h +++ b/drivers/net/mlx4/mlx4_rxtx.h @@ -78,6 +78,8 @@ struct rxq { struct rte_mbuf *(*elts)[]; /**< Rx elements. */ volatile struct mlx4_wqe_data_seg (*wqes)[]; /**< HW queue entries. */ volatile uint32_t *rq_db; /**< RQ doorbell record. */ + uint32_t csum:1; /**< Enable checksum offloading. */ + uint32_t csum_l2tun:1; /**< Same for L2 tunnels. */ struct mlx4_cq mcq; /**< Info for directly manipulating the CQ. */ struct mlx4_rxq_stats stats; /**< Rx queue counters. */ unsigned int socket; /**< CPU socket ID for allocations. */