From patchwork Wed Oct 4 18:48:57 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrien Mazarguil X-Patchwork-Id: 29618 Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C2A0D1B6F8; Wed, 4 Oct 2017 20:49:28 +0200 (CEST) Received: from mail-wm0-f54.google.com (mail-wm0-f54.google.com [74.125.82.54]) by dpdk.org (Postfix) with ESMTP id B1AC01B6DD for ; Wed, 4 Oct 2017 20:49:22 +0200 (CEST) Received: by mail-wm0-f54.google.com with SMTP id u138so25010555wmu.4 for ; Wed, 04 Oct 2017 11:49:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ykzKBHglMH/LuP7sdp+5F7QbMZshvBoKVxkQDusPwHo=; b=XHvbHU9CxlQSqg82hdTOGvjyKP5FpnDFlz5kQ73QzYmtw3WB30wdjrLFVtPXpRMg6j eJzoEUOt2m0lpo1DIskI6zdtjCD2OHvzl7Z0KYGhYN6QeoD0CcmajyFKc4v10LtsfNzu j7hOY+dgsZ9hh40a0xYj6mlmGG7pKUQrOWX8Il/IGWmyONNq8xY3IpcYgE5z8wHLOe+d eazq5cs4K1NfgMAfZhliVFKsv9fiOc/SCJL+4Qi2Y9/qPWvM7OaDrxaoHT0jfOzxR05Y DFxWN9o6enuSIIC6+qXBhCaCu+CwB5jzn10UAiwRXk+MFAIHxa7Hjd3LM/gQOHhNfN4O jScg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ykzKBHglMH/LuP7sdp+5F7QbMZshvBoKVxkQDusPwHo=; b=N0Mx0oxFRJjKsm5JBZu1SZpwyWYkF2HjGdSbhPkojo4LFLX9wpbDUVfYH1a2ziHQBh 3lsyM5W9abmNl9bMmcCwvS9F+J8mUNV+yhDapIKWqnJKwilnKDYpdPAFNUbt3ov9C0WI Kw13B1c5lyCusHjj9QArMhtkL8jj1WVgEhWlTU34mBQUt8tmwKgoRv/UjtCr9Vuca7D9 em8C6oix11QX+iy1aBxLvuVDuhzUmcUdPyUbx8I8kMAGNzS70xKLzlDNBUnvwfRzbxg5 SAYwDYXpzQKBADwEG1JbKqNSbCKBj8jFwsArJDRw7o3QLPBmkZNdu/rcr/XmWimeZDKR xjVw== X-Gm-Message-State: AHPjjUiqwK3x9bgsdwGOt9TfiSAUMQljz0GI+c/Q+40FkNjNMSIweUbd 0ifjDpfpCUDzwZ81CuHY9FHvs6zn X-Google-Smtp-Source: AOwi7QCJaQPf7w26MPSCKe/4XAtU3c9t0BqG/eE1KzP7xFILA5OKujXfEBCNO3MjieaFKEM0rKLNUg== X-Received: by 10.80.191.65 with SMTP id g1mr29480475edk.243.1507142962033; Wed, 04 Oct 2017 11:49:22 -0700 (PDT) Received: from 6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id g30sm16124189edb.63.2017.10.04.11.49.20 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 04 Oct 2017 11:49:21 -0700 (PDT) From: Adrien Mazarguil To: dev@dpdk.org Cc: Moti Haimovsky , Matan Azrad , Vasily Philipov Date: Wed, 4 Oct 2017 20:48:57 +0200 Message-Id: X-Mailer: git-send-email 2.1.4 In-Reply-To: References: <1507027711-879-1-git-send-email-matan@mellanox.com> Subject: [dpdk-dev] [PATCH v3 5/6] net/mlx4: restore Rx offloads X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Moti Haimovsky This patch adds hardware offloading support for IPV4, UDP and TCP checksum verification, including inner/outer checksums on supported tunnel types. It also restores packet type recognition support. Signed-off-by: Vasily Philipov Signed-off-by: Moti Haimovsky Acked-by: Adrien Mazarguil --- drivers/net/mlx4/mlx4_ethdev.c | 6 +- drivers/net/mlx4/mlx4_prm.h | 30 +++++++++ drivers/net/mlx4/mlx4_rxq.c | 5 ++ drivers/net/mlx4/mlx4_rxtx.c | 118 +++++++++++++++++++++++++++++++++++- drivers/net/mlx4/mlx4_rxtx.h | 2 + 5 files changed, 158 insertions(+), 3 deletions(-) diff --git a/drivers/net/mlx4/mlx4_ethdev.c b/drivers/net/mlx4/mlx4_ethdev.c index bec1787..6dbf273 100644 --- a/drivers/net/mlx4/mlx4_ethdev.c +++ b/drivers/net/mlx4/mlx4_ethdev.c @@ -553,10 +553,14 @@ mlx4_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *info) info->max_mac_addrs = 1; info->rx_offload_capa = 0; info->tx_offload_capa = 0; - if (priv->hw_csum) + if (priv->hw_csum) { info->tx_offload_capa |= (DEV_TX_OFFLOAD_IPV4_CKSUM | DEV_TX_OFFLOAD_UDP_CKSUM | DEV_TX_OFFLOAD_TCP_CKSUM); + info->rx_offload_capa |= (DEV_RX_OFFLOAD_IPV4_CKSUM | + DEV_RX_OFFLOAD_UDP_CKSUM | + DEV_RX_OFFLOAD_TCP_CKSUM); + } if (priv->hw_csum_l2tun) info->tx_offload_capa |= DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM; if (mlx4_get_ifname(priv, &ifname) == 0) diff --git a/drivers/net/mlx4/mlx4_prm.h b/drivers/net/mlx4/mlx4_prm.h index df5a6b4..0d76a73 100644 --- a/drivers/net/mlx4/mlx4_prm.h +++ b/drivers/net/mlx4/mlx4_prm.h @@ -70,6 +70,14 @@ #define MLX4_SIZE_TO_TXBBS(size) \ (RTE_ALIGN((size), (MLX4_TXBB_SIZE)) >> (MLX4_TXBB_SHIFT)) +/* CQE checksum flags. */ +enum { + MLX4_CQE_L2_TUNNEL_IPV4 = (int)(1u << 25), + MLX4_CQE_L2_TUNNEL_L4_CSUM = (int)(1u << 26), + MLX4_CQE_L2_TUNNEL = (int)(1u << 27), + MLX4_CQE_L2_TUNNEL_IPOK = (int)(1u << 31), +}; + /* Send queue information. */ struct mlx4_sq { uint8_t *buf; /**< SQ buffer. */ @@ -119,4 +127,26 @@ mlx4_get_cqe(struct mlx4_cq *cq, uint32_t index) (cq->cqe_64 << 5)); } +/** + * Transpose a flag in a value. + * + * @param val + * Input value. + * @param from + * Flag to retrieve from input value. + * @param to + * Flag to set in output value. + * + * @return + * Output value with transposed flag enabled if present on input. + */ +static inline uint64_t +mlx4_transpose(uint64_t val, uint64_t from, uint64_t to) +{ + return (from >= to ? + (val & from) / (from / to) : + (val & from) * (to / from)); +} + + #endif /* MLX4_PRM_H_ */ diff --git a/drivers/net/mlx4/mlx4_rxq.c b/drivers/net/mlx4/mlx4_rxq.c index 44d095d..a021a32 100644 --- a/drivers/net/mlx4/mlx4_rxq.c +++ b/drivers/net/mlx4/mlx4_rxq.c @@ -260,6 +260,11 @@ mlx4_rxq_setup(struct rte_eth_dev *dev, struct rxq *rxq, uint16_t desc, int ret; (void)conf; /* Thresholds configuration (ignored). */ + /* Toggle Rx checksum offload if hardware supports it. */ + if (priv->hw_csum) + tmpl.csum = !!dev->data->dev_conf.rxmode.hw_ip_checksum; + if (priv->hw_csum_l2tun) + tmpl.csum_l2tun = !!dev->data->dev_conf.rxmode.hw_ip_checksum; mb_len = rte_pktmbuf_data_room_size(mp); if (desc == 0) { rte_errno = EINVAL; diff --git a/drivers/net/mlx4/mlx4_rxtx.c b/drivers/net/mlx4/mlx4_rxtx.c index fe7d5d0..87c5261 100644 --- a/drivers/net/mlx4/mlx4_rxtx.c +++ b/drivers/net/mlx4/mlx4_rxtx.c @@ -557,6 +557,107 @@ mlx4_tx_burst(void *dpdk_txq, struct rte_mbuf **pkts, uint16_t pkts_n) } /** + * Translate Rx completion flags to packet type. + * + * @param flags + * Rx completion flags returned by mlx4_cqe_flags(). + * + * @return + * Packet type in mbuf format. + */ +static inline uint32_t +rxq_cq_to_pkt_type(uint32_t flags) +{ + uint32_t pkt_type; + + if (flags & MLX4_CQE_L2_TUNNEL) + pkt_type = + mlx4_transpose(flags, + MLX4_CQE_L2_TUNNEL_IPV4, + RTE_PTYPE_L3_IPV4_EXT_UNKNOWN) | + mlx4_transpose(flags, + MLX4_CQE_STATUS_IPV4_PKT, + RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN); + else + pkt_type = mlx4_transpose(flags, + MLX4_CQE_STATUS_IPV4_PKT, + RTE_PTYPE_L3_IPV4_EXT_UNKNOWN); + return pkt_type; +} + +/** + * Translate Rx completion flags to offload flags. + * + * @param flags + * Rx completion flags returned by mlx4_cqe_flags(). + * @param csum + * Whether Rx checksums are enabled. + * @param csum_l2tun + * Whether Rx L2 tunnel checksums are enabled. + * + * @return + * Offload flags (ol_flags) in mbuf format. + */ +static inline uint32_t +rxq_cq_to_ol_flags(uint32_t flags, int csum, int csum_l2tun) +{ + uint32_t ol_flags = 0; + + if (csum) + ol_flags |= + mlx4_transpose(flags, + MLX4_CQE_STATUS_IP_HDR_CSUM_OK, + PKT_RX_IP_CKSUM_GOOD) | + mlx4_transpose(flags, + MLX4_CQE_STATUS_TCP_UDP_CSUM_OK, + PKT_RX_L4_CKSUM_GOOD); + if ((flags & MLX4_CQE_L2_TUNNEL) && csum_l2tun) + ol_flags |= + mlx4_transpose(flags, + MLX4_CQE_L2_TUNNEL_IPOK, + PKT_RX_IP_CKSUM_GOOD) | + mlx4_transpose(flags, + MLX4_CQE_L2_TUNNEL_L4_CSUM, + PKT_RX_L4_CKSUM_GOOD); + return ol_flags; +} + +/** + * Extract checksum information from CQE flags. + * + * @param cqe + * Pointer to CQE structure. + * @param csum + * Whether Rx checksums are enabled. + * @param csum_l2tun + * Whether Rx L2 tunnel checksums are enabled. + * + * @return + * CQE checksum information. + */ +static inline uint32_t +mlx4_cqe_flags(struct mlx4_cqe *cqe, int csum, int csum_l2tun) +{ + uint32_t flags = 0; + + /* + * The relevant bits are in different locations on their + * CQE fields therefore we can join them in one 32bit + * variable. + */ + if (csum) + flags = (rte_be_to_cpu_32(cqe->status) & + MLX4_CQE_STATUS_IPV4_CSUM_OK); + if (csum_l2tun) + flags |= (rte_be_to_cpu_32(cqe->vlan_my_qpn) & + (MLX4_CQE_L2_TUNNEL | + MLX4_CQE_L2_TUNNEL_IPOK | + MLX4_CQE_L2_TUNNEL_L4_CSUM | + MLX4_CQE_L2_TUNNEL_IPV4)); + return flags; +} + +/** * Poll one CQE from CQ. * * @param rxq @@ -664,8 +765,21 @@ mlx4_rx_burst(void *dpdk_rxq, struct rte_mbuf **pkts, uint16_t pkts_n) goto skip; } pkt = seg; - pkt->packet_type = 0; - pkt->ol_flags = 0; + if (rxq->csum | rxq->csum_l2tun) { + uint32_t flags = + mlx4_cqe_flags(cqe, + rxq->csum, + rxq->csum_l2tun); + + pkt->ol_flags = + rxq_cq_to_ol_flags(flags, + rxq->csum, + rxq->csum_l2tun); + pkt->packet_type = rxq_cq_to_pkt_type(flags); + } else { + pkt->packet_type = 0; + pkt->ol_flags = 0; + } pkt->pkt_len = len; } rep->nb_segs = 1; diff --git a/drivers/net/mlx4/mlx4_rxtx.h b/drivers/net/mlx4/mlx4_rxtx.h index a742f61..6aad41a 100644 --- a/drivers/net/mlx4/mlx4_rxtx.h +++ b/drivers/net/mlx4/mlx4_rxtx.h @@ -77,6 +77,8 @@ struct rxq { struct rte_mbuf *(*elts)[]; /**< Rx elements. */ volatile struct mlx4_wqe_data_seg (*wqes)[]; /**< HW queue entries. */ volatile uint32_t *rq_db; /**< RQ doorbell record. */ + uint32_t csum:1; /**< Enable checksum offloading. */ + uint32_t csum_l2tun:1; /**< Same for L2 tunnels. */ struct mlx4_cq mcq; /**< Info for directly manipulating the CQ. */ struct mlx4_rxq_stats stats; /**< Rx queue counters. */ unsigned int socket; /**< CPU socket ID for allocations. */