From patchwork Thu Apr 18 20:59:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eduard Serra X-Patchwork-Id: 52949 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 374631BC15; Thu, 18 Apr 2019 22:59:42 +0200 (CEST) Received: from NAM04-SN1-obe.outbound.protection.outlook.com (mail-eopbgr700052.outbound.protection.outlook.com [40.107.70.52]) by dpdk.org (Postfix) with ESMTP id 4CEED1BBF7 for ; Thu, 18 Apr 2019 22:59:40 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vmware.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eLLbDlTL2wSFFyYolwxhUqBArbz0tBOjaNhw+8YOzIQ=; b=GxWcf2y5AXhUAl6sEAOBvddi6ZQ7Zq+2wpE0mRem++MNzdck2QDF6PGVEAFLuVW1+2fqhbI1B75NIwcVEhxNNYL2wOEvd1RGedH1UJeg5gAMx8Y57Y8iCO/zM9rjGaKuthS1IbmD3ibS6Mqy9ErV/15ziYvHSpHThSSDT4bNrmo= Received: from MWHPR05MB3472.namprd05.prod.outlook.com (10.174.248.35) by MWHPR05MB2815.namprd05.prod.outlook.com (10.168.245.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1835.7; Thu, 18 Apr 2019 20:59:38 +0000 Received: from MWHPR05MB3472.namprd05.prod.outlook.com ([fe80::c0af:3553:570:5a48]) by MWHPR05MB3472.namprd05.prod.outlook.com ([fe80::c0af:3553:570:5a48%2]) with mapi id 15.20.1813.009; Thu, 18 Apr 2019 20:59:38 +0000 From: Eduard Serra Miralles To: Yong Wang CC: "dev@dpdk.org" , Eduard Serra Miralles Thread-Topic: [PATCH v2] net/vmxnet3: v4 boot and guest UDP RSS configuration Thread-Index: AQHU9imi32binNRvzEWZRAl/uqUcSg== Date: Thu, 18 Apr 2019 20:59:37 +0000 Message-ID: <1555621175-103437-1-git-send-email-eserra@vmware.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR05CA0015.namprd05.prod.outlook.com (2603:10b6:a03:c0::28) To MWHPR05MB3472.namprd05.prod.outlook.com (2603:10b6:301:40::35) authentication-results: spf=none (sender IP is ) smtp.mailfrom=eserra@vmware.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.7.4 x-originating-ip: [66.170.99.2] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 35b4f3c9-cfc8-4773-5a0a-08d6c440c491 x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600141)(711020)(4605104)(2017052603328)(7193020); SRVR:MWHPR05MB2815; x-ms-traffictypediagnostic: MWHPR05MB2815: x-microsoft-antispam-prvs: x-forefront-prvs: 0011612A55 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(396003)(376002)(366004)(136003)(39860400002)(346002)(199004)(189003)(26005)(186003)(6506007)(386003)(8936002)(81156014)(81166006)(102836004)(256004)(8676002)(2616005)(25786009)(19627235002)(64756008)(66446008)(66946007)(14454004)(86362001)(478600001)(14444005)(36756003)(71200400001)(71190400001)(476003)(486006)(66066001)(4326008)(6862004)(99286004)(52116002)(68736007)(6512007)(107886003)(53936002)(7736002)(305945005)(6636002)(6486002)(97736004)(5660300002)(54906003)(50226002)(316002)(6436002)(6116002)(3846002)(37006003)(2906002)(66476007)(66556008); DIR:OUT; SFP:1101; SCL:1; SRVR:MWHPR05MB2815; H:MWHPR05MB3472.namprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: vmware.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: TWX1C3fsWH0SHNYHW6JhtqAclvH9UnitVVKar6stz7raFOHVz+TH5NYiOYBc7J1TpjUsZ9N42zO2HRqWHCsb0eorqzxtrayMzmVkfU8K+U0V3xhj2uM8mNEwk6K5ECMHCo3iJWQR+O6VSfp1gqYMI/xlLPhqqBdmoM9RDYT2UeszGC0XiKSOPMI2d3iwT6hhqeKTP5nm1QqonEl7CsEIOO1KiQy/PnWPLRU6dcdBBksi0kEuhCSyNCF/oVsTFyuAZhF4L5vHQzBaahCfxkx1+39dN3Ir6PpAj68LDk0DV4x2s52oSttPKOeQ1ZIrR1bYWgfjbjzAJyo3FPsAkWa2BFDGb8itX2etY4e3Zh8HSTVUI1T1IDE7aEw3PBo57+PykFozgPblrJfbVYmkgXTCFBdIeVw7vLWDvMuAlB8dmkw= MIME-Version: 1.0 X-OriginatorOrg: vmware.com X-MS-Exchange-CrossTenant-Network-Message-Id: 35b4f3c9-cfc8-4773-5a0a-08d6c440c491 X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Apr 2019 20:59:37.8197 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b39138ca-3cee-4b4a-a4d6-cd83d9dd62f0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR05MB2815 Subject: [dpdk-dev] [PATCH v2] net/vmxnet3: v4 boot and guest UDP RSS configuration X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Eduard Serra This patch introduces: - VMxnet3 v4 negotiation and, - entirely guest-driven UDP RSS support. VMxnet3 v3 already has UDP RSS support, however it depends on hypervisor provisioning on the VM through ESX specific flags, which are not transparent or known to the guest later on. Vmxnet3 v4 introduces a new API transaction which allows configuring RSS entirely from the guest. This API must be invoked after device shared mem region is initialized. IPv4 ESP RSS (SPI based) is also available, but currently there are no ESP RSS definitions on rte_eth layer to handle that. Signed-off-by: Eduard Serra Acked-by: Yong Wang --- v2: * Added missing defines, as we will not merge them separately --- drivers/net/vmxnet3/base/vmxnet3_defs.h | 12 ++++++++++ drivers/net/vmxnet3/vmxnet3_ethdev.c | 21 ++++++++++++++++- drivers/net/vmxnet3/vmxnet3_ethdev.h | 8 +++++++ drivers/net/vmxnet3/vmxnet3_rxtx.c | 40 +++++++++++++++++++++++++++++++++ 4 files changed, 80 insertions(+), 1 deletion(-) diff --git a/drivers/net/vmxnet3/base/vmxnet3_defs.h b/drivers/net/vmxnet3/base/vmxnet3_defs.h index bbec708..296d7e5 100644 --- a/drivers/net/vmxnet3/base/vmxnet3_defs.h +++ b/drivers/net/vmxnet3/base/vmxnet3_defs.h @@ -89,6 +89,7 @@ typedef enum { VMXNET3_CMD_RESERVED3, VMXNET3_CMD_RESERVED4, VMXNET3_CMD_REGISTER_MEMREGS, + VMXNET3_CMD_SET_RSS_FIELDS, VMXNET3_CMD_FIRST_GET = 0xF00D0000, VMXNET3_CMD_GET_QUEUE_STATUS = VMXNET3_CMD_FIRST_GET, @@ -749,6 +750,15 @@ struct Vmxnet3_MemRegs { #include "vmware_pack_end.h" Vmxnet3_MemRegs; +typedef enum Vmxnet3_RSSField { + VMXNET3_RSS_FIELDS_TCPIP4 = 0x0001, + VMXNET3_RSS_FIELDS_TCPIP6 = 0x0002, + VMXNET3_RSS_FIELDS_UDPIP4 = 0x0004, + VMXNET3_RSS_FIELDS_UDPIP6 = 0x0008, + VMXNET3_RSS_FIELDS_ESPIP4 = 0x0010, + VMXNET3_RSS_FIELDS_ESPIP6 = 0x0020, +} Vmxnet3_RSSField; + /* * If the command data <= 16 bytes, use the shared memory direcly. * Otherwise, use the variable length configuration descriptor. @@ -758,6 +768,8 @@ typedef union Vmxnet3_CmdInfo { Vmxnet3_VariableLenConfDesc varConf; Vmxnet3_SetPolling setPolling; + Vmxnet3_RSSField setRSSFields; + __le16 reserved[2]; __le64 data[2]; } #include "vmware_pack_end.h" diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c index 93e5de9..846d7fd 100644 --- a/drivers/net/vmxnet3/vmxnet3_ethdev.c +++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c @@ -266,7 +266,11 @@ eth_vmxnet3_dev_init(struct rte_eth_dev *eth_dev) ver = VMXNET3_READ_BAR1_REG(hw, VMXNET3_REG_VRRS); PMD_INIT_LOG(DEBUG, "Hardware version : %d", ver); - if (ver & (1 << VMXNET3_REV_3)) { + if (ver & (1 << VMXNET3_REV_4)) { + VMXNET3_WRITE_BAR1_REG(hw, VMXNET3_REG_VRRS, + 1 << VMXNET3_REV_4); + hw->version = VMXNET3_REV_4 + 1; + } else if (ver & (1 << VMXNET3_REV_3)) { VMXNET3_WRITE_BAR1_REG(hw, VMXNET3_REG_VRRS, 1 << VMXNET3_REV_3); hw->version = VMXNET3_REV_3 + 1; @@ -764,6 +768,15 @@ vmxnet3_dev_start(struct rte_eth_dev *dev) PMD_INIT_LOG(DEBUG, "Failed to setup memory region\n"); } + if (VMXNET3_VERSION_GE_4(hw)) { + /* Check for additional RSS */ + ret = vmxnet3_v4_rss_configure(dev); + if (ret != VMXNET3_SUCCESS) { + PMD_INIT_LOG(ERR, "Failed to configure v4 RSS"); + return ret; + } + } + /* Disable interrupts */ vmxnet3_disable_intr(hw); @@ -1141,6 +1154,8 @@ static void vmxnet3_dev_info_get(struct rte_eth_dev *dev __rte_unused, struct rte_eth_dev_info *dev_info) { + struct vmxnet3_hw *hw = dev->data->dev_private; + dev_info->max_rx_queues = VMXNET3_MAX_RX_QUEUES; dev_info->max_tx_queues = VMXNET3_MAX_TX_QUEUES; dev_info->min_rx_bufsize = 1518 + RTE_PKTMBUF_HEADROOM; @@ -1150,6 +1165,10 @@ vmxnet3_dev_info_get(struct rte_eth_dev *dev __rte_unused, dev_info->flow_type_rss_offloads = VMXNET3_RSS_OFFLOAD_ALL; + if (VMXNET3_VERSION_GE_4(hw)) { + dev_info->flow_type_rss_offloads |= VMXNET3_V4_RSS_MASK; + } + dev_info->rx_desc_lim = (struct rte_eth_desc_lim) { .nb_max = VMXNET3_RX_RING_MAX_SIZE, .nb_min = VMXNET3_DEF_RX_RING_SIZE, diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.h b/drivers/net/vmxnet3/vmxnet3_ethdev.h index 5bc3a84..319d739 100644 --- a/drivers/net/vmxnet3/vmxnet3_ethdev.h +++ b/drivers/net/vmxnet3/vmxnet3_ethdev.h @@ -34,6 +34,10 @@ ETH_RSS_IPV6 | \ ETH_RSS_NONFRAG_IPV6_TCP) +#define VMXNET3_V4_RSS_MASK ( \ + ETH_RSS_NONFRAG_IPV4_UDP | \ + ETH_RSS_NONFRAG_IPV6_UDP) + /* RSS configuration structure - shared with device through GPA */ typedef struct VMXNET3_RSSConf { uint16_t hashType; @@ -103,10 +107,12 @@ struct vmxnet3_hw { UPT1_RxStats snapshot_rx_stats[VMXNET3_MAX_RX_QUEUES]; }; +#define VMXNET3_REV_4 3 /* Vmxnet3 Rev. 4 */ #define VMXNET3_REV_3 2 /* Vmxnet3 Rev. 3 */ #define VMXNET3_REV_2 1 /* Vmxnet3 Rev. 2 */ #define VMXNET3_REV_1 0 /* Vmxnet3 Rev. 1 */ +#define VMXNET3_VERSION_GE_4(hw) ((hw)->version >= VMXNET3_REV_4 + 1) #define VMXNET3_VERSION_GE_3(hw) ((hw)->version >= VMXNET3_REV_3 + 1) #define VMXNET3_VERSION_GE_2(hw) ((hw)->version >= VMXNET3_REV_2 + 1) @@ -162,6 +168,8 @@ void vmxnet3_dev_clear_queues(struct rte_eth_dev *dev); void vmxnet3_dev_rx_queue_release(void *rxq); void vmxnet3_dev_tx_queue_release(void *txq); +int vmxnet3_v4_rss_configure(struct rte_eth_dev *dev); + int vmxnet3_dev_rx_queue_setup(struct rte_eth_dev *dev, uint16_t rx_queue_id, uint16_t nb_rx_desc, unsigned int socket_id, const struct rte_eth_rxconf *rx_conf, diff --git a/drivers/net/vmxnet3/vmxnet3_rxtx.c b/drivers/net/vmxnet3/vmxnet3_rxtx.c index d30914a..4867a64 100644 --- a/drivers/net/vmxnet3/vmxnet3_rxtx.c +++ b/drivers/net/vmxnet3/vmxnet3_rxtx.c @@ -1292,6 +1292,46 @@ static uint8_t rss_intel_key[40] = { }; /* + * Additional RSS configurations based on vmxnet v4+ APIs + */ +int +vmxnet3_v4_rss_configure(struct rte_eth_dev *dev) +{ + struct vmxnet3_hw *hw = dev->data->dev_private; + Vmxnet3_DriverShared *shared = hw->shared; + Vmxnet3_CmdInfo *cmdInfo = &shared->cu.cmdInfo; + struct rte_eth_rss_conf *port_rss_conf; + uint64_t rss_hf; + uint32_t ret; + + PMD_INIT_FUNC_TRACE(); + + cmdInfo->setRSSFields = 0; + port_rss_conf = &dev->data->dev_conf.rx_adv_conf.rss_conf; + rss_hf = port_rss_conf->rss_hf & + (VMXNET3_V4_RSS_MASK | VMXNET3_RSS_OFFLOAD_ALL); + + if (rss_hf & ETH_RSS_NONFRAG_IPV4_TCP) + cmdInfo->setRSSFields |= VMXNET3_RSS_FIELDS_TCPIP4; + if (rss_hf & ETH_RSS_NONFRAG_IPV6_TCP) + cmdInfo->setRSSFields |= VMXNET3_RSS_FIELDS_TCPIP6; + if (rss_hf & ETH_RSS_NONFRAG_IPV4_UDP) + cmdInfo->setRSSFields |= VMXNET3_RSS_FIELDS_UDPIP4; + if (rss_hf & ETH_RSS_NONFRAG_IPV6_UDP) + cmdInfo->setRSSFields |= VMXNET3_RSS_FIELDS_UDPIP6; + + VMXNET3_WRITE_BAR1_REG(hw, VMXNET3_REG_CMD, + VMXNET3_CMD_SET_RSS_FIELDS); + ret = VMXNET3_READ_BAR1_REG(hw, VMXNET3_REG_CMD); + + if (ret != VMXNET3_SUCCESS) { + PMD_DRV_LOG(ERR, "Set RSS fields (v4) failed: %d", ret); + } + + return ret; +} + +/* * Configure RSS feature */ int