From patchwork Fri Jul 29 19:30:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xiaoyun" X-Patchwork-Id: 114451 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E6CE3A00C4; Fri, 29 Jul 2022 21:31:04 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D299742C52; Fri, 29 Jul 2022 21:31:04 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by mails.dpdk.org (Postfix) with ESMTP id 7127742C41 for ; Fri, 29 Jul 2022 21:31:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1659123062; x=1690659062; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=rqfosgAXAbqZZdehTdVA25iepn+ALSjUbJjY42yquWI=; b=gBdF5WiMspMvfS6BUOpBjGMA2yjF5xYrE+wmpO5GZBvMDNB7C9RhkPhO HK92pNqOF4RWI7wPbpdwNVzTw2/4+zgAvQwrCqvWH88kbbygzCvylSaVJ ZUkFyZzjavhN9uuPrUQEJG/OErpwA19d1UvW2LtmAxH7uwgYL09matQ59 w7DjkKagWD+TqnkNh8e9MV297gaLWwOmBZ9yqW0uSCgf0UBugT+Pm3L/m CgZTwy1q1F7GqxV7RxD4/HBhErqIWv00mbG5MrIrlOBlE5JXYaEOEkW5c SbVQZ0YnhlfdWMkOhFFbpZEHrmmf7pwwPwrBnnxI33d0+rTPqg42huWLu Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10423"; a="268602906" X-IronPort-AV: E=Sophos;i="5.93,201,1654585200"; d="scan'208";a="268602906" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jul 2022 12:31:01 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,201,1654585200"; d="scan'208";a="577059514" Received: from silpixa00399779.ir.intel.com (HELO silpixa00399779.ger.corp.intel.com) ([10.237.223.111]) by orsmga006.jf.intel.com with ESMTP; 29 Jul 2022 12:30:58 -0700 From: Xiaoyun Li To: junfeng.guo@intel.com, qi.z.zhang@intel.com, awogbemila@google.com, bruce.richardson@intel.com Cc: dev@dpdk.org, Xiaoyun Li , Haiyue Wang Subject: [PATCH 01/10] net/gve: introduce GVE PMD base code Date: Fri, 29 Jul 2022 19:30:33 +0000 Message-Id: <20220729193042.2764633-2-xiaoyun.li@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220729193042.2764633-1-xiaoyun.li@intel.com> References: <20220729193042.2764633-1-xiaoyun.li@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The following base code is based on Google Virtual Ethernet (gve) driver v1.3.0 under MIT license. - gve_adminq.c - gve_adminq.h - gve_desc.h - gve_desc_dqo.h - gve_register.h The original code is in: https://github.com/GoogleCloudPlatform/compute-virtual-ethernet-linux/\ tree/v1.3.0/google/gve Signed-off-by: Xiaoyun Li Signed-off-by: Haiyue Wang --- drivers/net/gve/gve_adminq.c | 925 +++++++++++++++++++++++++++++++++ drivers/net/gve/gve_adminq.h | 381 ++++++++++++++ drivers/net/gve/gve_desc.h | 137 +++++ drivers/net/gve/gve_desc_dqo.h | 254 +++++++++ drivers/net/gve/gve_register.h | 28 + 5 files changed, 1725 insertions(+) create mode 100644 drivers/net/gve/gve_adminq.c create mode 100644 drivers/net/gve/gve_adminq.h create mode 100644 drivers/net/gve/gve_desc.h create mode 100644 drivers/net/gve/gve_desc_dqo.h create mode 100644 drivers/net/gve/gve_register.h diff --git a/drivers/net/gve/gve_adminq.c b/drivers/net/gve/gve_adminq.c new file mode 100644 index 0000000000..8a724f12c6 --- /dev/null +++ b/drivers/net/gve/gve_adminq.c @@ -0,0 +1,925 @@ +/* SPDX-License-Identifier: MIT + * Google Virtual Ethernet (gve) driver + * Version: 1.3.0 + * Copyright (C) 2015-2022 Google, Inc. + * Copyright(C) 2022 Intel Corporation + */ + +#include "gve_adminq.h" +#include "gve_register.h" + +#define GVE_MAX_ADMINQ_RELEASE_CHECK 500 +#define GVE_ADMINQ_SLEEP_LEN 20 +#define GVE_MAX_ADMINQ_EVENT_COUNTER_CHECK 100 + +#define GVE_DEVICE_OPTION_ERROR_FMT "%s option error:\n" \ +"Expected: length=%d, feature_mask=%x.\n" \ +"Actual: length=%d, feature_mask=%x." + +#define GVE_DEVICE_OPTION_TOO_BIG_FMT "Length of %s option larger than expected. Possible older version of guest driver." + +static +struct gve_device_option *gve_get_next_option(struct gve_device_descriptor *descriptor, + struct gve_device_option *option) +{ + uintptr_t option_end, descriptor_end; + + option_end = (uintptr_t)option + sizeof(*option) + be16_to_cpu(option->option_length); + descriptor_end = (uintptr_t)descriptor + be16_to_cpu(descriptor->total_length); + + return option_end > descriptor_end ? NULL : (struct gve_device_option *)option_end; +} + +static +void gve_parse_device_option(struct gve_priv *priv, + struct gve_device_option *option, + struct gve_device_option_gqi_rda **dev_op_gqi_rda, + struct gve_device_option_gqi_qpl **dev_op_gqi_qpl, + struct gve_device_option_dqo_rda **dev_op_dqo_rda, + struct gve_device_option_jumbo_frames **dev_op_jumbo_frames) +{ + u32 req_feat_mask = be32_to_cpu(option->required_features_mask); + u16 option_length = be16_to_cpu(option->option_length); + u16 option_id = be16_to_cpu(option->option_id); + + /* If the length or feature mask doesn't match, continue without + * enabling the feature. + */ + switch (option_id) { + case GVE_DEV_OPT_ID_GQI_RAW_ADDRESSING: + if (option_length != GVE_DEV_OPT_LEN_GQI_RAW_ADDRESSING || + req_feat_mask != GVE_DEV_OPT_REQ_FEAT_MASK_GQI_RAW_ADDRESSING) { + PMD_DRV_LOG(WARNING, GVE_DEVICE_OPTION_ERROR_FMT, + "Raw Addressing", + GVE_DEV_OPT_LEN_GQI_RAW_ADDRESSING, + GVE_DEV_OPT_REQ_FEAT_MASK_GQI_RAW_ADDRESSING, + option_length, req_feat_mask); + break; + } + + PMD_DRV_LOG(INFO, "Gqi raw addressing device option enabled."); + priv->queue_format = GVE_GQI_RDA_FORMAT; + break; + case GVE_DEV_OPT_ID_GQI_RDA: + if (option_length < sizeof(**dev_op_gqi_rda) || + req_feat_mask != GVE_DEV_OPT_REQ_FEAT_MASK_GQI_RDA) { + PMD_DRV_LOG(WARNING, GVE_DEVICE_OPTION_ERROR_FMT, + "GQI RDA", (int)sizeof(**dev_op_gqi_rda), + GVE_DEV_OPT_REQ_FEAT_MASK_GQI_RDA, + option_length, req_feat_mask); + break; + } + + if (option_length > sizeof(**dev_op_gqi_rda)) { + PMD_DRV_LOG(WARNING, + GVE_DEVICE_OPTION_TOO_BIG_FMT, "GQI RDA"); + } + *dev_op_gqi_rda = RTE_PTR_ADD(option, sizeof(*option)); + break; + case GVE_DEV_OPT_ID_GQI_QPL: + if (option_length < sizeof(**dev_op_gqi_qpl) || + req_feat_mask != GVE_DEV_OPT_REQ_FEAT_MASK_GQI_QPL) { + PMD_DRV_LOG(WARNING, GVE_DEVICE_OPTION_ERROR_FMT, + "GQI QPL", (int)sizeof(**dev_op_gqi_qpl), + GVE_DEV_OPT_REQ_FEAT_MASK_GQI_QPL, + option_length, req_feat_mask); + break; + } + + if (option_length > sizeof(**dev_op_gqi_qpl)) { + PMD_DRV_LOG(WARNING, + GVE_DEVICE_OPTION_TOO_BIG_FMT, "GQI QPL"); + } + *dev_op_gqi_qpl = RTE_PTR_ADD(option, sizeof(*option)); + break; + case GVE_DEV_OPT_ID_DQO_RDA: + if (option_length < sizeof(**dev_op_dqo_rda) || + req_feat_mask != GVE_DEV_OPT_REQ_FEAT_MASK_DQO_RDA) { + PMD_DRV_LOG(WARNING, GVE_DEVICE_OPTION_ERROR_FMT, + "DQO RDA", (int)sizeof(**dev_op_dqo_rda), + GVE_DEV_OPT_REQ_FEAT_MASK_DQO_RDA, + option_length, req_feat_mask); + break; + } + + if (option_length > sizeof(**dev_op_dqo_rda)) { + PMD_DRV_LOG(WARNING, + GVE_DEVICE_OPTION_TOO_BIG_FMT, "DQO RDA"); + } + *dev_op_dqo_rda = RTE_PTR_ADD(option, sizeof(*option)); + break; + case GVE_DEV_OPT_ID_JUMBO_FRAMES: + if (option_length < sizeof(**dev_op_jumbo_frames) || + req_feat_mask != GVE_DEV_OPT_REQ_FEAT_MASK_JUMBO_FRAMES) { + PMD_DRV_LOG(WARNING, GVE_DEVICE_OPTION_ERROR_FMT, + "Jumbo Frames", + (int)sizeof(**dev_op_jumbo_frames), + GVE_DEV_OPT_REQ_FEAT_MASK_JUMBO_FRAMES, + option_length, req_feat_mask); + break; + } + + if (option_length > sizeof(**dev_op_jumbo_frames)) { + PMD_DRV_LOG(WARNING, + GVE_DEVICE_OPTION_TOO_BIG_FMT, + "Jumbo Frames"); + } + *dev_op_jumbo_frames = RTE_PTR_ADD(option, sizeof(*option)); + break; + default: + /* If we don't recognize the option just continue + * without doing anything. + */ + PMD_DRV_LOG(DEBUG, "Unrecognized device option 0x%hx not enabled.\n", + option_id); + } +} + +/* Process all device options for a given describe device call. */ +static int +gve_process_device_options(struct gve_priv *priv, + struct gve_device_descriptor *descriptor, + struct gve_device_option_gqi_rda **dev_op_gqi_rda, + struct gve_device_option_gqi_qpl **dev_op_gqi_qpl, + struct gve_device_option_dqo_rda **dev_op_dqo_rda, + struct gve_device_option_jumbo_frames **dev_op_jumbo_frames) +{ + const int num_options = be16_to_cpu(descriptor->num_device_options); + struct gve_device_option *dev_opt; + int i; + + /* The options struct directly follows the device descriptor. */ + dev_opt = RTE_PTR_ADD(descriptor, sizeof(*descriptor)); + for (i = 0; i < num_options; i++) { + struct gve_device_option *next_opt; + + next_opt = gve_get_next_option(descriptor, dev_opt); + if (!next_opt) { + PMD_DRV_LOG(ERR, + "options exceed device_descriptor's total length.\n"); + return -EINVAL; + } + + gve_parse_device_option(priv, dev_opt, + dev_op_gqi_rda, dev_op_gqi_qpl, + dev_op_dqo_rda, dev_op_jumbo_frames); + dev_opt = next_opt; + } + + return 0; +} + +int gve_adminq_alloc(struct gve_priv *priv) +{ + priv->adminq = gve_alloc_dma_mem(&priv->adminq_dma_mem, PAGE_SIZE); + if (unlikely(!priv->adminq)) + return -ENOMEM; + + priv->adminq_mask = (PAGE_SIZE / sizeof(union gve_adminq_command)) - 1; + priv->adminq_prod_cnt = 0; + priv->adminq_cmd_fail = 0; + priv->adminq_timeouts = 0; + priv->adminq_describe_device_cnt = 0; + priv->adminq_cfg_device_resources_cnt = 0; + priv->adminq_register_page_list_cnt = 0; + priv->adminq_unregister_page_list_cnt = 0; + priv->adminq_create_tx_queue_cnt = 0; + priv->adminq_create_rx_queue_cnt = 0; + priv->adminq_destroy_tx_queue_cnt = 0; + priv->adminq_destroy_rx_queue_cnt = 0; + priv->adminq_dcfg_device_resources_cnt = 0; + priv->adminq_set_driver_parameter_cnt = 0; + priv->adminq_report_stats_cnt = 0; + priv->adminq_report_link_speed_cnt = 0; + priv->adminq_get_ptype_map_cnt = 0; + + /* Setup Admin queue with the device */ + iowrite32be(priv->adminq_dma_mem.pa / PAGE_SIZE, + &priv->reg_bar0->adminq_pfn); + + gve_set_admin_queue_ok(priv); + return 0; +} + +void gve_adminq_release(struct gve_priv *priv) +{ + int i = 0; + + /* Tell the device the adminq is leaving */ + iowrite32be(0x0, &priv->reg_bar0->adminq_pfn); + while (ioread32be(&priv->reg_bar0->adminq_pfn)) { + /* If this is reached the device is unrecoverable and still + * holding memory. Continue looping to avoid memory corruption, + * but WARN so it is visible what is going on. + */ + if (i == GVE_MAX_ADMINQ_RELEASE_CHECK) + PMD_DRV_LOG(WARNING, "Unrecoverable platform error!"); + i++; + msleep(GVE_ADMINQ_SLEEP_LEN); + } + gve_clear_device_rings_ok(priv); + gve_clear_device_resources_ok(priv); + gve_clear_admin_queue_ok(priv); +} + +void gve_adminq_free(struct gve_priv *priv) +{ + if (!gve_get_admin_queue_ok(priv)) + return; + gve_adminq_release(priv); + gve_free_dma_mem(&priv->adminq_dma_mem); + gve_clear_admin_queue_ok(priv); +} + +static void gve_adminq_kick_cmd(struct gve_priv *priv, u32 prod_cnt) +{ + iowrite32be(prod_cnt, &priv->reg_bar0->adminq_doorbell); +} + +static bool gve_adminq_wait_for_cmd(struct gve_priv *priv, u32 prod_cnt) +{ + int i; + + for (i = 0; i < GVE_MAX_ADMINQ_EVENT_COUNTER_CHECK; i++) { + if (ioread32be(&priv->reg_bar0->adminq_event_counter) + == prod_cnt) + return true; + msleep(GVE_ADMINQ_SLEEP_LEN); + } + + return false; +} + +static int gve_adminq_parse_err(struct gve_priv *priv, u32 status) +{ + if (status != GVE_ADMINQ_COMMAND_PASSED && + status != GVE_ADMINQ_COMMAND_UNSET) { + PMD_DRV_LOG(ERR, "AQ command failed with status %d", status); + priv->adminq_cmd_fail++; + } + switch (status) { + case GVE_ADMINQ_COMMAND_PASSED: + return 0; + case GVE_ADMINQ_COMMAND_UNSET: + PMD_DRV_LOG(ERR, "parse_aq_err: err and status both unset, this should not be possible."); + return -EINVAL; + case GVE_ADMINQ_COMMAND_ERROR_ABORTED: + case GVE_ADMINQ_COMMAND_ERROR_CANCELLED: + case GVE_ADMINQ_COMMAND_ERROR_DATALOSS: + case GVE_ADMINQ_COMMAND_ERROR_FAILED_PRECONDITION: + case GVE_ADMINQ_COMMAND_ERROR_UNAVAILABLE: + return -EAGAIN; + case GVE_ADMINQ_COMMAND_ERROR_ALREADY_EXISTS: + case GVE_ADMINQ_COMMAND_ERROR_INTERNAL_ERROR: + case GVE_ADMINQ_COMMAND_ERROR_INVALID_ARGUMENT: + case GVE_ADMINQ_COMMAND_ERROR_NOT_FOUND: + case GVE_ADMINQ_COMMAND_ERROR_OUT_OF_RANGE: + case GVE_ADMINQ_COMMAND_ERROR_UNKNOWN_ERROR: + return -EINVAL; + case GVE_ADMINQ_COMMAND_ERROR_DEADLINE_EXCEEDED: + return -ETIME; + case GVE_ADMINQ_COMMAND_ERROR_PERMISSION_DENIED: + case GVE_ADMINQ_COMMAND_ERROR_UNAUTHENTICATED: + return -EACCES; + case GVE_ADMINQ_COMMAND_ERROR_RESOURCE_EXHAUSTED: + return -ENOMEM; + case GVE_ADMINQ_COMMAND_ERROR_UNIMPLEMENTED: + return -ENOTSUP; + default: + PMD_DRV_LOG(ERR, "parse_aq_err: unknown status code %d", + status); + return -EINVAL; + } +} + +/* Flushes all AQ commands currently queued and waits for them to complete. + * If there are failures, it will return the first error. + */ +static int gve_adminq_kick_and_wait(struct gve_priv *priv) +{ + u32 tail, head; + u32 i; + + tail = ioread32be(&priv->reg_bar0->adminq_event_counter); + head = priv->adminq_prod_cnt; + + gve_adminq_kick_cmd(priv, head); + if (!gve_adminq_wait_for_cmd(priv, head)) { + PMD_DRV_LOG(ERR, "AQ commands timed out, need to reset AQ"); + priv->adminq_timeouts++; + return -ENOTRECOVERABLE; + } + + for (i = tail; i < head; i++) { + union gve_adminq_command *cmd; + u32 status, err; + + cmd = &priv->adminq[i & priv->adminq_mask]; + status = be32_to_cpu(READ_ONCE32(cmd->status)); + err = gve_adminq_parse_err(priv, status); + if (err) + /* Return the first error if we failed. */ + return err; + } + + return 0; +} + +/* This function is not threadsafe - the caller is responsible for any + * necessary locks. + */ +static int gve_adminq_issue_cmd(struct gve_priv *priv, + union gve_adminq_command *cmd_orig) +{ + union gve_adminq_command *cmd; + u32 opcode; + u32 tail; + + tail = ioread32be(&priv->reg_bar0->adminq_event_counter); + + /* Check if next command will overflow the buffer. */ + if (((priv->adminq_prod_cnt + 1) & priv->adminq_mask) == + (tail & priv->adminq_mask)) { + int err; + + /* Flush existing commands to make room. */ + err = gve_adminq_kick_and_wait(priv); + if (err) + return err; + + /* Retry. */ + tail = ioread32be(&priv->reg_bar0->adminq_event_counter); + if (((priv->adminq_prod_cnt + 1) & priv->adminq_mask) == + (tail & priv->adminq_mask)) { + /* This should never happen. We just flushed the + * command queue so there should be enough space. + */ + return -ENOMEM; + } + } + + cmd = &priv->adminq[priv->adminq_prod_cnt & priv->adminq_mask]; + priv->adminq_prod_cnt++; + + memcpy(cmd, cmd_orig, sizeof(*cmd_orig)); + opcode = be32_to_cpu(READ_ONCE32(cmd->opcode)); + + switch (opcode) { + case GVE_ADMINQ_DESCRIBE_DEVICE: + priv->adminq_describe_device_cnt++; + break; + case GVE_ADMINQ_CONFIGURE_DEVICE_RESOURCES: + priv->adminq_cfg_device_resources_cnt++; + break; + case GVE_ADMINQ_REGISTER_PAGE_LIST: + priv->adminq_register_page_list_cnt++; + break; + case GVE_ADMINQ_UNREGISTER_PAGE_LIST: + priv->adminq_unregister_page_list_cnt++; + break; + case GVE_ADMINQ_CREATE_TX_QUEUE: + priv->adminq_create_tx_queue_cnt++; + break; + case GVE_ADMINQ_CREATE_RX_QUEUE: + priv->adminq_create_rx_queue_cnt++; + break; + case GVE_ADMINQ_DESTROY_TX_QUEUE: + priv->adminq_destroy_tx_queue_cnt++; + break; + case GVE_ADMINQ_DESTROY_RX_QUEUE: + priv->adminq_destroy_rx_queue_cnt++; + break; + case GVE_ADMINQ_DECONFIGURE_DEVICE_RESOURCES: + priv->adminq_dcfg_device_resources_cnt++; + break; + case GVE_ADMINQ_SET_DRIVER_PARAMETER: + priv->adminq_set_driver_parameter_cnt++; + break; + case GVE_ADMINQ_REPORT_STATS: + priv->adminq_report_stats_cnt++; + break; + case GVE_ADMINQ_REPORT_LINK_SPEED: + priv->adminq_report_link_speed_cnt++; + break; + case GVE_ADMINQ_GET_PTYPE_MAP: + priv->adminq_get_ptype_map_cnt++; + break; + default: + PMD_DRV_LOG(ERR, "unknown AQ command opcode %d", opcode); + } + + return 0; +} + +/* This function is not threadsafe - the caller is responsible for any + * necessary locks. + * The caller is also responsible for making sure there are no commands + * waiting to be executed. + */ +static int gve_adminq_execute_cmd(struct gve_priv *priv, + union gve_adminq_command *cmd_orig) +{ + u32 tail, head; + int err; + + tail = ioread32be(&priv->reg_bar0->adminq_event_counter); + head = priv->adminq_prod_cnt; + if (tail != head) + /* This is not a valid path */ + return -EINVAL; + + err = gve_adminq_issue_cmd(priv, cmd_orig); + if (err) + return err; + + return gve_adminq_kick_and_wait(priv); +} + +/* The device specifies that the management vector can either be the first irq + * or the last irq. ntfy_blk_msix_base_idx indicates the first irq assigned to + * the ntfy blks. It if is 0 then the management vector is last, if it is 1 then + * the management vector is first. + * + * gve arranges the msix vectors so that the management vector is last. + */ +#define GVE_NTFY_BLK_BASE_MSIX_IDX 0 +int gve_adminq_configure_device_resources(struct gve_priv *priv, + dma_addr_t counter_array_bus_addr, + u32 num_counters, + dma_addr_t db_array_bus_addr, + u32 num_ntfy_blks) +{ + union gve_adminq_command cmd; + + memset(&cmd, 0, sizeof(cmd)); + cmd.opcode = cpu_to_be32(GVE_ADMINQ_CONFIGURE_DEVICE_RESOURCES); + cmd.configure_device_resources = + (struct gve_adminq_configure_device_resources) { + .counter_array = cpu_to_be64(counter_array_bus_addr), + .num_counters = cpu_to_be32(num_counters), + .irq_db_addr = cpu_to_be64(db_array_bus_addr), + .num_irq_dbs = cpu_to_be32(num_ntfy_blks), + .irq_db_stride = cpu_to_be32(sizeof(*priv->irq_dbs)), + .ntfy_blk_msix_base_idx = + cpu_to_be32(GVE_NTFY_BLK_BASE_MSIX_IDX), + .queue_format = priv->queue_format, + }; + + return gve_adminq_execute_cmd(priv, &cmd); +} + +int gve_adminq_deconfigure_device_resources(struct gve_priv *priv) +{ + union gve_adminq_command cmd; + + memset(&cmd, 0, sizeof(cmd)); + cmd.opcode = cpu_to_be32(GVE_ADMINQ_DECONFIGURE_DEVICE_RESOURCES); + + return gve_adminq_execute_cmd(priv, &cmd); +} + +static int gve_adminq_create_tx_queue(struct gve_priv *priv, u32 queue_index) +{ + struct gve_tx_queue *txq = priv->txqs[queue_index]; + union gve_adminq_command cmd; + + memset(&cmd, 0, sizeof(cmd)); + cmd.opcode = cpu_to_be32(GVE_ADMINQ_CREATE_TX_QUEUE); + cmd.create_tx_queue = (struct gve_adminq_create_tx_queue) { + .queue_id = cpu_to_be32(queue_index), + .queue_resources_addr = + cpu_to_be64(txq->qres_mz->iova), + .tx_ring_addr = cpu_to_be64(txq->tx_ring_phys_addr), + .ntfy_id = cpu_to_be32(txq->ntfy_id), + }; + + if (gve_is_gqi(priv)) { + u32 qpl_id = priv->queue_format == GVE_GQI_RDA_FORMAT ? + GVE_RAW_ADDRESSING_QPL_ID : txq->qpl->id; + + cmd.create_tx_queue.queue_page_list_id = cpu_to_be32(qpl_id); + } else { + cmd.create_tx_queue.tx_ring_size = + cpu_to_be16(txq->nb_tx_desc); + cmd.create_tx_queue.tx_comp_ring_addr = + cpu_to_be64(txq->complq->tx_ring_phys_addr); + cmd.create_tx_queue.tx_comp_ring_size = + cpu_to_be16(priv->tx_compq_size); + } + + return gve_adminq_issue_cmd(priv, &cmd); +} + +int gve_adminq_create_tx_queues(struct gve_priv *priv, u32 num_queues) +{ + int err; + u32 i; + + for (i = 0; i < num_queues; i++) { + err = gve_adminq_create_tx_queue(priv, i); + if (err) + return err; + } + + return gve_adminq_kick_and_wait(priv); +} + +static int gve_adminq_create_rx_queue(struct gve_priv *priv, u32 queue_index) +{ + struct gve_rx_queue *rxq = priv->rxqs[queue_index]; + union gve_adminq_command cmd; + + memset(&cmd, 0, sizeof(cmd)); + cmd.opcode = cpu_to_be32(GVE_ADMINQ_CREATE_RX_QUEUE); + cmd.create_rx_queue = (struct gve_adminq_create_rx_queue) { + .queue_id = cpu_to_be32(queue_index), + .ntfy_id = cpu_to_be32(rxq->ntfy_id), + .queue_resources_addr = cpu_to_be64(rxq->qres_mz->iova), + }; + + if (gve_is_gqi(priv)) { + u32 qpl_id = priv->queue_format == GVE_GQI_RDA_FORMAT ? + GVE_RAW_ADDRESSING_QPL_ID : rxq->qpl->id; + + cmd.create_rx_queue.rx_desc_ring_addr = + cpu_to_be64(rxq->mz->iova), + cmd.create_rx_queue.rx_data_ring_addr = + cpu_to_be64(rxq->data_mz->iova), + cmd.create_rx_queue.index = cpu_to_be32(queue_index); + cmd.create_rx_queue.queue_page_list_id = cpu_to_be32(qpl_id); + cmd.create_rx_queue.packet_buffer_size = cpu_to_be16(rxq->rx_buf_len); + } else { + cmd.create_rx_queue.rx_ring_size = + cpu_to_be16(priv->rx_desc_cnt); + cmd.create_rx_queue.rx_desc_ring_addr = + cpu_to_be64(rxq->rx_ring_phys_addr); + cmd.create_rx_queue.rx_data_ring_addr = + cpu_to_be64(rxq->bufq->rx_ring_phys_addr); + cmd.create_rx_queue.packet_buffer_size = + cpu_to_be16(rxq->rx_buf_len); + cmd.create_rx_queue.rx_buff_ring_size = + cpu_to_be16(priv->rx_bufq_size); + cmd.create_rx_queue.enable_rsc = !!(priv->enable_lsc); + } + + return gve_adminq_issue_cmd(priv, &cmd); +} + +int gve_adminq_create_rx_queues(struct gve_priv *priv, u32 num_queues) +{ + int err; + u32 i; + + for (i = 0; i < num_queues; i++) { + err = gve_adminq_create_rx_queue(priv, i); + if (err) + return err; + } + + return gve_adminq_kick_and_wait(priv); +} + +static int gve_adminq_destroy_tx_queue(struct gve_priv *priv, u32 queue_index) +{ + union gve_adminq_command cmd; + int err; + + memset(&cmd, 0, sizeof(cmd)); + cmd.opcode = cpu_to_be32(GVE_ADMINQ_DESTROY_TX_QUEUE); + cmd.destroy_tx_queue = (struct gve_adminq_destroy_tx_queue) { + .queue_id = cpu_to_be32(queue_index), + }; + + err = gve_adminq_issue_cmd(priv, &cmd); + if (err) + return err; + + return 0; +} + +int gve_adminq_destroy_tx_queues(struct gve_priv *priv, u32 num_queues) +{ + int err; + u32 i; + + for (i = 0; i < num_queues; i++) { + err = gve_adminq_destroy_tx_queue(priv, i); + if (err) + return err; + } + + return gve_adminq_kick_and_wait(priv); +} + +static int gve_adminq_destroy_rx_queue(struct gve_priv *priv, u32 queue_index) +{ + union gve_adminq_command cmd; + int err; + + memset(&cmd, 0, sizeof(cmd)); + cmd.opcode = cpu_to_be32(GVE_ADMINQ_DESTROY_RX_QUEUE); + cmd.destroy_rx_queue = (struct gve_adminq_destroy_rx_queue) { + .queue_id = cpu_to_be32(queue_index), + }; + + err = gve_adminq_issue_cmd(priv, &cmd); + if (err) + return err; + + return 0; +} + +int gve_adminq_destroy_rx_queues(struct gve_priv *priv, u32 num_queues) +{ + int err; + u32 i; + + for (i = 0; i < num_queues; i++) { + err = gve_adminq_destroy_rx_queue(priv, i); + if (err) + return err; + } + + return gve_adminq_kick_and_wait(priv); +} + +static int gve_set_desc_cnt(struct gve_priv *priv, + struct gve_device_descriptor *descriptor) +{ + priv->tx_desc_cnt = be16_to_cpu(descriptor->tx_queue_entries); + if (priv->tx_desc_cnt * sizeof(priv->txqs[0]->tx_desc_ring[0]) + < PAGE_SIZE) { + PMD_DRV_LOG(ERR, "Tx desc count %d too low", priv->tx_desc_cnt); + return -EINVAL; + } + priv->rx_desc_cnt = be16_to_cpu(descriptor->rx_queue_entries); + if (priv->rx_desc_cnt * sizeof(priv->rxqs[0]->rx_desc_ring[0]) + < PAGE_SIZE) { + PMD_DRV_LOG(ERR, "Rx desc count %d too low", priv->rx_desc_cnt); + return -EINVAL; + } + return 0; +} + +static int +gve_set_desc_cnt_dqo(struct gve_priv *priv, + const struct gve_device_descriptor *descriptor, + const struct gve_device_option_dqo_rda *dev_op_dqo_rda) +{ + priv->tx_desc_cnt = be16_to_cpu(descriptor->tx_queue_entries); + priv->tx_compq_size = be16_to_cpu(dev_op_dqo_rda->tx_comp_ring_entries); + priv->rx_desc_cnt = be16_to_cpu(descriptor->rx_queue_entries); + priv->rx_bufq_size = be16_to_cpu(dev_op_dqo_rda->rx_buff_ring_entries); + + return 0; +} + +static void gve_enable_supported_features(struct gve_priv *priv, + u32 supported_features_mask, + const struct gve_device_option_jumbo_frames + *dev_op_jumbo_frames) +{ + /* Before control reaches this point, the page-size-capped max MTU from + * the gve_device_descriptor field has already been stored in + * priv->dev->max_mtu. We overwrite it with the true max MTU below. + */ + if (dev_op_jumbo_frames && + (supported_features_mask & GVE_SUP_JUMBO_FRAMES_MASK)) { + PMD_DRV_LOG(INFO, "JUMBO FRAMES device option enabled."); + priv->max_mtu = be16_to_cpu(dev_op_jumbo_frames->max_mtu); + } +} + +int gve_adminq_describe_device(struct gve_priv *priv) +{ + struct gve_device_option_jumbo_frames *dev_op_jumbo_frames = NULL; + struct gve_device_option_gqi_rda *dev_op_gqi_rda = NULL; + struct gve_device_option_gqi_qpl *dev_op_gqi_qpl = NULL; + struct gve_device_option_dqo_rda *dev_op_dqo_rda = NULL; + struct gve_device_descriptor *descriptor; + struct gve_dma_mem descriptor_dma_mem; + u32 supported_features_mask = 0; + union gve_adminq_command cmd; + int err = 0; + u8 *mac; + u16 mtu; + + memset(&cmd, 0, sizeof(cmd)); + descriptor = gve_alloc_dma_mem(&descriptor_dma_mem, PAGE_SIZE); + if (!descriptor) + return -ENOMEM; + cmd.opcode = cpu_to_be32(GVE_ADMINQ_DESCRIBE_DEVICE); + cmd.describe_device.device_descriptor_addr = + cpu_to_be64(descriptor_dma_mem.pa); + cmd.describe_device.device_descriptor_version = + cpu_to_be32(GVE_ADMINQ_DEVICE_DESCRIPTOR_VERSION); + cmd.describe_device.available_length = cpu_to_be32(PAGE_SIZE); + + err = gve_adminq_execute_cmd(priv, &cmd); + if (err) + goto free_device_descriptor; + + err = gve_process_device_options(priv, descriptor, &dev_op_gqi_rda, + &dev_op_gqi_qpl, &dev_op_dqo_rda, + &dev_op_jumbo_frames); + if (err) + goto free_device_descriptor; + + /* If the GQI_RAW_ADDRESSING option is not enabled and the queue format + * is not set to GqiRda, choose the queue format in a priority order: + * DqoRda, GqiRda, GqiQpl. Use GqiQpl as default. + */ + if (dev_op_dqo_rda) { + priv->queue_format = GVE_DQO_RDA_FORMAT; + PMD_DRV_LOG(INFO, "Driver is running with DQO RDA queue format."); + supported_features_mask = + be32_to_cpu(dev_op_dqo_rda->supported_features_mask); + } else if (dev_op_gqi_rda) { + priv->queue_format = GVE_GQI_RDA_FORMAT; + PMD_DRV_LOG(INFO, "Driver is running with GQI RDA queue format."); + supported_features_mask = + be32_to_cpu(dev_op_gqi_rda->supported_features_mask); + } else if (priv->queue_format == GVE_GQI_RDA_FORMAT) { + PMD_DRV_LOG(INFO, "Driver is running with GQI RDA queue format."); + } else { + priv->queue_format = GVE_GQI_QPL_FORMAT; + if (dev_op_gqi_qpl) + supported_features_mask = + be32_to_cpu(dev_op_gqi_qpl->supported_features_mask); + PMD_DRV_LOG(INFO, "Driver is running with GQI QPL queue format."); + } + if (gve_is_gqi(priv)) { + err = gve_set_desc_cnt(priv, descriptor); + } else { + /* DQO supports LRO. */ + err = gve_set_desc_cnt_dqo(priv, descriptor, dev_op_dqo_rda); + } + if (err) + goto free_device_descriptor; + + priv->max_registered_pages = + be64_to_cpu(descriptor->max_registered_pages); + mtu = be16_to_cpu(descriptor->mtu); + if (mtu < ETH_MIN_MTU) { + PMD_DRV_LOG(ERR, "MTU %d below minimum MTU", mtu); + err = -EINVAL; + goto free_device_descriptor; + } + priv->max_mtu = mtu; + priv->num_event_counters = be16_to_cpu(descriptor->counters); + rte_memcpy(priv->dev_addr.addr_bytes, descriptor->mac, ETH_ALEN); + mac = descriptor->mac; + PMD_DRV_LOG(INFO, "MAC addr: %02x:%02x:%02x:%02x:%02x:%02x", + mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]); + priv->tx_pages_per_qpl = be16_to_cpu(descriptor->tx_pages_per_qpl); + priv->rx_data_slot_cnt = be16_to_cpu(descriptor->rx_pages_per_qpl); + + if (gve_is_gqi(priv) && priv->rx_data_slot_cnt < priv->rx_desc_cnt) { + PMD_DRV_LOG(ERR, "rx_data_slot_cnt cannot be smaller than rx_desc_cnt, setting rx_desc_cnt down to %d", + priv->rx_data_slot_cnt); + priv->rx_desc_cnt = priv->rx_data_slot_cnt; + } + priv->default_num_queues = be16_to_cpu(descriptor->default_num_queues); + + gve_enable_supported_features(priv, supported_features_mask, + dev_op_jumbo_frames); + +free_device_descriptor: + gve_free_dma_mem(&descriptor_dma_mem); + return err; +} + +int gve_adminq_register_page_list(struct gve_priv *priv, + struct gve_queue_page_list *qpl) +{ + struct gve_dma_mem page_list_dma_mem; + u32 num_entries = qpl->num_entries; + u32 size = num_entries * sizeof(qpl->page_buses[0]); + union gve_adminq_command cmd; + __be64 *page_list; + int err; + u32 i; + + memset(&cmd, 0, sizeof(cmd)); + page_list = gve_alloc_dma_mem(&page_list_dma_mem, size); + if (!page_list) + return -ENOMEM; + + for (i = 0; i < num_entries; i++) + page_list[i] = cpu_to_be64(qpl->page_buses[i]); + + cmd.opcode = cpu_to_be32(GVE_ADMINQ_REGISTER_PAGE_LIST); + cmd.reg_page_list = (struct gve_adminq_register_page_list) { + .page_list_id = cpu_to_be32(qpl->id), + .num_pages = cpu_to_be32(num_entries), + .page_address_list_addr = cpu_to_be64(page_list_dma_mem.pa), + }; + + err = gve_adminq_execute_cmd(priv, &cmd); + gve_free_dma_mem(&page_list_dma_mem); + return err; +} + +int gve_adminq_unregister_page_list(struct gve_priv *priv, u32 page_list_id) +{ + union gve_adminq_command cmd; + + memset(&cmd, 0, sizeof(cmd)); + cmd.opcode = cpu_to_be32(GVE_ADMINQ_UNREGISTER_PAGE_LIST); + cmd.unreg_page_list = (struct gve_adminq_unregister_page_list) { + .page_list_id = cpu_to_be32(page_list_id), + }; + + return gve_adminq_execute_cmd(priv, &cmd); +} + +int gve_adminq_set_mtu(struct gve_priv *priv, u64 mtu) +{ + union gve_adminq_command cmd; + + memset(&cmd, 0, sizeof(cmd)); + cmd.opcode = cpu_to_be32(GVE_ADMINQ_SET_DRIVER_PARAMETER); + cmd.set_driver_param = (struct gve_adminq_set_driver_parameter) { + .parameter_type = cpu_to_be32(GVE_SET_PARAM_MTU), + .parameter_value = cpu_to_be64(mtu), + }; + + return gve_adminq_execute_cmd(priv, &cmd); +} + +int gve_adminq_report_stats(struct gve_priv *priv, u64 stats_report_len, + dma_addr_t stats_report_addr, u64 interval) +{ + union gve_adminq_command cmd; + + memset(&cmd, 0, sizeof(cmd)); + cmd.opcode = cpu_to_be32(GVE_ADMINQ_REPORT_STATS); + cmd.report_stats = (struct gve_adminq_report_stats) { + .stats_report_len = cpu_to_be64(stats_report_len), + .stats_report_addr = cpu_to_be64(stats_report_addr), + .interval = cpu_to_be64(interval), + }; + + return gve_adminq_execute_cmd(priv, &cmd); +} + +int gve_adminq_report_link_speed(struct gve_priv *priv) +{ + struct gve_dma_mem link_speed_region_dma_mem; + union gve_adminq_command gvnic_cmd; + u64 *link_speed_region; + int err; + + link_speed_region = gve_alloc_dma_mem(&link_speed_region_dma_mem, + sizeof(*link_speed_region)); + + if (!link_speed_region) + return -ENOMEM; + + memset(&gvnic_cmd, 0, sizeof(gvnic_cmd)); + gvnic_cmd.opcode = cpu_to_be32(GVE_ADMINQ_REPORT_LINK_SPEED); + gvnic_cmd.report_link_speed.link_speed_address = + cpu_to_be64(link_speed_region_dma_mem.pa); + + err = gve_adminq_execute_cmd(priv, &gvnic_cmd); + + priv->link_speed = be64_to_cpu(*link_speed_region); + gve_free_dma_mem(&link_speed_region_dma_mem); + return err; +} + +int gve_adminq_get_ptype_map_dqo(struct gve_priv *priv, + struct gve_ptype_lut *ptype_lut) +{ + struct gve_dma_mem ptype_map_dma_mem; + struct gve_ptype_map *ptype_map; + union gve_adminq_command cmd; + int err = 0; + int i; + + memset(&cmd, 0, sizeof(cmd)); + ptype_map = gve_alloc_dma_mem(&ptype_map_dma_mem, sizeof(*ptype_map)); + if (!ptype_map) + return -ENOMEM; + + cmd.opcode = cpu_to_be32(GVE_ADMINQ_GET_PTYPE_MAP); + cmd.get_ptype_map = (struct gve_adminq_get_ptype_map) { + .ptype_map_len = cpu_to_be64(sizeof(*ptype_map)), + .ptype_map_addr = cpu_to_be64(ptype_map_dma_mem.pa), + }; + + err = gve_adminq_execute_cmd(priv, &cmd); + if (err) + goto err; + + /* Populate ptype_lut. */ + for (i = 0; i < GVE_NUM_PTYPES; i++) { + ptype_lut->ptypes[i].l3_type = + ptype_map->ptypes[i].l3_type; + ptype_lut->ptypes[i].l4_type = + ptype_map->ptypes[i].l4_type; + } +err: + gve_free_dma_mem(&ptype_map_dma_mem); + return err; +} diff --git a/drivers/net/gve/gve_adminq.h b/drivers/net/gve/gve_adminq.h new file mode 100644 index 0000000000..c7114cc883 --- /dev/null +++ b/drivers/net/gve/gve_adminq.h @@ -0,0 +1,381 @@ +/* SPDX-License-Identifier: MIT + * Google Virtual Ethernet (gve) driver + * Version: 1.3.0 + * Copyright (C) 2015-2022 Google, Inc. + * Copyright(C) 2022 Intel Corporation + */ + +#ifndef _GVE_ADMINQ_H +#define _GVE_ADMINQ_H + +/* Admin queue opcodes */ +enum gve_adminq_opcodes { + GVE_ADMINQ_DESCRIBE_DEVICE = 0x1, + GVE_ADMINQ_CONFIGURE_DEVICE_RESOURCES = 0x2, + GVE_ADMINQ_REGISTER_PAGE_LIST = 0x3, + GVE_ADMINQ_UNREGISTER_PAGE_LIST = 0x4, + GVE_ADMINQ_CREATE_TX_QUEUE = 0x5, + GVE_ADMINQ_CREATE_RX_QUEUE = 0x6, + GVE_ADMINQ_DESTROY_TX_QUEUE = 0x7, + GVE_ADMINQ_DESTROY_RX_QUEUE = 0x8, + GVE_ADMINQ_DECONFIGURE_DEVICE_RESOURCES = 0x9, + GVE_ADMINQ_SET_DRIVER_PARAMETER = 0xB, + GVE_ADMINQ_REPORT_STATS = 0xC, + GVE_ADMINQ_REPORT_LINK_SPEED = 0xD, + GVE_ADMINQ_GET_PTYPE_MAP = 0xE, +}; + +/* Admin queue status codes */ +enum gve_adminq_statuses { + GVE_ADMINQ_COMMAND_UNSET = 0x0, + GVE_ADMINQ_COMMAND_PASSED = 0x1, + GVE_ADMINQ_COMMAND_ERROR_ABORTED = 0xFFFFFFF0, + GVE_ADMINQ_COMMAND_ERROR_ALREADY_EXISTS = 0xFFFFFFF1, + GVE_ADMINQ_COMMAND_ERROR_CANCELLED = 0xFFFFFFF2, + GVE_ADMINQ_COMMAND_ERROR_DATALOSS = 0xFFFFFFF3, + GVE_ADMINQ_COMMAND_ERROR_DEADLINE_EXCEEDED = 0xFFFFFFF4, + GVE_ADMINQ_COMMAND_ERROR_FAILED_PRECONDITION = 0xFFFFFFF5, + GVE_ADMINQ_COMMAND_ERROR_INTERNAL_ERROR = 0xFFFFFFF6, + GVE_ADMINQ_COMMAND_ERROR_INVALID_ARGUMENT = 0xFFFFFFF7, + GVE_ADMINQ_COMMAND_ERROR_NOT_FOUND = 0xFFFFFFF8, + GVE_ADMINQ_COMMAND_ERROR_OUT_OF_RANGE = 0xFFFFFFF9, + GVE_ADMINQ_COMMAND_ERROR_PERMISSION_DENIED = 0xFFFFFFFA, + GVE_ADMINQ_COMMAND_ERROR_UNAUTHENTICATED = 0xFFFFFFFB, + GVE_ADMINQ_COMMAND_ERROR_RESOURCE_EXHAUSTED = 0xFFFFFFFC, + GVE_ADMINQ_COMMAND_ERROR_UNAVAILABLE = 0xFFFFFFFD, + GVE_ADMINQ_COMMAND_ERROR_UNIMPLEMENTED = 0xFFFFFFFE, + GVE_ADMINQ_COMMAND_ERROR_UNKNOWN_ERROR = 0xFFFFFFFF, +}; + +#define GVE_ADMINQ_DEVICE_DESCRIPTOR_VERSION 1 + +/* All AdminQ command structs should be naturally packed. + * GVE_CHECK_STRUCT/UNION_LEN will check struct/union length and throw + * error at compile time when the size is not correct. + */ + +struct gve_adminq_describe_device { + __be64 device_descriptor_addr; + __be32 device_descriptor_version; + __be32 available_length; +}; + +GVE_CHECK_STRUCT_LEN(16, gve_adminq_describe_device); + +struct gve_device_descriptor { + __be64 max_registered_pages; + __be16 reserved1; + __be16 tx_queue_entries; + __be16 rx_queue_entries; + __be16 default_num_queues; + __be16 mtu; + __be16 counters; + __be16 tx_pages_per_qpl; + __be16 rx_pages_per_qpl; + u8 mac[ETH_ALEN]; + __be16 num_device_options; + __be16 total_length; + u8 reserved2[6]; +}; + +GVE_CHECK_STRUCT_LEN(40, gve_device_descriptor); + +struct gve_device_option { + __be16 option_id; + __be16 option_length; + __be32 required_features_mask; +}; + +GVE_CHECK_STRUCT_LEN(8, gve_device_option); + +struct gve_device_option_gqi_rda { + __be32 supported_features_mask; +}; + +GVE_CHECK_STRUCT_LEN(4, gve_device_option_gqi_rda); + +struct gve_device_option_gqi_qpl { + __be32 supported_features_mask; +}; + +GVE_CHECK_STRUCT_LEN(4, gve_device_option_gqi_qpl); + +struct gve_device_option_dqo_rda { + __be32 supported_features_mask; + __be16 tx_comp_ring_entries; + __be16 rx_buff_ring_entries; +}; + +GVE_CHECK_STRUCT_LEN(8, gve_device_option_dqo_rda); + +struct gve_device_option_jumbo_frames { + __be32 supported_features_mask; + __be16 max_mtu; + u8 padding[2]; +}; + +GVE_CHECK_STRUCT_LEN(8, gve_device_option_jumbo_frames); + +/* Terminology: + * + * RDA - Raw DMA Addressing - Buffers associated with SKBs are directly DMA + * mapped and read/updated by the device. + * + * QPL - Queue Page Lists - Driver uses bounce buffers which are DMA mapped with + * the device for read/write and data is copied from/to SKBs. + */ +enum gve_dev_opt_id { + GVE_DEV_OPT_ID_GQI_RAW_ADDRESSING = 0x1, + GVE_DEV_OPT_ID_GQI_RDA = 0x2, + GVE_DEV_OPT_ID_GQI_QPL = 0x3, + GVE_DEV_OPT_ID_DQO_RDA = 0x4, + GVE_DEV_OPT_ID_JUMBO_FRAMES = 0x8, +}; + +enum gve_dev_opt_req_feat_mask { + GVE_DEV_OPT_REQ_FEAT_MASK_GQI_RAW_ADDRESSING = 0x0, + GVE_DEV_OPT_REQ_FEAT_MASK_GQI_RDA = 0x0, + GVE_DEV_OPT_REQ_FEAT_MASK_GQI_QPL = 0x0, + GVE_DEV_OPT_REQ_FEAT_MASK_DQO_RDA = 0x0, + GVE_DEV_OPT_REQ_FEAT_MASK_JUMBO_FRAMES = 0x0, +}; + +enum gve_sup_feature_mask { + GVE_SUP_JUMBO_FRAMES_MASK = 1 << 2, +}; + +#define GVE_DEV_OPT_LEN_GQI_RAW_ADDRESSING 0x0 + +struct gve_adminq_configure_device_resources { + __be64 counter_array; + __be64 irq_db_addr; + __be32 num_counters; + __be32 num_irq_dbs; + __be32 irq_db_stride; + __be32 ntfy_blk_msix_base_idx; + u8 queue_format; + u8 padding[7]; +}; + +GVE_CHECK_STRUCT_LEN(40, gve_adminq_configure_device_resources); + +struct gve_adminq_register_page_list { + __be32 page_list_id; + __be32 num_pages; + __be64 page_address_list_addr; +}; + +GVE_CHECK_STRUCT_LEN(16, gve_adminq_register_page_list); + +struct gve_adminq_unregister_page_list { + __be32 page_list_id; +}; + +GVE_CHECK_STRUCT_LEN(4, gve_adminq_unregister_page_list); + +#define GVE_RAW_ADDRESSING_QPL_ID 0xFFFFFFFF + +struct gve_adminq_create_tx_queue { + __be32 queue_id; + __be32 reserved; + __be64 queue_resources_addr; + __be64 tx_ring_addr; + __be32 queue_page_list_id; + __be32 ntfy_id; + __be64 tx_comp_ring_addr; + __be16 tx_ring_size; + __be16 tx_comp_ring_size; + u8 padding[4]; +}; + +GVE_CHECK_STRUCT_LEN(48, gve_adminq_create_tx_queue); + +struct gve_adminq_create_rx_queue { + __be32 queue_id; + __be32 index; + __be32 reserved; + __be32 ntfy_id; + __be64 queue_resources_addr; + __be64 rx_desc_ring_addr; + __be64 rx_data_ring_addr; + __be32 queue_page_list_id; + __be16 rx_ring_size; + __be16 packet_buffer_size; + __be16 rx_buff_ring_size; + u8 enable_rsc; + u8 padding[5]; +}; + +GVE_CHECK_STRUCT_LEN(56, gve_adminq_create_rx_queue); + +/* Queue resources that are shared with the device */ +struct gve_queue_resources { + union { + struct { + __be32 db_index; /* Device -> Guest */ + __be32 counter_index; /* Device -> Guest */ + }; + u8 reserved[64]; + }; +}; + +GVE_CHECK_STRUCT_LEN(64, gve_queue_resources); + +struct gve_adminq_destroy_tx_queue { + __be32 queue_id; +}; + +GVE_CHECK_STRUCT_LEN(4, gve_adminq_destroy_tx_queue); + +struct gve_adminq_destroy_rx_queue { + __be32 queue_id; +}; + +GVE_CHECK_STRUCT_LEN(4, gve_adminq_destroy_rx_queue); + +/* GVE Set Driver Parameter Types */ +enum gve_set_driver_param_types { + GVE_SET_PARAM_MTU = 0x1, +}; + +struct gve_adminq_set_driver_parameter { + __be32 parameter_type; + u8 reserved[4]; + __be64 parameter_value; +}; + +GVE_CHECK_STRUCT_LEN(16, gve_adminq_set_driver_parameter); + +struct gve_adminq_report_stats { + __be64 stats_report_len; + __be64 stats_report_addr; + __be64 interval; +}; + +GVE_CHECK_STRUCT_LEN(24, gve_adminq_report_stats); + +struct gve_adminq_report_link_speed { + __be64 link_speed_address; +}; + +GVE_CHECK_STRUCT_LEN(8, gve_adminq_report_link_speed); + +struct stats { + __be32 stat_name; + __be32 queue_id; + __be64 value; +}; + +GVE_CHECK_STRUCT_LEN(16, stats); + +struct gve_stats_report { + __be64 written_count; + struct stats stats[]; +}; + +GVE_CHECK_STRUCT_LEN(8, gve_stats_report); + +enum gve_stat_names { + /* stats from gve */ + TX_WAKE_CNT = 1, + TX_STOP_CNT = 2, + TX_FRAMES_SENT = 3, + TX_BYTES_SENT = 4, + TX_LAST_COMPLETION_PROCESSED = 5, + RX_NEXT_EXPECTED_SEQUENCE = 6, + RX_BUFFERS_POSTED = 7, + TX_TIMEOUT_CNT = 8, + /* stats from NIC */ + RX_QUEUE_DROP_CNT = 65, + RX_NO_BUFFERS_POSTED = 66, + RX_DROPS_PACKET_OVER_MRU = 67, + RX_DROPS_INVALID_CHECKSUM = 68, +}; + +enum gve_l3_type { + /* Must be zero so zero initialized LUT is unknown. */ + GVE_L3_TYPE_UNKNOWN = 0, + GVE_L3_TYPE_OTHER, + GVE_L3_TYPE_IPV4, + GVE_L3_TYPE_IPV6, +}; + +enum gve_l4_type { + /* Must be zero so zero initialized LUT is unknown. */ + GVE_L4_TYPE_UNKNOWN = 0, + GVE_L4_TYPE_OTHER, + GVE_L4_TYPE_TCP, + GVE_L4_TYPE_UDP, + GVE_L4_TYPE_ICMP, + GVE_L4_TYPE_SCTP, +}; + +/* These are control path types for PTYPE which are the same as the data path + * types. + */ +struct gve_ptype_entry { + u8 l3_type; + u8 l4_type; +}; + +struct gve_ptype_map { + struct gve_ptype_entry ptypes[1 << 10]; /* PTYPES are always 10 bits. */ +}; + +struct gve_adminq_get_ptype_map { + __be64 ptype_map_len; + __be64 ptype_map_addr; +}; + +union gve_adminq_command { + struct { + __be32 opcode; + __be32 status; + union { + struct gve_adminq_configure_device_resources + configure_device_resources; + struct gve_adminq_create_tx_queue create_tx_queue; + struct gve_adminq_create_rx_queue create_rx_queue; + struct gve_adminq_destroy_tx_queue destroy_tx_queue; + struct gve_adminq_destroy_rx_queue destroy_rx_queue; + struct gve_adminq_describe_device describe_device; + struct gve_adminq_register_page_list reg_page_list; + struct gve_adminq_unregister_page_list unreg_page_list; + struct gve_adminq_set_driver_parameter set_driver_param; + struct gve_adminq_report_stats report_stats; + struct gve_adminq_report_link_speed report_link_speed; + struct gve_adminq_get_ptype_map get_ptype_map; + }; + }; + u8 reserved[64]; +}; + +GVE_CHECK_UNION_LEN(64, gve_adminq_command); + +int gve_adminq_alloc(struct gve_priv *priv); +void gve_adminq_free(struct gve_priv *priv); +void gve_adminq_release(struct gve_priv *priv); +int gve_adminq_describe_device(struct gve_priv *priv); +int gve_adminq_configure_device_resources(struct gve_priv *priv, + dma_addr_t counter_array_bus_addr, + u32 num_counters, + dma_addr_t db_array_bus_addr, + u32 num_ntfy_blks); +int gve_adminq_deconfigure_device_resources(struct gve_priv *priv); +int gve_adminq_create_tx_queues(struct gve_priv *priv, u32 num_queues); +int gve_adminq_destroy_tx_queues(struct gve_priv *priv, u32 queue_id); +int gve_adminq_create_rx_queues(struct gve_priv *priv, u32 num_queues); +int gve_adminq_destroy_rx_queues(struct gve_priv *priv, u32 queue_id); +int gve_adminq_register_page_list(struct gve_priv *priv, + struct gve_queue_page_list *qpl); +int gve_adminq_unregister_page_list(struct gve_priv *priv, u32 page_list_id); +int gve_adminq_set_mtu(struct gve_priv *priv, u64 mtu); +int gve_adminq_report_stats(struct gve_priv *priv, u64 stats_report_len, + dma_addr_t stats_report_addr, u64 interval); +int gve_adminq_report_link_speed(struct gve_priv *priv); + +struct gve_ptype_lut; +int gve_adminq_get_ptype_map_dqo(struct gve_priv *priv, + struct gve_ptype_lut *ptype_lut); + +#endif /* _GVE_ADMINQ_H */ diff --git a/drivers/net/gve/gve_desc.h b/drivers/net/gve/gve_desc.h new file mode 100644 index 0000000000..b531669bc0 --- /dev/null +++ b/drivers/net/gve/gve_desc.h @@ -0,0 +1,137 @@ +/* SPDX-License-Identifier: MIT + * Google Virtual Ethernet (gve) driver + * Version: 1.3.0 + * Copyright (C) 2015-2022 Google, Inc. + */ + +/* GVE Transmit Descriptor formats */ + +#ifndef _GVE_DESC_H_ +#define _GVE_DESC_H_ + +/* A note on seg_addrs + * + * Base addresses encoded in seg_addr are not assumed to be physical + * addresses. The ring format assumes these come from some linear address + * space. This could be physical memory, kernel virtual memory, user virtual + * memory. + * If raw dma addressing is not supported then gVNIC uses lists of registered + * pages. Each queue is assumed to be associated with a single such linear + * address space to ensure a consistent meaning for seg_addrs posted to its + * rings. + */ + +struct gve_tx_pkt_desc { + u8 type_flags; /* desc type is lower 4 bits, flags upper */ + u8 l4_csum_offset; /* relative offset of L4 csum word */ + u8 l4_hdr_offset; /* Offset of start of L4 headers in packet */ + u8 desc_cnt; /* Total descriptors for this packet */ + __be16 len; /* Total length of this packet (in bytes) */ + __be16 seg_len; /* Length of this descriptor's segment */ + __be64 seg_addr; /* Base address (see note) of this segment */ +} __packed; + +struct gve_tx_mtd_desc { + u8 type_flags; /* type is lower 4 bits, subtype upper */ + u8 path_state; /* state is lower 4 bits, hash type upper */ + __be16 reserved0; + __be32 path_hash; + __be64 reserved1; +} __packed; + +struct gve_tx_seg_desc { + u8 type_flags; /* type is lower 4 bits, flags upper */ + u8 l3_offset; /* TSO: 2 byte units to start of IPH */ + __be16 reserved; + __be16 mss; /* TSO MSS */ + __be16 seg_len; + __be64 seg_addr; +} __packed; + +/* GVE Transmit Descriptor Types */ +#define GVE_TXD_STD (0x0 << 4) /* Std with Host Address */ +#define GVE_TXD_TSO (0x1 << 4) /* TSO with Host Address */ +#define GVE_TXD_SEG (0x2 << 4) /* Seg with Host Address */ +#define GVE_TXD_MTD (0x3 << 4) /* Metadata */ + +/* GVE Transmit Descriptor Flags for Std Pkts */ +#define GVE_TXF_L4CSUM BIT(0) /* Need csum offload */ +#define GVE_TXF_TSTAMP BIT(2) /* Timestamp required */ + +/* GVE Transmit Descriptor Flags for TSO Segs */ +#define GVE_TXSF_IPV6 BIT(1) /* IPv6 TSO */ + +/* GVE Transmit Descriptor Options for MTD Segs */ +#define GVE_MTD_SUBTYPE_PATH 0 + +#define GVE_MTD_PATH_STATE_DEFAULT 0 +#define GVE_MTD_PATH_STATE_TIMEOUT 1 +#define GVE_MTD_PATH_STATE_CONGESTION 2 +#define GVE_MTD_PATH_STATE_RETRANSMIT 3 + +#define GVE_MTD_PATH_HASH_NONE (0x0 << 4) +#define GVE_MTD_PATH_HASH_L4 (0x1 << 4) + +/* GVE Receive Packet Descriptor */ +/* The start of an ethernet packet comes 2 bytes into the rx buffer. + * gVNIC adds this padding so that both the DMA and the L3/4 protocol header + * access is aligned. + */ +#define GVE_RX_PAD 2 + +struct gve_rx_desc { + u8 padding[48]; + __be32 rss_hash; /* Receive-side scaling hash (Toeplitz for gVNIC) */ + __be16 mss; + __be16 reserved; /* Reserved to zero */ + u8 hdr_len; /* Header length (L2-L4) including padding */ + u8 hdr_off; /* 64-byte-scaled offset into RX_DATA entry */ + __sum16 csum; /* 1's-complement partial checksum of L3+ bytes */ + __be16 len; /* Length of the received packet */ + __be16 flags_seq; /* Flags [15:3] and sequence number [2:0] (1-7) */ +} __packed; +static_assert(sizeof(struct gve_rx_desc) == 64); + +/* If the device supports raw dma addressing then the addr in data slot is + * the dma address of the buffer. + * If the device only supports registered segments then the addr is a byte + * offset into the registered segment (an ordered list of pages) where the + * buffer is. + */ +union gve_rx_data_slot { + __be64 qpl_offset; + __be64 addr; +}; + +/* GVE Receive Packet Descriptor Seq No */ +#define GVE_SEQNO(x) (be16_to_cpu(x) & 0x7) + +/* GVE Receive Packet Descriptor Flags */ +#define GVE_RXFLG(x) cpu_to_be16(1 << (3 + (x))) +#define GVE_RXF_FRAG GVE_RXFLG(3) /* IP Fragment */ +#define GVE_RXF_IPV4 GVE_RXFLG(4) /* IPv4 */ +#define GVE_RXF_IPV6 GVE_RXFLG(5) /* IPv6 */ +#define GVE_RXF_TCP GVE_RXFLG(6) /* TCP Packet */ +#define GVE_RXF_UDP GVE_RXFLG(7) /* UDP Packet */ +#define GVE_RXF_ERR GVE_RXFLG(8) /* Packet Error Detected */ +#define GVE_RXF_PKT_CONT GVE_RXFLG(10) /* Multi Fragment RX packet */ + +/* GVE IRQ */ +#define GVE_IRQ_ACK BIT(31) +#define GVE_IRQ_MASK BIT(30) +#define GVE_IRQ_EVENT BIT(29) + +static inline bool gve_needs_rss(__be16 flag) +{ + if (flag & GVE_RXF_FRAG) + return false; + if (flag & (GVE_RXF_IPV4 | GVE_RXF_IPV6)) + return true; + return false; +} + +static inline u8 gve_next_seqno(u8 seq) +{ + return (seq + 1) == 8 ? 1 : seq + 1; +} +#endif /* _GVE_DESC_H_ */ diff --git a/drivers/net/gve/gve_desc_dqo.h b/drivers/net/gve/gve_desc_dqo.h new file mode 100644 index 0000000000..0d533abcd1 --- /dev/null +++ b/drivers/net/gve/gve_desc_dqo.h @@ -0,0 +1,254 @@ +/* SPDX-License-Identifier: MIT + * Google Virtual Ethernet (gve) driver + * Version: 1.3.0 + * Copyright (C) 2015-2022 Google, Inc. + */ + +/* GVE DQO Descriptor formats */ + +#ifndef _GVE_DESC_DQO_H_ +#define _GVE_DESC_DQO_H_ + +#define GVE_TX_MAX_HDR_SIZE_DQO 255 +#define GVE_TX_MIN_TSO_MSS_DQO 88 + +#ifndef __LITTLE_ENDIAN_BITFIELD +#error "Only little endian supported" +#endif + +/* Basic TX descriptor (DTYPE 0x0C) */ +struct gve_tx_pkt_desc_dqo { + __le64 buf_addr; + + /* Must be GVE_TX_PKT_DESC_DTYPE_DQO (0xc) */ + u8 dtype: 5; + + /* Denotes the last descriptor of a packet. */ + u8 end_of_packet: 1; + u8 checksum_offload_enable: 1; + + /* If set, will generate a descriptor completion for this descriptor. */ + u8 report_event: 1; + u8 reserved0; + __le16 reserved1; + + /* The TX completion associated with this packet will contain this tag. + */ + __le16 compl_tag; + u16 buf_size: 14; + u16 reserved2: 2; +} __packed; +GVE_CHECK_STRUCT_LEN(16, gve_tx_pkt_desc_dqo); + +#define GVE_TX_PKT_DESC_DTYPE_DQO 0xc +#define GVE_TX_MAX_BUF_SIZE_DQO ((16 * 1024) - 1) + +/* Maximum number of data descriptors allowed per packet, or per-TSO segment. */ +#define GVE_TX_MAX_DATA_DESCS 10 + +/* Min gap between tail and head to avoid cacheline overlap */ +#define GVE_TX_MIN_DESC_PREVENT_CACHE_OVERLAP 4 + +/* "report_event" on TX packet descriptors may only be reported on the last + * descriptor of a TX packet, and they must be spaced apart with at least this + * value. + */ +#define GVE_TX_MIN_RE_INTERVAL 32 + +struct gve_tx_context_cmd_dtype { + u8 dtype: 5; + u8 tso: 1; + u8 reserved1: 2; + + u8 reserved2; +}; + +GVE_CHECK_STRUCT_LEN(2, gve_tx_context_cmd_dtype); + +/* TX Native TSO Context DTYPE (0x05) + * + * "flex" fields allow the driver to send additional packet context to HW. + */ +struct gve_tx_tso_context_desc_dqo { + /* The L4 payload bytes that should be segmented. */ + u32 tso_total_len: 24; + u32 flex10: 8; + + /* Max segment size in TSO excluding headers. */ + u16 mss: 14; + u16 reserved: 2; + + u8 header_len; /* Header length to use for TSO offload */ + u8 flex11; + struct gve_tx_context_cmd_dtype cmd_dtype; + u8 flex0; + u8 flex5; + u8 flex6; + u8 flex7; + u8 flex8; + u8 flex9; +} __packed; +GVE_CHECK_STRUCT_LEN(16, gve_tx_tso_context_desc_dqo); + +#define GVE_TX_TSO_CTX_DESC_DTYPE_DQO 0x5 + +/* General context descriptor for sending metadata. */ +struct gve_tx_general_context_desc_dqo { + u8 flex4; + u8 flex5; + u8 flex6; + u8 flex7; + u8 flex8; + u8 flex9; + u8 flex10; + u8 flex11; + struct gve_tx_context_cmd_dtype cmd_dtype; + u16 reserved; + u8 flex0; + u8 flex1; + u8 flex2; + u8 flex3; +} __packed; +GVE_CHECK_STRUCT_LEN(16, gve_tx_general_context_desc_dqo); + +#define GVE_TX_GENERAL_CTX_DESC_DTYPE_DQO 0x4 + +/* Logical structure of metadata which is packed into context descriptor flex + * fields. + */ +struct gve_tx_metadata_dqo { + union { + struct { + u8 version; + + /* If `skb->l4_hash` is set, this value should be + * derived from `skb->hash`. + * + * A zero value means no l4_hash was associated with the + * skb. + */ + u16 path_hash: 15; + + /* Should be set to 1 if the flow associated with the + * skb had a rehash from the TCP stack. + */ + u16 rehash_event: 1; + } __packed; + u8 bytes[12]; + }; +} __packed; +GVE_CHECK_STRUCT_LEN(12, gve_tx_metadata_dqo); + +#define GVE_TX_METADATA_VERSION_DQO 0 + +/* TX completion descriptor */ +struct gve_tx_compl_desc { + /* For types 0-4 this is the TX queue ID associated with this + * completion. + */ + u16 id: 11; + + /* See: GVE_COMPL_TYPE_DQO* */ + u16 type: 3; + u16 reserved0: 1; + + /* Flipped by HW to notify the descriptor is populated. */ + u16 generation: 1; + union { + /* For descriptor completions, this is the last index fetched + * by HW + 1. + */ + __le16 tx_head; + + /* For packet completions, this is the completion tag set on the + * TX packet descriptors. + */ + __le16 completion_tag; + }; + __le32 reserved1; +} __packed; +GVE_CHECK_STRUCT_LEN(8, gve_tx_compl_desc); + +#define GVE_COMPL_TYPE_DQO_PKT 0x2 /* Packet completion */ +#define GVE_COMPL_TYPE_DQO_DESC 0x4 /* Descriptor completion */ +#define GVE_COMPL_TYPE_DQO_MISS 0x1 /* Miss path completion */ +#define GVE_COMPL_TYPE_DQO_REINJECTION 0x3 /* Re-injection completion */ + +/* Descriptor to post buffers to HW on buffer queue. */ +struct gve_rx_desc_dqo { + __le16 buf_id; /* ID returned in Rx completion descriptor */ + __le16 reserved0; + __le32 reserved1; + __le64 buf_addr; /* DMA address of the buffer */ + __le64 header_buf_addr; + __le64 reserved2; +} __packed; +GVE_CHECK_STRUCT_LEN(32, gve_rx_desc_dqo); + +/* Descriptor for HW to notify SW of new packets received on RX queue. */ +struct gve_rx_compl_desc_dqo { + /* Must be 1 */ + u8 rxdid: 4; + u8 reserved0: 4; + + /* Packet originated from this system rather than the network. */ + u8 loopback: 1; + /* Set when IPv6 packet contains a destination options header or routing + * header. + */ + u8 ipv6_ex_add: 1; + /* Invalid packet was received. */ + u8 rx_error: 1; + u8 reserved1: 5; + + u16 packet_type: 10; + u16 ip_hdr_err: 1; + u16 udp_len_err: 1; + u16 raw_cs_invalid: 1; + u16 reserved2: 3; + + u16 packet_len: 14; + /* Flipped by HW to notify the descriptor is populated. */ + u16 generation: 1; + /* Should be zero. */ + u16 buffer_queue_id: 1; + + u16 header_len: 10; + u16 rsc: 1; + u16 split_header: 1; + u16 reserved3: 4; + + u8 descriptor_done: 1; + u8 end_of_packet: 1; + u8 header_buffer_overflow: 1; + u8 l3_l4_processed: 1; + u8 csum_ip_err: 1; + u8 csum_l4_err: 1; + u8 csum_external_ip_err: 1; + u8 csum_external_udp_err: 1; + + u8 status_error1; + + __le16 reserved5; + __le16 buf_id; /* Buffer ID which was sent on the buffer queue. */ + + union { + /* Packet checksum. */ + __le16 raw_cs; + /* Segment length for RSC packets. */ + __le16 rsc_seg_len; + }; + __le32 hash; + __le32 reserved6; + __le64 reserved7; +} __packed; + +GVE_CHECK_STRUCT_LEN(32, gve_rx_compl_desc_dqo); + +/* Ringing the doorbell too often can hurt performance. + * + * HW requires this value to be at least 8. + */ +#define GVE_RX_BUF_THRESH_DQO 32 + +#endif /* _GVE_DESC_DQO_H_ */ diff --git a/drivers/net/gve/gve_register.h b/drivers/net/gve/gve_register.h new file mode 100644 index 0000000000..b65f336be2 --- /dev/null +++ b/drivers/net/gve/gve_register.h @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: MIT + * Google Virtual Ethernet (gve) driver + * Version: 1.3.0 + * Copyright (C) 2015-2022 Google, Inc. + */ + +#ifndef _GVE_REGISTER_H_ +#define _GVE_REGISTER_H_ + +/* Fixed Configuration Registers */ +struct gve_registers { + __be32 device_status; + __be32 driver_status; + __be32 max_tx_queues; + __be32 max_rx_queues; + __be32 adminq_pfn; + __be32 adminq_doorbell; + __be32 adminq_event_counter; + u8 reserved[3]; + u8 driver_version; +}; + +enum gve_device_status_flags { + GVE_DEVICE_STATUS_RESET_MASK = BIT(1), + GVE_DEVICE_STATUS_LINK_STATUS_MASK = BIT(2), + GVE_DEVICE_STATUS_REPORT_STATS_MASK = BIT(3), +}; +#endif /* _GVE_REGISTER_H_ */ From patchwork Fri Jul 29 19:30:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xiaoyun" X-Patchwork-Id: 114452 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 09392A00C4; Fri, 29 Jul 2022 21:31:15 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2BB0F42C51; Fri, 29 Jul 2022 21:31:08 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by mails.dpdk.org (Postfix) with ESMTP id 7458842C56 for ; Fri, 29 Jul 2022 21:31:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1659123065; x=1690659065; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=cvuZOR8YOS3SYn+udmLI3QGK9kkz1X3Lr9y7o5mzcQI=; b=ZWA9mK2xKZmhIstU0ATZTJ7AHDRjMQe20kxNK5OEoupTnSTWS54NL077 U6asjkqKmyHYu+2+ynZyaY7/THY8VaFydGH8NEyQNWFKDONx3Ot32KDuu iEFclj1+NTr9Hh6aLT7xrpcm/vcedIjljygr3S2hr4aQi2PlfRql8Rf6m V2/E2Z50kMnI/kLyJb145mIKu6lL+DgX/BanA4Auf6AjFmzu9i4OLDtAY Op+xvmrIK6qSdynkhZRPv2XD9pGCuh1U/a4Pumwg1ukeAhdnseiW4jdOL uaqIBt9ARyyQzQoa/JekZj1yMIDgrRdRqEF4OSREeXD2eIPJe7jZcGDGz Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10423"; a="268602912" X-IronPort-AV: E=Sophos;i="5.93,201,1654585200"; d="scan'208";a="268602912" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jul 2022 12:31:05 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,201,1654585200"; d="scan'208";a="577059531" Received: from silpixa00399779.ir.intel.com (HELO silpixa00399779.ger.corp.intel.com) ([10.237.223.111]) by orsmga006.jf.intel.com with ESMTP; 29 Jul 2022 12:31:03 -0700 From: Xiaoyun Li To: junfeng.guo@intel.com, qi.z.zhang@intel.com, awogbemila@google.com, bruce.richardson@intel.com Cc: dev@dpdk.org, Xiaoyun Li , Haiyue Wang Subject: [PATCH 02/10] net/gve: add logs and OS specific implementation Date: Fri, 29 Jul 2022 19:30:34 +0000 Message-Id: <20220729193042.2764633-3-xiaoyun.li@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220729193042.2764633-1-xiaoyun.li@intel.com> References: <20220729193042.2764633-1-xiaoyun.li@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add GVE PMD logs. Add some MACRO definitions and memory operations which are specific for DPDK. Signed-off-by: Haiyue Wang Signed-off-by: Xiaoyun Li --- drivers/net/gve/gve_adminq.h | 2 + drivers/net/gve/gve_desc.h | 2 + drivers/net/gve/gve_desc_dqo.h | 2 + drivers/net/gve/gve_logs.h | 22 +++++ drivers/net/gve/gve_osdep.h | 149 +++++++++++++++++++++++++++++++++ drivers/net/gve/gve_register.h | 2 + 6 files changed, 179 insertions(+) create mode 100644 drivers/net/gve/gve_logs.h create mode 100644 drivers/net/gve/gve_osdep.h diff --git a/drivers/net/gve/gve_adminq.h b/drivers/net/gve/gve_adminq.h index c7114cc883..cd496760ae 100644 --- a/drivers/net/gve/gve_adminq.h +++ b/drivers/net/gve/gve_adminq.h @@ -8,6 +8,8 @@ #ifndef _GVE_ADMINQ_H #define _GVE_ADMINQ_H +#include "gve_osdep.h" + /* Admin queue opcodes */ enum gve_adminq_opcodes { GVE_ADMINQ_DESCRIBE_DEVICE = 0x1, diff --git a/drivers/net/gve/gve_desc.h b/drivers/net/gve/gve_desc.h index b531669bc0..049792b43e 100644 --- a/drivers/net/gve/gve_desc.h +++ b/drivers/net/gve/gve_desc.h @@ -9,6 +9,8 @@ #ifndef _GVE_DESC_H_ #define _GVE_DESC_H_ +#include "gve_osdep.h" + /* A note on seg_addrs * * Base addresses encoded in seg_addr are not assumed to be physical diff --git a/drivers/net/gve/gve_desc_dqo.h b/drivers/net/gve/gve_desc_dqo.h index 0d533abcd1..5031752b43 100644 --- a/drivers/net/gve/gve_desc_dqo.h +++ b/drivers/net/gve/gve_desc_dqo.h @@ -9,6 +9,8 @@ #ifndef _GVE_DESC_DQO_H_ #define _GVE_DESC_DQO_H_ +#include "gve_osdep.h" + #define GVE_TX_MAX_HDR_SIZE_DQO 255 #define GVE_TX_MIN_TSO_MSS_DQO 88 diff --git a/drivers/net/gve/gve_logs.h b/drivers/net/gve/gve_logs.h new file mode 100644 index 0000000000..a050253f59 --- /dev/null +++ b/drivers/net/gve/gve_logs.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Intel Corporation + */ + +#ifndef _GVE_LOGS_H_ +#define _GVE_LOGS_H_ + +extern int gve_logtype_init; +extern int gve_logtype_driver; + +#define PMD_INIT_LOG(level, fmt, args...) \ + rte_log(RTE_LOG_ ## level, gve_logtype_init, "%s(): " fmt "\n", \ + __func__, ##args) + +#define PMD_DRV_LOG_RAW(level, fmt, args...) \ + rte_log(RTE_LOG_ ## level, gve_logtype_driver, "%s(): " fmt, \ + __func__, ## args) + +#define PMD_DRV_LOG(level, fmt, args...) \ + PMD_DRV_LOG_RAW(level, fmt "\n", ## args) + +#endif diff --git a/drivers/net/gve/gve_osdep.h b/drivers/net/gve/gve_osdep.h new file mode 100644 index 0000000000..92acccf846 --- /dev/null +++ b/drivers/net/gve/gve_osdep.h @@ -0,0 +1,149 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Intel Corporation + */ + +#ifndef _GVE_OSDEP_H_ +#define _GVE_OSDEP_H_ + +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "gve_logs.h" + +typedef uint8_t u8; +typedef uint16_t u16; +typedef uint32_t u32; +typedef uint64_t u64; + +typedef rte_be16_t __sum16; + +typedef rte_be16_t __be16; +typedef rte_be32_t __be32; +typedef rte_be64_t __be64; + +typedef rte_iova_t dma_addr_t; + +#define ETH_MIN_MTU RTE_ETHER_MIN_MTU +#define ETH_ALEN RTE_ETHER_ADDR_LEN +#define PAGE_SIZE 4096 + +#define BIT(nr) RTE_BIT32(nr) + +#define be16_to_cpu(x) rte_be_to_cpu_16(x) +#define be32_to_cpu(x) rte_be_to_cpu_32(x) +#define be64_to_cpu(x) rte_be_to_cpu_64(x) + +#define cpu_to_be16(x) rte_cpu_to_be_16(x) +#define cpu_to_be32(x) rte_cpu_to_be_32(x) +#define cpu_to_be64(x) rte_cpu_to_be_64(x) + +#define READ_ONCE32(x) rte_read32(&(x)) + +#define ____cacheline_aligned __rte_cache_aligned +#define __packed __rte_packed +#define __iomem + +#define msleep(ms) rte_delay_ms(ms) + +/* These macros are used to generate compilation errors if a struct/union + * is not exactly the correct length. It gives a divide by zero error if + * the struct/union is not of the correct size, otherwise it creates an + * enum that is never used. + */ +#define GVE_CHECK_STRUCT_LEN(n, X) enum gve_static_assert_enum_##X \ + { gve_static_assert_##X = (n) / ((sizeof(struct X) == (n)) ? 1 : 0) } +#define GVE_CHECK_UNION_LEN(n, X) enum gve_static_asset_enum_##X \ + { gve_static_assert_##X = (n) / ((sizeof(union X) == (n)) ? 1 : 0) } + +static __rte_always_inline u8 +readb(volatile void *addr) +{ + return rte_read8(addr); +} + +static __rte_always_inline void +writeb(u8 value, volatile void *addr) +{ + rte_write8(value, addr); +} + +static __rte_always_inline void +writel(u32 value, volatile void *addr) +{ + rte_write32(value, addr); +} + +static __rte_always_inline u32 +ioread32be(const volatile void *addr) +{ + return rte_be_to_cpu_32(rte_read32(addr)); +} + +static __rte_always_inline void +iowrite32be(u32 value, volatile void *addr) +{ + writel(rte_cpu_to_be_32(value), addr); +} + +/* DMA memory allocation tracking */ +struct gve_dma_mem { + void *va; + rte_iova_t pa; + uint32_t size; + const void *zone; +}; + +static inline void * +gve_alloc_dma_mem(struct gve_dma_mem *mem, u64 size) +{ + static uint16_t gve_dma_memzone_id; + const struct rte_memzone *mz = NULL; + char z_name[RTE_MEMZONE_NAMESIZE]; + + if (!mem) + return NULL; + + snprintf(z_name, sizeof(z_name), "gve_dma_%u", + __atomic_fetch_add(&gve_dma_memzone_id, 1, __ATOMIC_RELAXED)); + mz = rte_memzone_reserve_aligned(z_name, size, SOCKET_ID_ANY, + RTE_MEMZONE_IOVA_CONTIG, + PAGE_SIZE); + if (!mz) + return NULL; + + mem->size = size; + mem->va = mz->addr; + mem->pa = mz->iova; + mem->zone = mz; + PMD_DRV_LOG(DEBUG, "memzone %s is allocated", mz->name); + + return mem->va; +} + +static inline void +gve_free_dma_mem(struct gve_dma_mem *mem) +{ + PMD_DRV_LOG(DEBUG, "memzone %s to be freed", + ((const struct rte_memzone *)mem->zone)->name); + + rte_memzone_free(mem->zone); + mem->zone = NULL; + mem->va = NULL; + mem->pa = 0; +} + +#endif /* _GVE_OSDEP_H_ */ diff --git a/drivers/net/gve/gve_register.h b/drivers/net/gve/gve_register.h index b65f336be2..a599c1a08e 100644 --- a/drivers/net/gve/gve_register.h +++ b/drivers/net/gve/gve_register.h @@ -7,6 +7,8 @@ #ifndef _GVE_REGISTER_H_ #define _GVE_REGISTER_H_ +#include "gve_osdep.h" + /* Fixed Configuration Registers */ struct gve_registers { __be32 device_status; From patchwork Fri Jul 29 19:30:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xiaoyun" X-Patchwork-Id: 114453 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B7C00A00C4; Fri, 29 Jul 2022 21:31:21 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0E91B42C5B; Fri, 29 Jul 2022 21:31:10 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by mails.dpdk.org (Postfix) with ESMTP id B1E9F42C41 for ; Fri, 29 Jul 2022 21:31:07 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1659123067; x=1690659067; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=sSHYyj3vBod3wSi+G4VatTwrf0vQH+qajLVq7tkk4pI=; b=MWXcr1yuWgI3u5J6dBhG/Y2jVbvX2ga29S/2q88oQ3wvna9w+G9j6Sxg F4qnilEyJ6Z/6qdocC/Hbtgpkqkg5rK7YbySXL4pSuhW9isa3J/mE6v3p 9lMlVaaWwoy71jSR+9Xe80oO9Vcy85Oh1H2jtSa7hQi/M88mMmBq3ZwCT PscUVM1gFc0kkZKriReFKBdrMh4gDiLVr9l15NOB4r7JfcRRperG2KocP 99cE+w0XhZiw6Xg1xWXJ09TL9I32OGMBKViCnFW+q9Y880OTkms+K7ea7 ALWN+kYXhMfUP4AJqAX3AScee6ol+SakaFgkpnlOxht5PkHxjKGJro6lo w==; X-IronPort-AV: E=McAfee;i="6400,9594,10423"; a="268602917" X-IronPort-AV: E=Sophos;i="5.93,201,1654585200"; d="scan'208";a="268602917" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jul 2022 12:31:07 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,201,1654585200"; d="scan'208";a="577059536" Received: from silpixa00399779.ir.intel.com (HELO silpixa00399779.ger.corp.intel.com) ([10.237.223.111]) by orsmga006.jf.intel.com with ESMTP; 29 Jul 2022 12:31:05 -0700 From: Xiaoyun Li To: junfeng.guo@intel.com, qi.z.zhang@intel.com, awogbemila@google.com, bruce.richardson@intel.com Cc: dev@dpdk.org, Xiaoyun Li , Haiyue Wang Subject: [PATCH 03/10] net/gve: support device initialization Date: Fri, 29 Jul 2022 19:30:35 +0000 Message-Id: <20220729193042.2764633-4-xiaoyun.li@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220729193042.2764633-1-xiaoyun.li@intel.com> References: <20220729193042.2764633-1-xiaoyun.li@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Support device init and the fowllowing devops: - dev_configure - dev_start - dev_stop - dev_close Signed-off-by: Haiyue Wang Signed-off-by: Xiaoyun Li --- drivers/net/gve/gve.h | 249 +++++++++++++++++++++++ drivers/net/gve/gve_adminq.c | 1 + drivers/net/gve/gve_ethdev.c | 375 +++++++++++++++++++++++++++++++++++ drivers/net/gve/meson.build | 13 ++ drivers/net/gve/version.map | 3 + drivers/net/meson.build | 1 + 6 files changed, 642 insertions(+) create mode 100644 drivers/net/gve/gve.h create mode 100644 drivers/net/gve/gve_ethdev.c create mode 100644 drivers/net/gve/meson.build create mode 100644 drivers/net/gve/version.map diff --git a/drivers/net/gve/gve.h b/drivers/net/gve/gve.h new file mode 100644 index 0000000000..704c88983c --- /dev/null +++ b/drivers/net/gve/gve.h @@ -0,0 +1,249 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Intel Corporation + */ + +#ifndef _GVE_H_ +#define _GVE_H_ + +#include +#include +#include + +#include "gve_desc.h" + +#ifndef GOOGLE_VENDOR_ID +#define GOOGLE_VENDOR_ID 0x1ae0 +#endif + +#define GVE_DEV_ID 0x0042 + +#define GVE_REG_BAR 0 +#define GVE_DB_BAR 2 + +/* 1 for management, 1 for rx, 1 for tx */ +#define GVE_MIN_MSIX 3 + +/* PTYPEs are always 10 bits. */ +#define GVE_NUM_PTYPES 1024 + +/* A list of pages registered with the device during setup and used by a queue + * as buffers + */ +struct gve_queue_page_list { + uint32_t id; /* unique id */ + uint32_t num_entries; + dma_addr_t *page_buses; /* the dma addrs of the pages */ + const struct rte_memzone *mz; +}; + +/* A TX desc ring entry */ +union gve_tx_desc { + struct gve_tx_pkt_desc pkt; /* first desc for a packet */ + struct gve_tx_seg_desc seg; /* subsequent descs for a packet */ +}; + +struct gve_tx_queue { + volatile union gve_tx_desc *tx_desc_ring; + const struct rte_memzone *mz; + uint64_t tx_ring_phys_addr; + + uint16_t nb_tx_desc; + + /* Only valid for DQO_QPL queue format */ + struct gve_queue_page_list *qpl; + + uint16_t port_id; + uint16_t queue_id; + + uint16_t ntfy_id; + volatile rte_be32_t *ntfy_addr; + + struct gve_priv *hw; + const struct rte_memzone *qres_mz; + struct gve_queue_resources *qres; + + /* Only valid for DQO_RDA queue format */ + struct gve_tx_queue *complq; +}; + +struct gve_rx_queue { + volatile struct gve_rx_desc *rx_desc_ring; + volatile union gve_rx_data_slot *rx_data_ring; + const struct rte_memzone *mz; + const struct rte_memzone *data_mz; + uint64_t rx_ring_phys_addr; + + uint16_t nb_rx_desc; + + volatile rte_be32_t *ntfy_addr; + + /* only valid for GQI_QPL queue format */ + struct gve_queue_page_list *qpl; + + struct gve_priv *hw; + const struct rte_memzone *qres_mz; + struct gve_queue_resources *qres; + + uint16_t port_id; + uint16_t queue_id; + uint16_t ntfy_id; + uint16_t rx_buf_len; + + /* Only valid for DQO_RDA queue format */ + struct gve_rx_queue *bufq; +}; + +struct gve_irq_db { + rte_be32_t id; +} ____cacheline_aligned; + +struct gve_ptype { + uint8_t l3_type; /* `gve_l3_type` in gve_adminq.h */ + uint8_t l4_type; /* `gve_l4_type` in gve_adminq.h */ +}; + +struct gve_ptype_lut { + struct gve_ptype ptypes[GVE_NUM_PTYPES]; +}; + +enum gve_queue_format { + GVE_QUEUE_FORMAT_UNSPECIFIED = 0x0, /* default unspecified */ + GVE_GQI_RDA_FORMAT = 0x1, /* GQI Raw Addressing */ + GVE_GQI_QPL_FORMAT = 0x2, /* GQI Queue Page List */ + GVE_DQO_RDA_FORMAT = 0x3, /* DQO Raw Addressing */ +}; + +struct gve_priv { + struct gve_irq_db *irq_dbs; /* array of num_ntfy_blks */ + const struct rte_memzone *irq_dbs_mz; + uint32_t mgmt_msix_idx; + rte_be32_t *cnt_array; /* array of num_event_counters */ + const struct rte_memzone *cnt_array_mz; + + uint16_t num_event_counters; + uint16_t tx_desc_cnt; /* txq size */ + uint16_t rx_desc_cnt; /* rxq size */ + uint16_t tx_pages_per_qpl; /* tx buffer length */ + uint16_t rx_data_slot_cnt; /* rx buffer length */ + + /* Only valid for DQO_RDA queue format */ + uint16_t tx_compq_size; /* tx completion queue size */ + uint16_t rx_bufq_size; /* rx buff queue size */ + + uint64_t max_registered_pages; + uint64_t num_registered_pages; /* num pages registered with NIC */ + uint16_t default_num_queues; /* default num queues to set up */ + enum gve_queue_format queue_format; /* see enum gve_queue_format */ + uint8_t enable_lsc; + + uint16_t max_nb_txq; + uint16_t max_nb_rxq; + uint32_t num_ntfy_blks; /* spilt between TX and RX so must be even */ + + struct gve_registers __iomem *reg_bar0; /* see gve_register.h */ + rte_be32_t __iomem *db_bar2; /* "array" of doorbells */ + struct rte_pci_device *pci_dev; + + /* Admin queue - see gve_adminq.h*/ + union gve_adminq_command *adminq; + struct gve_dma_mem adminq_dma_mem; + uint32_t adminq_mask; /* masks prod_cnt to adminq size */ + uint32_t adminq_prod_cnt; /* free-running count of AQ cmds executed */ + uint32_t adminq_cmd_fail; /* free-running count of AQ cmds failed */ + uint32_t adminq_timeouts; /* free-running count of AQ cmds timeouts */ + /* free-running count of per AQ cmd executed */ + uint32_t adminq_describe_device_cnt; + uint32_t adminq_cfg_device_resources_cnt; + uint32_t adminq_register_page_list_cnt; + uint32_t adminq_unregister_page_list_cnt; + uint32_t adminq_create_tx_queue_cnt; + uint32_t adminq_create_rx_queue_cnt; + uint32_t adminq_destroy_tx_queue_cnt; + uint32_t adminq_destroy_rx_queue_cnt; + uint32_t adminq_dcfg_device_resources_cnt; + uint32_t adminq_set_driver_parameter_cnt; + uint32_t adminq_report_stats_cnt; + uint32_t adminq_report_link_speed_cnt; + uint32_t adminq_get_ptype_map_cnt; + + volatile uint32_t state_flags; + + /* Gvnic device link speed from hypervisor. */ + uint64_t link_speed; + + uint16_t max_mtu; + struct rte_ether_addr dev_addr; /* mac address */ + + struct gve_queue_page_list *qpl; + + struct gve_tx_queue **txqs; + struct gve_rx_queue **rxqs; +}; + +enum gve_state_flags_bit { + GVE_PRIV_FLAGS_ADMIN_QUEUE_OK = 1, + GVE_PRIV_FLAGS_DEVICE_RESOURCES_OK = 2, + GVE_PRIV_FLAGS_DEVICE_RINGS_OK = 3, + GVE_PRIV_FLAGS_NAPI_ENABLED = 4, +}; + +static inline bool gve_is_gqi(struct gve_priv *priv) +{ + return priv->queue_format == GVE_GQI_RDA_FORMAT || + priv->queue_format == GVE_GQI_QPL_FORMAT; +} + +static inline bool gve_get_admin_queue_ok(struct gve_priv *priv) +{ + return !!rte_bit_relaxed_get32(GVE_PRIV_FLAGS_ADMIN_QUEUE_OK, + &priv->state_flags); +} + +static inline void gve_set_admin_queue_ok(struct gve_priv *priv) +{ + rte_bit_relaxed_set32(GVE_PRIV_FLAGS_ADMIN_QUEUE_OK, + &priv->state_flags); +} + +static inline void gve_clear_admin_queue_ok(struct gve_priv *priv) +{ + rte_bit_relaxed_clear32(GVE_PRIV_FLAGS_ADMIN_QUEUE_OK, + &priv->state_flags); +} + +static inline bool gve_get_device_resources_ok(struct gve_priv *priv) +{ + return !!rte_bit_relaxed_get32(GVE_PRIV_FLAGS_DEVICE_RESOURCES_OK, + &priv->state_flags); +} + +static inline void gve_set_device_resources_ok(struct gve_priv *priv) +{ + rte_bit_relaxed_set32(GVE_PRIV_FLAGS_DEVICE_RESOURCES_OK, + &priv->state_flags); +} + +static inline void gve_clear_device_resources_ok(struct gve_priv *priv) +{ + rte_bit_relaxed_clear32(GVE_PRIV_FLAGS_DEVICE_RESOURCES_OK, + &priv->state_flags); +} + +static inline bool gve_get_device_rings_ok(struct gve_priv *priv) +{ + return !!rte_bit_relaxed_get32(GVE_PRIV_FLAGS_DEVICE_RINGS_OK, + &priv->state_flags); +} + +static inline void gve_set_device_rings_ok(struct gve_priv *priv) +{ + rte_bit_relaxed_set32(GVE_PRIV_FLAGS_DEVICE_RINGS_OK, + &priv->state_flags); +} + +static inline void gve_clear_device_rings_ok(struct gve_priv *priv) +{ + rte_bit_relaxed_clear32(GVE_PRIV_FLAGS_DEVICE_RINGS_OK, + &priv->state_flags); +} +#endif /* _GVE_H_ */ diff --git a/drivers/net/gve/gve_adminq.c b/drivers/net/gve/gve_adminq.c index 8a724f12c6..438ca2070e 100644 --- a/drivers/net/gve/gve_adminq.c +++ b/drivers/net/gve/gve_adminq.c @@ -5,6 +5,7 @@ * Copyright(C) 2022 Intel Corporation */ +#include "gve.h" #include "gve_adminq.h" #include "gve_register.h" diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c new file mode 100644 index 0000000000..f10f273f7d --- /dev/null +++ b/drivers/net/gve/gve_ethdev.c @@ -0,0 +1,375 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Intel Corporation + */ +#include + +#include "gve.h" +#include "gve_adminq.h" +#include "gve_register.h" + +#define GVE_VERSION "1.3.0" +#define GVE_VERSION_PREFIX "GVE-" + +const char gve_version_str[] = GVE_VERSION; +static const char gve_version_prefix[] = GVE_VERSION_PREFIX; + +static void +gve_write_version(uint8_t *driver_version_register) +{ + const char *c = gve_version_prefix; + + while (*c) { + writeb(*c, driver_version_register); + c++; + } + + c = gve_version_str; + while (*c) { + writeb(*c, driver_version_register); + c++; + } + writeb('\n', driver_version_register); +} + +static int +gve_dev_configure(__rte_unused struct rte_eth_dev *dev) +{ + return 0; +} + +static int +gve_dev_start(struct rte_eth_dev *dev) +{ + dev->data->dev_started = 1; + + return 0; +} + +static int +gve_dev_stop(struct rte_eth_dev *dev) +{ + dev->data->dev_link.link_status = RTE_ETH_LINK_DOWN; + dev->data->dev_started = 0; + + return 0; +} + +static int +gve_dev_close(struct rte_eth_dev *dev) +{ + int err = 0; + + if (dev->data->dev_started) { + err = gve_dev_stop(dev); + if (err != 0) + PMD_DRV_LOG(ERR, "Failed to stop dev."); + } + + return err; +} + +static const struct eth_dev_ops gve_eth_dev_ops = { + .dev_configure = gve_dev_configure, + .dev_start = gve_dev_start, + .dev_stop = gve_dev_stop, + .dev_close = gve_dev_close, +}; + +static void +gve_free_counter_array(struct gve_priv *priv) +{ + rte_memzone_free(priv->cnt_array_mz); + priv->cnt_array = NULL; +} + +static void +gve_free_irq_db(struct gve_priv *priv) +{ + rte_memzone_free(priv->irq_dbs_mz); + priv->irq_dbs = NULL; +} + +static void +gve_teardown_device_resources(struct gve_priv *priv) +{ + int err; + + /* Tell device its resources are being freed */ + if (gve_get_device_resources_ok(priv)) { + err = gve_adminq_deconfigure_device_resources(priv); + if (err) + PMD_DRV_LOG(ERR, "Could not deconfigure device resources: err=%d\n", err); + } + gve_free_counter_array(priv); + gve_free_irq_db(priv); + gve_clear_device_resources_ok(priv); +} + +static uint8_t +pci_dev_find_capability(struct rte_pci_device *pdev, int cap) +{ + uint8_t pos, id; + uint16_t ent; + int loops; + int ret; + + ret = rte_pci_read_config(pdev, &pos, sizeof(pos), PCI_CAPABILITY_LIST); + if (ret != sizeof(pos)) + return 0; + + loops = (PCI_CFG_SPACE_SIZE - PCI_STD_HEADER_SIZEOF) / PCI_CAP_SIZEOF; + + while (pos && loops--) { + ret = rte_pci_read_config(pdev, &ent, sizeof(ent), pos); + if (ret != sizeof(ent)) + return 0; + + id = ent & 0xff; + if (id == 0xff) + break; + + if (id == cap) + return pos; + + pos = (ent >> 8); + } + + return 0; +} + +static int +pci_dev_msix_vec_count(struct rte_pci_device *pdev) +{ + uint8_t msix_cap = pci_dev_find_capability(pdev, PCI_CAP_ID_MSIX); + uint16_t control; + int ret; + + if (!msix_cap) + return 0; + + ret = rte_pci_read_config(pdev, &control, sizeof(control), msix_cap + PCI_MSIX_FLAGS); + if (ret != sizeof(control)) + return 0; + + return (control & PCI_MSIX_FLAGS_QSIZE) + 1; +} + +static int +gve_setup_device_resources(struct gve_priv *priv) +{ + char z_name[RTE_MEMZONE_NAMESIZE]; + const struct rte_memzone *mz; + int err = 0; + + snprintf(z_name, sizeof(z_name), "gve_%s_cnt_arr", priv->pci_dev->device.name); + mz = rte_memzone_reserve_aligned(z_name, + priv->num_event_counters * sizeof(*priv->cnt_array), + rte_socket_id(), RTE_MEMZONE_IOVA_CONTIG, + PAGE_SIZE); + if (mz == NULL) { + PMD_INIT_LOG(ERR, "Could not alloc memzone for count array"); + return -ENOMEM; + } + priv->cnt_array = (rte_be32_t *)mz->addr; + priv->cnt_array_mz = mz; + + snprintf(z_name, sizeof(z_name), "gve_%s_irqmz", priv->pci_dev->device.name); + mz = rte_memzone_reserve_aligned(z_name, + sizeof(*priv->irq_dbs) * (priv->num_ntfy_blks), + rte_socket_id(), RTE_MEMZONE_IOVA_CONTIG, + PAGE_SIZE); + if (mz == NULL) { + PMD_INIT_LOG(ERR, "Could not alloc memzone for irq_dbs"); + err = -ENOMEM; + goto free_cnt_array; + } + priv->irq_dbs = (struct gve_irq_db *)mz->addr; + priv->irq_dbs_mz = mz; + + err = gve_adminq_configure_device_resources(priv, + priv->cnt_array_mz->iova, + priv->num_event_counters, + priv->irq_dbs_mz->iova, + priv->num_ntfy_blks); + if (unlikely(err)) { + PMD_INIT_LOG(ERR, "Could not config device resources: err=%d", err); + goto free_irq_dbs; + } + return 0; + +free_irq_dbs: + gve_free_irq_db(priv); +free_cnt_array: + gve_free_counter_array(priv); + + return err; +} + +static int +gve_init_priv(struct gve_priv *priv, bool skip_describe_device) +{ + int num_ntfy; + int err; + + /* Set up the adminq */ + err = gve_adminq_alloc(priv); + if (err) { + PMD_INIT_LOG(ERR, "Failed to alloc admin queue: err=%d", err); + return err; + } + + if (skip_describe_device) + goto setup_device; + + /* Get the initial information we need from the device */ + err = gve_adminq_describe_device(priv); + if (err) { + PMD_INIT_LOG(ERR, "Could not get device information: err=%d", err); + goto free_adminq; + } + + num_ntfy = pci_dev_msix_vec_count(priv->pci_dev); + if (num_ntfy <= 0) { + PMD_DRV_LOG(ERR, "Could not count MSI-x vectors"); + err = -EIO; + goto free_adminq; + } else if (num_ntfy < GVE_MIN_MSIX) { + PMD_DRV_LOG(ERR, "GVE needs at least %d MSI-x vectors, but only has %d", + GVE_MIN_MSIX, num_ntfy); + err = -EINVAL; + goto free_adminq; + } + + priv->num_registered_pages = 0; + + /* gvnic has one Notification Block per MSI-x vector, except for the + * management vector + */ + priv->num_ntfy_blks = (num_ntfy - 1) & ~0x1; + priv->mgmt_msix_idx = priv->num_ntfy_blks; + + priv->max_nb_txq = RTE_MIN(priv->max_nb_txq, priv->num_ntfy_blks / 2); + priv->max_nb_rxq = RTE_MIN(priv->max_nb_rxq, priv->num_ntfy_blks / 2); + + if (priv->default_num_queues > 0) { + priv->max_nb_txq = RTE_MIN(priv->default_num_queues, priv->max_nb_txq); + priv->max_nb_rxq = RTE_MIN(priv->default_num_queues, priv->max_nb_rxq); + } + + PMD_DRV_LOG(INFO, "Max TX queues %d, Max RX queues %d", + priv->max_nb_txq, priv->max_nb_rxq); + +setup_device: + err = gve_setup_device_resources(priv); + if (!err) + return 0; +free_adminq: + gve_adminq_free(priv); + return err; +} + +static void +gve_teardown_priv_resources(struct gve_priv *priv) +{ + gve_teardown_device_resources(priv); + gve_adminq_free(priv); +} + +static int +gve_dev_init(struct rte_eth_dev *eth_dev) +{ + struct gve_priv *priv = eth_dev->data->dev_private; + int max_tx_queues, max_rx_queues; + struct rte_pci_device *pci_dev; + struct gve_registers *reg_bar; + rte_be32_t *db_bar; + int err; + + eth_dev->dev_ops = &gve_eth_dev_ops; + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + pci_dev = RTE_DEV_TO_PCI(eth_dev->device); + + reg_bar = pci_dev->mem_resource[GVE_REG_BAR].addr; + if (!reg_bar) { + PMD_INIT_LOG(ERR, "Failed to map pci bar!\n"); + return -ENOMEM; + } + + db_bar = pci_dev->mem_resource[GVE_DB_BAR].addr; + if (!db_bar) { + PMD_INIT_LOG(ERR, "Failed to map doorbell bar!\n"); + return -ENOMEM; + } + + gve_write_version(®_bar->driver_version); + /* Get max queues to alloc etherdev */ + max_tx_queues = ioread32be(®_bar->max_tx_queues); + max_rx_queues = ioread32be(®_bar->max_rx_queues); + + priv->reg_bar0 = reg_bar; + priv->db_bar2 = db_bar; + priv->pci_dev = pci_dev; + priv->state_flags = 0x0; + + priv->max_nb_txq = max_tx_queues; + priv->max_nb_rxq = max_rx_queues; + + err = gve_init_priv(priv, false); + if (err) + return err; + + eth_dev->data->mac_addrs = rte_zmalloc("gve_mac", sizeof(struct rte_ether_addr), 0); + if (!eth_dev->data->mac_addrs) { + PMD_INIT_LOG(ERR, "Failed to allocate memory to store mac address"); + return -ENOMEM; + } + rte_ether_addr_copy(&priv->dev_addr, eth_dev->data->mac_addrs); + + return 0; +} + +static int +gve_dev_uninit(struct rte_eth_dev *eth_dev) +{ + struct gve_priv *priv = eth_dev->data->dev_private; + + eth_dev->data->mac_addrs = NULL; + + gve_teardown_priv_resources(priv); + + return 0; +} + +static int +gve_pci_probe(__rte_unused struct rte_pci_driver *pci_drv, + struct rte_pci_device *pci_dev) +{ + return rte_eth_dev_pci_generic_probe(pci_dev, sizeof(struct gve_priv), gve_dev_init); +} + +static int +gve_pci_remove(struct rte_pci_device *pci_dev) +{ + return rte_eth_dev_pci_generic_remove(pci_dev, gve_dev_uninit); +} + +static const struct rte_pci_id pci_id_gve_map[] = { + { RTE_PCI_DEVICE(GOOGLE_VENDOR_ID, GVE_DEV_ID) }, + { .device_id = 0 }, +}; + +static struct rte_pci_driver rte_gve_pmd = { + .id_table = pci_id_gve_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC, + .probe = gve_pci_probe, + .remove = gve_pci_remove, +}; + +RTE_PMD_REGISTER_PCI(net_gve, rte_gve_pmd); +RTE_PMD_REGISTER_PCI_TABLE(net_gve, pci_id_gve_map); +RTE_PMD_REGISTER_KMOD_DEP(net_gve, "* igb_uio | vfio-pci"); +RTE_LOG_REGISTER_SUFFIX(gve_logtype_init, init, NOTICE); +RTE_LOG_REGISTER_SUFFIX(gve_logtype_driver, driver, NOTICE); diff --git a/drivers/net/gve/meson.build b/drivers/net/gve/meson.build new file mode 100644 index 0000000000..9a22cc9abe --- /dev/null +++ b/drivers/net/gve/meson.build @@ -0,0 +1,13 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(C) 2022 Intel Corporation + +if is_windows + build = false + reason = 'not supported on Windows' + subdir_done() +endif + +sources = files( + 'gve_adminq.c', + 'gve_ethdev.c', +) diff --git a/drivers/net/gve/version.map b/drivers/net/gve/version.map new file mode 100644 index 0000000000..c2e0723b4c --- /dev/null +++ b/drivers/net/gve/version.map @@ -0,0 +1,3 @@ +DPDK_22 { + local: *; +}; diff --git a/drivers/net/meson.build b/drivers/net/meson.build index e35652fe63..f1a0ee2cef 100644 --- a/drivers/net/meson.build +++ b/drivers/net/meson.build @@ -23,6 +23,7 @@ drivers = [ 'enic', 'failsafe', 'fm10k', + 'gve', 'hinic', 'hns3', 'i40e', From patchwork Fri Jul 29 19:30:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xiaoyun" X-Patchwork-Id: 114454 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 43D74A00C4; Fri, 29 Jul 2022 21:31:29 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D482842C67; Fri, 29 Jul 2022 21:31:11 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by mails.dpdk.org (Postfix) with ESMTP id 5547042C5B for ; Fri, 29 Jul 2022 21:31:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1659123069; x=1690659069; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=O4EHdOKf8wZFxrl7YNWatdF754h4BTk/VetjvDTiMrE=; b=Mlu6SBHm0IGPyhs3q6skmpVcS+zQQTsy5rAs+IlYl4zr5SqZ8ILmRNwu Z866Rv0QLFLov26nzmu2bue79A/hBGLHX8AasQtwstnF3Nvxhi5karCcB +UeKzYDmrtWOMGwCY0Ivz88aNxR+6MzVQQ+A83e4HiRSWFWarwNKRUdjY rz7iJi0fRnnkt/hx7MicIfRcjKiQBPkLKG8jQq1D+S8JLwO/LCSlf6lqu zgKX+cWywAlCg3+k1XRndSlk31JHYXWTGBjgi5uyNcE/ieCh5Vvoc2Kbk uDZcq8+PCfYoVjt6k15GYRrOjRI+mUtHFJVViRJjNvlp/MsKpNVMuN4yZ A==; X-IronPort-AV: E=McAfee;i="6400,9594,10423"; a="268602924" X-IronPort-AV: E=Sophos;i="5.93,201,1654585200"; d="scan'208";a="268602924" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jul 2022 12:31:08 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,201,1654585200"; d="scan'208";a="577059542" Received: from silpixa00399779.ir.intel.com (HELO silpixa00399779.ger.corp.intel.com) ([10.237.223.111]) by orsmga006.jf.intel.com with ESMTP; 29 Jul 2022 12:31:07 -0700 From: Xiaoyun Li To: junfeng.guo@intel.com, qi.z.zhang@intel.com, awogbemila@google.com, bruce.richardson@intel.com Cc: dev@dpdk.org, Xiaoyun Li Subject: [PATCH 04/10] net/gve: add link update support Date: Fri, 29 Jul 2022 19:30:36 +0000 Message-Id: <20220729193042.2764633-5-xiaoyun.li@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220729193042.2764633-1-xiaoyun.li@intel.com> References: <20220729193042.2764633-1-xiaoyun.li@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Support dev_ops link_update. Signed-off-by: Xiaoyun Li --- drivers/net/gve/gve_ethdev.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c index f10f273f7d..435115c047 100644 --- a/drivers/net/gve/gve_ethdev.c +++ b/drivers/net/gve/gve_ethdev.c @@ -37,10 +37,39 @@ gve_dev_configure(__rte_unused struct rte_eth_dev *dev) return 0; } +static int +gve_link_update(struct rte_eth_dev *dev, __rte_unused int wait_to_complete) +{ + struct gve_priv *priv = dev->data->dev_private; + struct rte_eth_link link; + int err; + + memset(&link, 0, sizeof(link)); + link.link_duplex = RTE_ETH_LINK_FULL_DUPLEX; + link.link_autoneg = RTE_ETH_LINK_AUTONEG; + + if (!dev->data->dev_started) { + link.link_status = RTE_ETH_LINK_DOWN; + link.link_speed = RTE_ETH_SPEED_NUM_NONE; + } else { + link.link_status = RTE_ETH_LINK_UP; + PMD_INIT_LOG(DEBUG, "Get link status from hw"); + err = gve_adminq_report_link_speed(priv); + if (err) { + PMD_DRV_LOG(ERR, "Failed to get link speed."); + priv->link_speed = RTE_ETH_SPEED_NUM_UNKNOWN; + } + link.link_speed = priv->link_speed; + } + + return rte_eth_linkstatus_set(dev, &link); +} + static int gve_dev_start(struct rte_eth_dev *dev) { dev->data->dev_started = 1; + gve_link_update(dev, 0); return 0; } @@ -73,6 +102,7 @@ static const struct eth_dev_ops gve_eth_dev_ops = { .dev_start = gve_dev_start, .dev_stop = gve_dev_stop, .dev_close = gve_dev_close, + .link_update = gve_link_update, }; static void From patchwork Fri Jul 29 19:30:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xiaoyun" X-Patchwork-Id: 114455 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3C350A00C4; Fri, 29 Jul 2022 21:31:39 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0084342C65; Fri, 29 Jul 2022 21:31:14 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by mails.dpdk.org (Postfix) with ESMTP id 1B07942C65 for ; Fri, 29 Jul 2022 21:31:10 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1659123071; x=1690659071; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=HQbiMWB4WaxmYzhxYDsr4AAT4AV5M4VuMJC6UJh+72I=; b=B/nVX0edONY2H6nqF61EXJ+BtKQ3ZHHLvq8WKtuUIvNtJvsTcuqgoMwc cbUUKw0lKWTftcrx/piQmR1lkQoVslXb03WgWoX4K/jjqo1m2ia3qOxwc XzplJpy6g4Mmq0EPvI93rqIUK8B+wcRK5dsLZFUoigxXxiKA6MpJ/xMbn 05LuNxtbOTGbutOnXmv8Oa306q/NUqaypkQarlN9ytmig7OVOPf7TMJWe ppUZCqAvW6T15mil/YmQVuJzvSO2InqXjrHFHhh1Kw2Ul8q/gNkGDQ3Xl kfsQTjtO3vXJmZak+PMbVS7pAzeqjICzo5fOk3ZH9ZMjrKOZPc4jLjPIV w==; X-IronPort-AV: E=McAfee;i="6400,9594,10423"; a="268602933" X-IronPort-AV: E=Sophos;i="5.93,201,1654585200"; d="scan'208";a="268602933" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jul 2022 12:31:10 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,201,1654585200"; d="scan'208";a="577059552" Received: from silpixa00399779.ir.intel.com (HELO silpixa00399779.ger.corp.intel.com) ([10.237.223.111]) by orsmga006.jf.intel.com with ESMTP; 29 Jul 2022 12:31:09 -0700 From: Xiaoyun Li To: junfeng.guo@intel.com, qi.z.zhang@intel.com, awogbemila@google.com, bruce.richardson@intel.com Cc: dev@dpdk.org, Xiaoyun Li Subject: [PATCH 05/10] net/gve: add MTU set support Date: Fri, 29 Jul 2022 19:30:37 +0000 Message-Id: <20220729193042.2764633-6-xiaoyun.li@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220729193042.2764633-1-xiaoyun.li@intel.com> References: <20220729193042.2764633-1-xiaoyun.li@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Support dev_ops mtu_set. Signed-off-by: Xiaoyun Li --- drivers/net/gve/gve_ethdev.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c index 435115c047..26b45fde6f 100644 --- a/drivers/net/gve/gve_ethdev.c +++ b/drivers/net/gve/gve_ethdev.c @@ -97,12 +97,41 @@ gve_dev_close(struct rte_eth_dev *dev) return err; } +static int +gve_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu) +{ + struct gve_priv *priv = dev->data->dev_private; + int err; + + if (mtu < RTE_ETHER_MIN_MTU || mtu > priv->max_mtu) { + PMD_DRV_LOG(ERR, "MIN MTU is %u MAX MTU is %u", RTE_ETHER_MIN_MTU, priv->max_mtu); + return -EINVAL; + } + + /* mtu setting is forbidden if port is start */ + if (dev->data->dev_started) { + PMD_DRV_LOG(ERR, "Port must be stopped before configuration"); + return -EBUSY; + } + + dev->data->dev_conf.rxmode.mtu = mtu + RTE_ETHER_HDR_LEN; + + err = gve_adminq_set_mtu(priv, mtu); + if (err) { + PMD_DRV_LOG(ERR, "Failed to set mtu as %u err = %d", mtu, err); + return err; + } + + return 0; +} + static const struct eth_dev_ops gve_eth_dev_ops = { .dev_configure = gve_dev_configure, .dev_start = gve_dev_start, .dev_stop = gve_dev_stop, .dev_close = gve_dev_close, .link_update = gve_link_update, + .mtu_set = gve_dev_mtu_set, }; static void From patchwork Fri Jul 29 19:30:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xiaoyun" X-Patchwork-Id: 114456 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8272DA00C4; Fri, 29 Jul 2022 21:31:45 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DE00642C72; Fri, 29 Jul 2022 21:31:15 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by mails.dpdk.org (Postfix) with ESMTP id 5D7D442C59 for ; Fri, 29 Jul 2022 21:31:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1659123073; x=1690659073; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=q7tE98W7Xbl6oczvuAi6JlUYf19J726/Ki3Z4xsyWao=; b=Asbtc3ecX/T3kwVeYOu5UNfPfZrdS0pLY8OaoqnCgERkSz5+jsPkdGTY 1SqosqrqZHVgD8hEYzlBjfyv5mvfKQ1hn/oxmOkB9rkIVpcz/wrwmETzP 4h909C+x2lkjH7r0o6voxXEqzkYMJaqPpBDVTfGv216mLiOO5KEfzbL86 BBdo+9qMua3npCbO81x9nKgzbhAWkyRztAZGytF8EBehAQt+Fahwq1i0p w9bw+UG9X3kXhT541prS0TfjyB6xxlicOQz0LldtiQKhoG7bqiMbJBRUi JHN19yBPH5+yd66Uy9z8cmDGhcwdi0+XJ+Ia26HtM59JnqflPqmq5kahn w==; X-IronPort-AV: E=McAfee;i="6400,9594,10423"; a="268602948" X-IronPort-AV: E=Sophos;i="5.93,201,1654585200"; d="scan'208";a="268602948" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jul 2022 12:31:12 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,201,1654585200"; d="scan'208";a="577059563" Received: from silpixa00399779.ir.intel.com (HELO silpixa00399779.ger.corp.intel.com) ([10.237.223.111]) by orsmga006.jf.intel.com with ESMTP; 29 Jul 2022 12:31:10 -0700 From: Xiaoyun Li To: junfeng.guo@intel.com, qi.z.zhang@intel.com, awogbemila@google.com, bruce.richardson@intel.com Cc: dev@dpdk.org, Xiaoyun Li Subject: [PATCH 06/10] net/gve: add queue operations Date: Fri, 29 Jul 2022 19:30:38 +0000 Message-Id: <20220729193042.2764633-7-xiaoyun.li@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220729193042.2764633-1-xiaoyun.li@intel.com> References: <20220729193042.2764633-1-xiaoyun.li@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add support for queue operations: - setup rx/tx queue - release rx/tx queue - start rx/tx queues - stop rx/tx queues Signed-off-by: Xiaoyun Li --- drivers/net/gve/gve.h | 52 +++++++++ drivers/net/gve/gve_ethdev.c | 203 +++++++++++++++++++++++++++++++++ drivers/net/gve/gve_rx.c | 212 ++++++++++++++++++++++++++++++++++ drivers/net/gve/gve_tx.c | 214 +++++++++++++++++++++++++++++++++++ drivers/net/gve/meson.build | 2 + 5 files changed, 683 insertions(+) create mode 100644 drivers/net/gve/gve_rx.c create mode 100644 drivers/net/gve/gve_tx.c diff --git a/drivers/net/gve/gve.h b/drivers/net/gve/gve.h index 704c88983c..a53a852a5f 100644 --- a/drivers/net/gve/gve.h +++ b/drivers/net/gve/gve.h @@ -23,6 +23,9 @@ /* 1 for management, 1 for rx, 1 for tx */ #define GVE_MIN_MSIX 3 +#define GVE_DEFAULT_RX_FREE_THRESH 512 +#define GVE_DEFAULT_TX_FREE_THRESH 256 + /* PTYPEs are always 10 bits. */ #define GVE_NUM_PTYPES 1024 @@ -42,15 +45,35 @@ union gve_tx_desc { struct gve_tx_seg_desc seg; /* subsequent descs for a packet */ }; +struct gve_tx_iovec { + uint32_t iov_base; /* offset in fifo */ + uint32_t iov_len; +}; + struct gve_tx_queue { volatile union gve_tx_desc *tx_desc_ring; const struct rte_memzone *mz; uint64_t tx_ring_phys_addr; + struct rte_mbuf **sw_ring; + volatile rte_be32_t *qtx_tail; + volatile rte_be32_t *qtx_head; + uint32_t tx_tail; uint16_t nb_tx_desc; + uint16_t nb_free; + uint32_t next_to_clean; + uint16_t free_thresh; /* Only valid for DQO_QPL queue format */ + uint16_t sw_tail; + uint16_t sw_ntc; + uint16_t sw_nb_free; + uint32_t fifo_size; + uint32_t fifo_head; + uint32_t fifo_avail; + uint64_t fifo_base; struct gve_queue_page_list *qpl; + struct gve_tx_iovec *iov_ring; uint16_t port_id; uint16_t queue_id; @@ -64,6 +87,8 @@ struct gve_tx_queue { /* Only valid for DQO_RDA queue format */ struct gve_tx_queue *complq; + + uint8_t is_gqi_qpl; }; struct gve_rx_queue { @@ -72,9 +97,17 @@ struct gve_rx_queue { const struct rte_memzone *mz; const struct rte_memzone *data_mz; uint64_t rx_ring_phys_addr; + struct rte_mbuf **sw_ring; + struct rte_mempool *mpool; + uint16_t rx_tail; uint16_t nb_rx_desc; + uint16_t expected_seqno; /* the next expected seqno */ + uint16_t free_thresh; + uint32_t next_avail; + uint32_t nb_avail; + volatile rte_be32_t *qrx_tail; volatile rte_be32_t *ntfy_addr; /* only valid for GQI_QPL queue format */ @@ -91,6 +124,8 @@ struct gve_rx_queue { /* Only valid for DQO_RDA queue format */ struct gve_rx_queue *bufq; + + uint8_t is_gqi_qpl; }; struct gve_irq_db { @@ -246,4 +281,21 @@ static inline void gve_clear_device_rings_ok(struct gve_priv *priv) rte_bit_relaxed_clear32(GVE_PRIV_FLAGS_DEVICE_RINGS_OK, &priv->state_flags); } + +int +gve_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_id, uint16_t nb_desc, + unsigned int socket_id, const struct rte_eth_rxconf *conf, + struct rte_mempool *pool); +int +gve_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_id, uint16_t nb_desc, + unsigned int socket_id, const struct rte_eth_txconf *conf); + +void gve_tx_queue_release(void *txq); + +void gve_rx_queue_release(void *rxq); + +void gve_stop_tx_queues(struct rte_eth_dev *dev); + +void gve_stop_rx_queues(struct rte_eth_dev *dev); + #endif /* _GVE_H_ */ diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c index 26b45fde6f..5201398664 100644 --- a/drivers/net/gve/gve_ethdev.c +++ b/drivers/net/gve/gve_ethdev.c @@ -31,12 +31,111 @@ gve_write_version(uint8_t *driver_version_register) writeb('\n', driver_version_register); } +static int +gve_alloc_queue_page_list(struct gve_priv *priv, uint32_t id, uint32_t pages) +{ + char z_name[RTE_MEMZONE_NAMESIZE]; + struct gve_queue_page_list *qpl; + const struct rte_memzone *mz; + dma_addr_t page_bus; + uint32_t i; + + if (priv->num_registered_pages + pages > + priv->max_registered_pages) { + PMD_DRV_LOG(ERR, "Pages %" PRIu64 " > max registered pages %" PRIu64, + priv->num_registered_pages + pages, + priv->max_registered_pages); + return -EINVAL; + } + qpl = &priv->qpl[id]; + snprintf(z_name, sizeof(z_name), "gve_%s_qpl%d", priv->pci_dev->device.name, id); + mz = rte_memzone_reserve_aligned(z_name, pages * PAGE_SIZE, + rte_socket_id(), + RTE_MEMZONE_IOVA_CONTIG, PAGE_SIZE); + if (mz == NULL) { + PMD_DRV_LOG(ERR, "Failed to alloc %s.", z_name); + return -ENOMEM; + } + qpl->page_buses = rte_zmalloc("qpl page buses", pages * sizeof(dma_addr_t), 0); + if (qpl->page_buses == NULL) { + PMD_DRV_LOG(ERR, "Failed to alloc qpl %u page buses", id); + return -ENOMEM; + } + page_bus = mz->iova; + for (i = 0; i < pages; i++) { + qpl->page_buses[i] = page_bus; + page_bus += PAGE_SIZE; + } + qpl->id = id; + qpl->mz = mz; + qpl->num_entries = pages; + + priv->num_registered_pages += pages; + + return 0; +} + +static void +gve_free_qpls(struct gve_priv *priv) +{ + uint16_t nb_txqs = priv->max_nb_txq; + uint16_t nb_rxqs = priv->max_nb_rxq; + uint32_t i; + + for (i = 0; i < nb_txqs + nb_rxqs; i++) { + if (priv->qpl[i].mz != NULL) + rte_memzone_free(priv->qpl[i].mz); + if (priv->qpl[i].page_buses != NULL) + rte_free(priv->qpl[i].page_buses); + } + + if (priv->qpl != NULL) + rte_free(priv->qpl); +} + static int gve_dev_configure(__rte_unused struct rte_eth_dev *dev) { return 0; } +static int +gve_refill_pages(struct gve_rx_queue *rxq) +{ + struct rte_mbuf *nmb; + uint16_t i; + int diag; + + diag = rte_pktmbuf_alloc_bulk(rxq->mpool, &rxq->sw_ring[0], rxq->nb_rx_desc); + if (diag < 0) { + for (i = 0; i < rxq->nb_rx_desc - 1; i++) { + nmb = rte_pktmbuf_alloc(rxq->mpool); + if (!nmb) + break; + rxq->sw_ring[i] = nmb; + } + if (i < rxq->nb_rx_desc - 1) + return -ENOMEM; + } + rxq->nb_avail = 0; + rxq->next_avail = rxq->nb_rx_desc - 1; + + for (i = 0; i < rxq->nb_rx_desc; i++) { + if (rxq->is_gqi_qpl) { + rxq->rx_data_ring[i].addr = rte_cpu_to_be_64(i * PAGE_SIZE); + } else { + if (i == rxq->nb_rx_desc - 1) + break; + nmb = rxq->sw_ring[i]; + rxq->rx_data_ring[i].addr = rte_cpu_to_be_64(rte_mbuf_data_iova(nmb)); + } + } + + rte_write32(rte_cpu_to_be_32(rxq->next_avail), rxq->qrx_tail); + + return 0; +} + static int gve_link_update(struct rte_eth_dev *dev, __rte_unused int wait_to_complete) { @@ -68,16 +167,70 @@ gve_link_update(struct rte_eth_dev *dev, __rte_unused int wait_to_complete) static int gve_dev_start(struct rte_eth_dev *dev) { + uint16_t num_queues = dev->data->nb_tx_queues; + struct gve_priv *priv = dev->data->dev_private; + struct gve_tx_queue *txq; + struct gve_rx_queue *rxq; + uint16_t i; + int err; + + priv->txqs = (struct gve_tx_queue **)dev->data->tx_queues; + err = gve_adminq_create_tx_queues(priv, num_queues); + if (err) { + PMD_DRV_LOG(ERR, "failed to create %u tx queues.", num_queues); + return err; + } + for (i = 0; i < num_queues; i++) { + txq = priv->txqs[i]; + txq->qtx_tail = + &priv->db_bar2[rte_be_to_cpu_32(txq->qres->db_index)]; + txq->qtx_head = + &priv->cnt_array[rte_be_to_cpu_32(txq->qres->counter_index)]; + + rte_write32(rte_cpu_to_be_32(GVE_IRQ_MASK), txq->ntfy_addr); + } + + num_queues = dev->data->nb_rx_queues; + priv->rxqs = (struct gve_rx_queue **)dev->data->rx_queues; + err = gve_adminq_create_rx_queues(priv, num_queues); + if (err) { + PMD_DRV_LOG(ERR, "failed to create %u rx queues.", num_queues); + goto err_tx; + } + for (i = 0; i < num_queues; i++) { + rxq = priv->rxqs[i]; + rxq->qrx_tail = + &priv->db_bar2[rte_be_to_cpu_32(rxq->qres->db_index)]; + + rte_write32(rte_cpu_to_be_32(GVE_IRQ_MASK), rxq->ntfy_addr); + + err = gve_refill_pages(rxq); + if (err) { + PMD_DRV_LOG(ERR, "Failed to refill for RX"); + goto err_rx; + } + } + dev->data->dev_started = 1; gve_link_update(dev, 0); return 0; + +err_rx: + gve_stop_rx_queues(dev); +err_tx: + gve_stop_tx_queues(dev); + return err; } static int gve_dev_stop(struct rte_eth_dev *dev) { dev->data->dev_link.link_status = RTE_ETH_LINK_DOWN; + + gve_stop_tx_queues(dev); + gve_stop_rx_queues(dev); + dev->data->dev_started = 0; return 0; @@ -86,7 +239,11 @@ gve_dev_stop(struct rte_eth_dev *dev) static int gve_dev_close(struct rte_eth_dev *dev) { + struct gve_priv *priv = dev->data->dev_private; + struct gve_tx_queue *txq; + struct gve_rx_queue *rxq; int err = 0; + uint16_t i; if (dev->data->dev_started) { err = gve_dev_stop(dev); @@ -94,6 +251,18 @@ gve_dev_close(struct rte_eth_dev *dev) PMD_DRV_LOG(ERR, "Failed to stop dev."); } + for (i = 0; i < dev->data->nb_tx_queues; i++) { + txq = dev->data->tx_queues[i]; + gve_tx_queue_release(txq); + } + + for (i = 0; i < dev->data->nb_rx_queues; i++) { + rxq = dev->data->rx_queues[i]; + gve_rx_queue_release(rxq); + } + + gve_free_qpls(priv); + return err; } @@ -130,6 +299,8 @@ static const struct eth_dev_ops gve_eth_dev_ops = { .dev_start = gve_dev_start, .dev_stop = gve_dev_stop, .dev_close = gve_dev_close, + .rx_queue_setup = gve_rx_queue_setup, + .tx_queue_setup = gve_tx_queue_setup, .link_update = gve_link_update, .mtu_set = gve_dev_mtu_set, }; @@ -267,7 +438,9 @@ gve_setup_device_resources(struct gve_priv *priv) static int gve_init_priv(struct gve_priv *priv, bool skip_describe_device) { + uint16_t pages; int num_ntfy; + uint32_t i; int err; /* Set up the adminq */ @@ -318,10 +491,40 @@ gve_init_priv(struct gve_priv *priv, bool skip_describe_device) PMD_DRV_LOG(INFO, "Max TX queues %d, Max RX queues %d", priv->max_nb_txq, priv->max_nb_rxq); + /* In GQI_QPL queue format: + * Allocate queue page lists according to max queue number + * tx qpl id should start from 0 while rx qpl id should start + * from priv->max_nb_txq + */ + if (priv->queue_format == GVE_GQI_QPL_FORMAT) { + priv->qpl = rte_zmalloc("gve_qpl", + (priv->max_nb_txq + priv->max_nb_rxq) * + sizeof(struct gve_queue_page_list), 0); + if (priv->qpl == NULL) { + PMD_DRV_LOG(ERR, "Failed to alloc qpl."); + err = -ENOMEM; + goto free_adminq; + } + + for (i = 0; i < priv->max_nb_txq + priv->max_nb_rxq; i++) { + if (i < priv->max_nb_txq) + pages = priv->tx_pages_per_qpl; + else + pages = priv->rx_data_slot_cnt; + err = gve_alloc_queue_page_list(priv, i, pages); + if (err != 0) { + PMD_DRV_LOG(ERR, "Failed to alloc qpl %u.", i); + goto err_qpl; + } + } + } + setup_device: err = gve_setup_device_resources(priv); if (!err) return 0; +err_qpl: + gve_free_qpls(priv); free_adminq: gve_adminq_free(priv); return err; diff --git a/drivers/net/gve/gve_rx.c b/drivers/net/gve/gve_rx.c new file mode 100644 index 0000000000..7298b4cc86 --- /dev/null +++ b/drivers/net/gve/gve_rx.c @@ -0,0 +1,212 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Intel Corporation + */ + +#include "gve.h" +#include "gve_adminq.h" + +static inline void +gve_reset_rxq(struct gve_rx_queue *rxq) +{ + struct rte_mbuf **sw_ring = rxq->sw_ring; + uint32_t size, i; + + if (rxq == NULL) { + PMD_DRV_LOG(ERR, "pointer to rxq is NULL"); + return; + } + + size = rxq->nb_rx_desc * sizeof(struct gve_rx_desc); + for (i = 0; i < size; i++) + ((volatile char *)rxq->rx_desc_ring)[i] = 0; + + size = rxq->nb_rx_desc * sizeof(union gve_rx_data_slot); + for (i = 0; i < size; i++) + ((volatile char *)rxq->rx_data_ring)[i] = 0; + + for (i = 0; i < rxq->nb_rx_desc; i++) + sw_ring[i] = NULL; + + rxq->rx_tail = 0; + rxq->next_avail = 0; + rxq->nb_avail = rxq->nb_rx_desc; + rxq->expected_seqno = 1; +} + +static inline void +gve_release_rxq_mbufs(struct gve_rx_queue *rxq) +{ + uint16_t i; + + for (i = 0; i < rxq->nb_rx_desc; i++) { + if (rxq->sw_ring[i]) { + rte_pktmbuf_free_seg(rxq->sw_ring[i]); + rxq->sw_ring[i] = NULL; + } + } + + rxq->nb_avail = rxq->nb_rx_desc; +} + +void +gve_rx_queue_release(void *rxq) +{ + struct gve_rx_queue *q = rxq; + + if (!q) + return; + + if (q->is_gqi_qpl) { + gve_adminq_unregister_page_list(q->hw, q->qpl->id); + q->qpl = NULL; + } + + gve_release_rxq_mbufs(q); + rte_free(q->sw_ring); + rte_memzone_free(q->data_mz); + rte_memzone_free(q->mz); + rte_memzone_free(q->qres_mz); + q->qres = NULL; + rte_free(q); +} + +int +gve_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_id, + uint16_t nb_desc, unsigned int socket_id, + const struct rte_eth_rxconf *conf, struct rte_mempool *pool) +{ + struct gve_priv *hw = dev->data->dev_private; + const struct rte_memzone *mz; + struct gve_rx_queue *rxq; + uint16_t free_thresh; + int err = 0; + + if (nb_desc != hw->rx_desc_cnt) { + PMD_INIT_LOG(WARNING, "gve doesn't support nb_desc config, use hw nb_desc %u.", + hw->rx_desc_cnt); + } + nb_desc = hw->rx_desc_cnt; + + /* Free memory if needed. */ + if (dev->data->rx_queues[queue_id]) { + gve_rx_queue_release(dev->data->rx_queues[queue_id]); + dev->data->rx_queues[queue_id] = NULL; + } + + /* Allocate the RX queue data structure. */ + rxq = rte_zmalloc_socket("gve rxq", + sizeof(struct gve_rx_queue), + RTE_CACHE_LINE_SIZE, + socket_id); + if (!rxq) { + PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue structure"); + err = -ENOMEM; + goto err_rxq; + } + + free_thresh = conf->rx_free_thresh ? conf->rx_free_thresh : GVE_DEFAULT_RX_FREE_THRESH; + if (free_thresh >= nb_desc) { + PMD_INIT_LOG(ERR, "rx_free_thresh (%u) must be less than nb_desc (%u) minus 3.", + free_thresh, rxq->nb_rx_desc); + err = -EINVAL; + goto err_rxq; + } + + rxq->nb_rx_desc = nb_desc; + rxq->free_thresh = free_thresh; + rxq->queue_id = queue_id; + rxq->port_id = dev->data->port_id; + rxq->ntfy_id = hw->num_ntfy_blks / 2 + queue_id; + rxq->is_gqi_qpl = hw->queue_format == GVE_GQI_QPL_FORMAT; + rxq->mpool = pool; + rxq->hw = hw; + rxq->ntfy_addr = &hw->db_bar2[rte_be_to_cpu_32(hw->irq_dbs[rxq->ntfy_id].id)]; + + rxq->rx_buf_len = rte_pktmbuf_data_room_size(rxq->mpool) - RTE_PKTMBUF_HEADROOM; + + /* Allocate software ring */ + rxq->sw_ring = rte_zmalloc_socket("gve rx sw ring", sizeof(struct rte_mbuf *) * nb_desc, + RTE_CACHE_LINE_SIZE, socket_id); + if (!rxq->sw_ring) { + PMD_INIT_LOG(ERR, "Failed to allocate memory for SW RX ring"); + err = -ENOMEM; + goto err_rxq; + } + + mz = rte_eth_dma_zone_reserve(dev, "rx_ring", queue_id, + nb_desc * sizeof(struct gve_rx_desc), + PAGE_SIZE, socket_id); + if (mz == NULL) { + PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX"); + err = -ENOMEM; + goto err_sw_ring; + } + rxq->rx_desc_ring = (struct gve_rx_desc *)mz->addr; + rxq->rx_ring_phys_addr = mz->iova; + rxq->mz = mz; + + mz = rte_eth_dma_zone_reserve(dev, "gve rx data ring", queue_id, + sizeof(union gve_rx_data_slot) * nb_desc, + PAGE_SIZE, socket_id); + if (mz == NULL) { + PMD_INIT_LOG(ERR, "Failed to allocate memory for RX data ring"); + err = -ENOMEM; + goto err_rx_ring; + } + rxq->rx_data_ring = (union gve_rx_data_slot *)mz->addr; + rxq->data_mz = mz; + if (rxq->is_gqi_qpl) { + rxq->qpl = &hw->qpl[rxq->ntfy_id]; + err = gve_adminq_register_page_list(hw, rxq->qpl); + if (err != 0) { + PMD_INIT_LOG(ERR, "Failed to register qpl %u", queue_id); + goto err_data_ring; + } + } + + mz = rte_eth_dma_zone_reserve(dev, "rxq_res", queue_id, + sizeof(struct gve_queue_resources), + PAGE_SIZE, socket_id); + if (mz == NULL) { + PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX resource"); + err = -ENOMEM; + goto err_data_ring; + } + rxq->qres = (struct gve_queue_resources *)mz->addr; + rxq->qres_mz = mz; + + gve_reset_rxq(rxq); + + dev->data->rx_queues[queue_id] = rxq; + + return 0; + +err_data_ring: + rte_memzone_free(rxq->data_mz); +err_rx_ring: + rte_memzone_free(rxq->mz); +err_sw_ring: + rte_free(rxq->sw_ring); +err_rxq: + rte_free(rxq); + return err; +} + +void +gve_stop_rx_queues(struct rte_eth_dev *dev) +{ + struct gve_priv *hw = dev->data->dev_private; + struct gve_rx_queue *rxq; + uint16_t i; + int err; + + err = gve_adminq_destroy_rx_queues(hw, dev->data->nb_rx_queues); + if (err != 0) + PMD_DRV_LOG(WARNING, "failed to destroy rxqs"); + + for (i = 0; i < dev->data->nb_rx_queues; i++) { + rxq = dev->data->rx_queues[i]; + gve_release_rxq_mbufs(rxq); + gve_reset_rxq(rxq); + } +} diff --git a/drivers/net/gve/gve_tx.c b/drivers/net/gve/gve_tx.c new file mode 100644 index 0000000000..947c9d1627 --- /dev/null +++ b/drivers/net/gve/gve_tx.c @@ -0,0 +1,214 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Intel Corporation + */ + +#include "gve.h" +#include "gve_adminq.h" + +static inline void +gve_reset_txq(struct gve_tx_queue *txq) +{ + struct rte_mbuf **sw_ring = txq->sw_ring; + uint32_t size, i; + + if (txq == NULL) { + PMD_DRV_LOG(ERR, "Pointer to txq is NULL"); + return; + } + + size = txq->nb_tx_desc * sizeof(union gve_tx_desc); + for (i = 0; i < size; i++) + ((volatile char *)txq->tx_desc_ring)[i] = 0; + + for (i = 0; i < txq->nb_tx_desc; i++) { + sw_ring[i] = NULL; + if (txq->is_gqi_qpl) { + txq->iov_ring[i].iov_base = 0; + txq->iov_ring[i].iov_len = 0; + } + } + + txq->tx_tail = 0; + txq->nb_free = txq->nb_tx_desc - 1; + txq->next_to_clean = 0; + + if (txq->is_gqi_qpl) { + txq->fifo_size = PAGE_SIZE * txq->hw->tx_pages_per_qpl; + txq->fifo_avail = txq->fifo_size; + txq->fifo_head = 0; + txq->fifo_base = (uint64_t)(txq->qpl->mz->addr); + + txq->sw_tail = 0; + txq->sw_nb_free = txq->nb_tx_desc - 1; + txq->sw_ntc = 0; + } +} + +static inline void +gve_release_txq_mbufs(struct gve_tx_queue *txq) +{ + uint16_t i; + + for (i = 0; i < txq->nb_tx_desc; i++) { + if (txq->sw_ring[i]) { + rte_pktmbuf_free_seg(txq->sw_ring[i]); + txq->sw_ring[i] = NULL; + } + } +} + +void +gve_tx_queue_release(void *txq) +{ + struct gve_tx_queue *q = txq; + + if (!q) + return; + + if (q->is_gqi_qpl) { + gve_adminq_unregister_page_list(q->hw, q->qpl->id); + rte_free(q->iov_ring); + q->qpl = NULL; + } + + gve_release_txq_mbufs(q); + rte_free(q->sw_ring); + rte_memzone_free(q->mz); + rte_memzone_free(q->qres_mz); + q->qres = NULL; + rte_free(q); +} + +int +gve_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_id, uint16_t nb_desc, + unsigned int socket_id, const struct rte_eth_txconf *conf) +{ + struct gve_priv *hw = dev->data->dev_private; + const struct rte_memzone *mz; + struct gve_tx_queue *txq; + uint16_t free_thresh; + int err = 0; + + if (nb_desc != hw->tx_desc_cnt) { + PMD_INIT_LOG(WARNING, "gve doesn't support nb_desc config, use hw nb_desc %u.", + hw->tx_desc_cnt); + } + nb_desc = hw->tx_desc_cnt; + + /* Free memory if needed. */ + if (dev->data->tx_queues[queue_id]) { + gve_tx_queue_release(dev->data->tx_queues[queue_id]); + dev->data->tx_queues[queue_id] = NULL; + } + + /* Allocate the TX queue data structure. */ + txq = rte_zmalloc_socket("gve txq", sizeof(struct gve_tx_queue), + RTE_CACHE_LINE_SIZE, socket_id); + if (!txq) { + PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure"); + err = -ENOMEM; + goto err_txq; + } + + free_thresh = conf->tx_free_thresh ? conf->tx_free_thresh : GVE_DEFAULT_TX_FREE_THRESH; + if (free_thresh >= nb_desc - 3) { + PMD_INIT_LOG(ERR, "tx_free_thresh (%u) must be less than nb_desc (%u) minus 3.", + free_thresh, txq->nb_tx_desc); + err = -EINVAL; + goto err_txq; + } + + txq->nb_tx_desc = nb_desc; + txq->free_thresh = free_thresh; + txq->queue_id = queue_id; + txq->port_id = dev->data->port_id; + txq->ntfy_id = queue_id; + txq->is_gqi_qpl = hw->queue_format == GVE_GQI_QPL_FORMAT; + txq->hw = hw; + txq->ntfy_addr = &hw->db_bar2[rte_be_to_cpu_32(hw->irq_dbs[txq->ntfy_id].id)]; + + /* Allocate software ring */ + txq->sw_ring = rte_zmalloc_socket("gve tx sw ring", + sizeof(struct rte_mbuf *) * nb_desc, + RTE_CACHE_LINE_SIZE, socket_id); + if (!txq->sw_ring) { + PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring"); + err = -ENOMEM; + goto err_txq; + } + + mz = rte_eth_dma_zone_reserve(dev, "tx_ring", queue_id, + nb_desc * sizeof(union gve_tx_desc), + PAGE_SIZE, socket_id); + if (mz == NULL) { + PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX"); + err = -ENOMEM; + goto err_sw_ring; + } + txq->tx_desc_ring = (union gve_tx_desc *)mz->addr; + txq->tx_ring_phys_addr = mz->iova; + txq->mz = mz; + + if (txq->is_gqi_qpl) { + txq->iov_ring = rte_zmalloc_socket("gve tx iov ring", + sizeof(struct gve_tx_iovec) * nb_desc, + RTE_CACHE_LINE_SIZE, socket_id); + if (!txq->iov_ring) { + PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring"); + err = -ENOMEM; + goto err_tx_ring; + } + txq->qpl = &hw->qpl[queue_id]; + err = gve_adminq_register_page_list(hw, txq->qpl); + if (err != 0) { + PMD_INIT_LOG(ERR, "Failed to register qpl %u", queue_id); + goto err_iov_ring; + } + } + + mz = rte_eth_dma_zone_reserve(dev, "txq_res", queue_id, sizeof(struct gve_queue_resources), + PAGE_SIZE, socket_id); + if (mz == NULL) { + PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX resource"); + err = -ENOMEM; + goto err_iov_ring; + } + txq->qres = (struct gve_queue_resources *)mz->addr; + txq->qres_mz = mz; + + gve_reset_txq(txq); + + dev->data->tx_queues[queue_id] = txq; + + return 0; + +err_iov_ring: + if (txq->is_gqi_qpl) + rte_free(txq->iov_ring); +err_tx_ring: + rte_memzone_free(txq->mz); +err_sw_ring: + rte_free(txq->sw_ring); +err_txq: + rte_free(txq); + return err; +} + +void +gve_stop_tx_queues(struct rte_eth_dev *dev) +{ + struct gve_priv *hw = dev->data->dev_private; + struct gve_tx_queue *txq; + uint16_t i; + int err; + + err = gve_adminq_destroy_tx_queues(hw, dev->data->nb_tx_queues); + if (err != 0) + PMD_DRV_LOG(WARNING, "failed to destroy txqs"); + + for (i = 0; i < dev->data->nb_tx_queues; i++) { + txq = dev->data->tx_queues[i]; + gve_release_txq_mbufs(txq); + gve_reset_txq(txq); + } +} diff --git a/drivers/net/gve/meson.build b/drivers/net/gve/meson.build index 9a22cc9abe..c4fd013ef2 100644 --- a/drivers/net/gve/meson.build +++ b/drivers/net/gve/meson.build @@ -9,5 +9,7 @@ endif sources = files( 'gve_adminq.c', + 'gve_rx.c', + 'gve_tx.c', 'gve_ethdev.c', ) From patchwork Fri Jul 29 19:30:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xiaoyun" X-Patchwork-Id: 114457 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B0A0FA00C4; Fri, 29 Jul 2022 21:31:52 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CEC0342C75; Fri, 29 Jul 2022 21:31:17 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by mails.dpdk.org (Postfix) with ESMTP id 641D842C71 for ; Fri, 29 Jul 2022 21:31:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1659123075; x=1690659075; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=h1cDiXzdjfMwxgtfIHm8yIv8qZvmkGKCFe8bzDo/A0k=; b=mfSIlIZjy13mawxiK9OeGApXx6u/SqyI/96K4QoSzPxaVAftFdYG0ckY EwguNdvmd27hVXzwzauPNwHFHdHrVKydIXZs1Q4w6/SmrLZxGP6PC25qf Tp30F8tXjidVJxhUWtaerMdbtYFfbPJEhUoByF/KjyW53SN6yqf0AOsjp IL0KOb9T/v39ry5h5I9xWPxZ40KP7g0kSOZFw0Rg41D19UmL3DgStLyrU 2bYLUZLsYTMAvvwY9pHdnKBhWHBNsKF+qU5quJNnq2+KJP1VbDKDTLMLw +kT3pii//TFqVbriDKZnGNbg7TR7444dQSP511jg5anC3WvKHheP7Ogyv g==; X-IronPort-AV: E=McAfee;i="6400,9594,10423"; a="268602960" X-IronPort-AV: E=Sophos;i="5.93,201,1654585200"; d="scan'208";a="268602960" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jul 2022 12:31:14 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,201,1654585200"; d="scan'208";a="577059571" Received: from silpixa00399779.ir.intel.com (HELO silpixa00399779.ger.corp.intel.com) ([10.237.223.111]) by orsmga006.jf.intel.com with ESMTP; 29 Jul 2022 12:31:13 -0700 From: Xiaoyun Li To: junfeng.guo@intel.com, qi.z.zhang@intel.com, awogbemila@google.com, bruce.richardson@intel.com Cc: dev@dpdk.org, Xiaoyun Li Subject: [PATCH 07/10] net/gve: add Rx/Tx support Date: Fri, 29 Jul 2022 19:30:39 +0000 Message-Id: <20220729193042.2764633-8-xiaoyun.li@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220729193042.2764633-1-xiaoyun.li@intel.com> References: <20220729193042.2764633-1-xiaoyun.li@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add Rx/Tx of GQI_QPL queue format and GQI_RDA queue format. Signed-off-by: Xiaoyun Li --- drivers/net/gve/gve.h | 17 ++ drivers/net/gve/gve_ethdev.c | 5 + drivers/net/gve/gve_rx.c | 143 +++++++++++ drivers/net/gve/gve_tx.c | 452 +++++++++++++++++++++++++++++++++++ 4 files changed, 617 insertions(+) diff --git a/drivers/net/gve/gve.h b/drivers/net/gve/gve.h index a53a852a5f..7f4d0e37f3 100644 --- a/drivers/net/gve/gve.h +++ b/drivers/net/gve/gve.h @@ -25,6 +25,7 @@ #define GVE_DEFAULT_RX_FREE_THRESH 512 #define GVE_DEFAULT_TX_FREE_THRESH 256 +#define GVE_TX_MAX_FREE_SZ 512 /* PTYPEs are always 10 bits. */ #define GVE_NUM_PTYPES 1024 @@ -45,6 +46,18 @@ union gve_tx_desc { struct gve_tx_seg_desc seg; /* subsequent descs for a packet */ }; +/* Offload features */ +union gve_tx_offload { + uint64_t data; + struct { + uint64_t l2_len:7; /* L2 (MAC) Header Length. */ + uint64_t l3_len:9; /* L3 (IP) Header Length. */ + uint64_t l4_len:8; /* L4 Header Length. */ + uint64_t tso_segsz:16; /* TCP TSO segment size */ + /* uint64_t unused : 24; */ + }; +}; + struct gve_tx_iovec { uint32_t iov_base; /* offset in fifo */ uint32_t iov_len; @@ -298,4 +311,8 @@ void gve_stop_tx_queues(struct rte_eth_dev *dev); void gve_stop_rx_queues(struct rte_eth_dev *dev); +uint16_t gve_rx_burst(void *rxq, struct rte_mbuf **rx_pkts, uint16_t nb_pkts); + +uint16_t gve_tx_burst(void *txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts); + #endif /* _GVE_H_ */ diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c index 5201398664..5ebe2c30ea 100644 --- a/drivers/net/gve/gve_ethdev.c +++ b/drivers/net/gve/gve_ethdev.c @@ -583,6 +583,11 @@ gve_dev_init(struct rte_eth_dev *eth_dev) if (err) return err; + if (gve_is_gqi(priv)) { + eth_dev->rx_pkt_burst = gve_rx_burst; + eth_dev->tx_pkt_burst = gve_tx_burst; + } + eth_dev->data->mac_addrs = rte_zmalloc("gve_mac", sizeof(struct rte_ether_addr), 0); if (!eth_dev->data->mac_addrs) { PMD_INIT_LOG(ERR, "Failed to allocate memory to store mac address"); diff --git a/drivers/net/gve/gve_rx.c b/drivers/net/gve/gve_rx.c index 7298b4cc86..8f560ae592 100644 --- a/drivers/net/gve/gve_rx.c +++ b/drivers/net/gve/gve_rx.c @@ -5,6 +5,149 @@ #include "gve.h" #include "gve_adminq.h" +static inline void +gve_rx_refill(struct gve_rx_queue *rxq) +{ + uint16_t mask = rxq->nb_rx_desc - 1; + uint16_t idx = rxq->next_avail & mask; + uint32_t next_avail = rxq->next_avail; + uint16_t nb_alloc, i; + struct rte_mbuf *nmb; + int diag; + + /* wrap around */ + nb_alloc = rxq->nb_rx_desc - idx; + if (nb_alloc <= rxq->nb_avail) { + diag = rte_pktmbuf_alloc_bulk(rxq->mpool, &rxq->sw_ring[idx], nb_alloc); + if (diag < 0) { + for (i = 0; i < nb_alloc; i++) { + nmb = rte_pktmbuf_alloc(rxq->mpool); + if (!nmb) + break; + rxq->sw_ring[idx + i] = nmb; + } + if (i != nb_alloc) + nb_alloc = i; + } + rxq->nb_avail -= nb_alloc; + next_avail += nb_alloc; + + /* queue page list mode doesn't need real refill. */ + if (rxq->is_gqi_qpl) { + idx += nb_alloc; + } else { + for (i = 0; i < nb_alloc; i++) { + nmb = rxq->sw_ring[idx]; + rxq->rx_data_ring[idx].addr = + rte_cpu_to_be_64(rte_mbuf_data_iova(nmb)); + idx++; + } + } + if (idx == rxq->nb_rx_desc) + idx = 0; + } + + if (rxq->nb_avail > 0) { + nb_alloc = rxq->nb_avail; + if (rxq->nb_rx_desc < idx + rxq->nb_avail) + nb_alloc = rxq->nb_rx_desc - idx; + diag = rte_pktmbuf_alloc_bulk(rxq->mpool, &rxq->sw_ring[idx], nb_alloc); + if (diag < 0) { + for (i = 0; i < nb_alloc; i++) { + nmb = rte_pktmbuf_alloc(rxq->mpool); + if (!nmb) + break; + rxq->sw_ring[idx + i] = nmb; + } + nb_alloc = i; + } + rxq->nb_avail -= nb_alloc; + next_avail += nb_alloc; + + if (!rxq->is_gqi_qpl) { + for (i = 0; i < nb_alloc; i++) { + nmb = rxq->sw_ring[idx]; + rxq->rx_data_ring[idx].addr = + rte_cpu_to_be_64(rte_mbuf_data_iova(nmb)); + idx++; + } + } + } + + if (next_avail != rxq->next_avail) { + rte_write32(rte_cpu_to_be_32(next_avail), rxq->qrx_tail); + rxq->next_avail = next_avail; + } +} + +uint16_t +gve_rx_burst(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) +{ + volatile struct gve_rx_desc *rxr, *rxd; + struct gve_rx_queue *rxq = rx_queue; + uint16_t rx_id = rxq->rx_tail; + struct rte_mbuf *rxe; + uint16_t nb_rx, len; + uint64_t addr; + + rxr = rxq->rx_desc_ring; + + for (nb_rx = 0; nb_rx < nb_pkts; nb_rx++) { + rxd = &rxr[rx_id]; + if (GVE_SEQNO(rxd->flags_seq) != rxq->expected_seqno) + break; + + if (rxd->flags_seq & GVE_RXF_ERR) + continue; + + len = rte_be_to_cpu_16(rxd->len) - GVE_RX_PAD; + rxe = rxq->sw_ring[rx_id]; + rxe->data_off = RTE_PKTMBUF_HEADROOM; + if (rxq->is_gqi_qpl) { + addr = (uint64_t)(rxq->qpl->mz->addr) + rx_id * PAGE_SIZE + GVE_RX_PAD; + rte_memcpy((void *)((uint64_t)rxe->buf_addr + rxe->data_off), + (void *)addr, len); + } + rxe->nb_segs = 1; + rxe->next = NULL; + rxe->pkt_len = len; + rxe->data_len = len; + rxe->port = rxq->port_id; + rxe->packet_type = 0; + rxe->ol_flags = 0; + + if (rxd->flags_seq & GVE_RXF_TCP) + rxe->packet_type |= RTE_PTYPE_L4_TCP; + if (rxd->flags_seq & GVE_RXF_UDP) + rxe->packet_type |= RTE_PTYPE_L4_UDP; + if (rxd->flags_seq & GVE_RXF_IPV4) + rxe->packet_type |= RTE_PTYPE_L3_IPV4; + if (rxd->flags_seq & GVE_RXF_IPV6) + rxe->packet_type |= RTE_PTYPE_L3_IPV6; + + if (gve_needs_rss(rxd->flags_seq)) { + rxe->ol_flags |= RTE_MBUF_F_RX_RSS_HASH; + rxe->hash.rss = rte_be_to_cpu_32(rxd->rss_hash); + } + + rxq->expected_seqno = gve_next_seqno(rxq->expected_seqno); + + rx_id++; + if (rx_id == rxq->nb_rx_desc) + rx_id = 0; + + rx_pkts[nb_rx] = rxe; + } + + rxq->nb_avail += nb_rx; + rxq->rx_tail = rx_id; + + if (rxq->nb_avail > rxq->free_thresh) + gve_rx_refill(rxq); + + return nb_rx; +} + static inline void gve_reset_rxq(struct gve_rx_queue *rxq) { diff --git a/drivers/net/gve/gve_tx.c b/drivers/net/gve/gve_tx.c index 947c9d1627..2dc3411672 100644 --- a/drivers/net/gve/gve_tx.c +++ b/drivers/net/gve/gve_tx.c @@ -5,6 +5,458 @@ #include "gve.h" #include "gve_adminq.h" +static inline void +gve_free_bulk_mbuf(struct rte_mbuf **txep, int num) +{ + struct rte_mbuf *m, *free[GVE_TX_MAX_FREE_SZ]; + int nb_free = 0; + int i, s; + + if (unlikely(num == 0)) + return; + + /* Find the 1st mbuf which needs to be free */ + for (s = 0; s < num; s++) { + if (txep[s] != NULL) { + m = rte_pktmbuf_prefree_seg(txep[s]); + if (m != NULL) + break; + } + } + + if (s == num) + return; + + free[0] = m; + nb_free = 1; + for (i = s + 1; i < num; i++) { + if (likely(txep[i] != NULL)) { + m = rte_pktmbuf_prefree_seg(txep[i]); + if (likely(m != NULL)) { + if (likely(m->pool == free[0]->pool)) { + free[nb_free++] = m; + } else { + rte_mempool_put_bulk(free[0]->pool, (void *)free, nb_free); + free[0] = m; + nb_free = 1; + } + } + txep[i] = NULL; + } + } + rte_mempool_put_bulk(free[0]->pool, (void **)free, nb_free); +} + +static inline void +gve_tx_clean(struct gve_tx_queue *txq) +{ + uint16_t mask = txq->nb_tx_desc - 1; + uint32_t start = txq->next_to_clean & mask; + uint32_t ntc, nb_clean, i; + struct gve_tx_iovec *iov; + + ntc = rte_be_to_cpu_32(rte_read32(txq->qtx_head)); + ntc = ntc & mask; + + if (ntc == start) + return; + + /* if wrap around, free twice. */ + if (ntc < start) { + nb_clean = txq->nb_tx_desc - start; + if (nb_clean > GVE_TX_MAX_FREE_SZ) + nb_clean = GVE_TX_MAX_FREE_SZ; + if (txq->is_gqi_qpl) { + for (i = start; i < start + nb_clean; i++) { + iov = &txq->iov_ring[i]; + txq->fifo_avail += iov->iov_len; + iov->iov_base = 0; + iov->iov_len = 0; + } + } else { + gve_free_bulk_mbuf(&txq->sw_ring[start], nb_clean); + } + txq->nb_free += nb_clean; + start += nb_clean; + if (start == txq->nb_tx_desc) + start = 0; + txq->next_to_clean += nb_clean; + } + + if (ntc > start) { + nb_clean = ntc - start; + if (nb_clean > GVE_TX_MAX_FREE_SZ) + nb_clean = GVE_TX_MAX_FREE_SZ; + if (txq->is_gqi_qpl) { + for (i = start; i < start + nb_clean; i++) { + iov = &txq->iov_ring[i]; + txq->fifo_avail += iov->iov_len; + iov->iov_base = 0; + iov->iov_len = 0; + } + } else { + gve_free_bulk_mbuf(&txq->sw_ring[start], nb_clean); + } + txq->nb_free += nb_clean; + txq->next_to_clean += nb_clean; + } +} + +static inline void +gve_tx_clean_swr_qpl(struct gve_tx_queue *txq) +{ + uint32_t start = txq->sw_ntc; + uint32_t ntc, nb_clean; + + ntc = txq->sw_tail; + + if (ntc == start) + return; + + /* if wrap around, free twice. */ + if (ntc < start) { + nb_clean = txq->nb_tx_desc - start; + if (nb_clean > GVE_TX_MAX_FREE_SZ) + nb_clean = GVE_TX_MAX_FREE_SZ; + gve_free_bulk_mbuf(&txq->sw_ring[start], nb_clean); + + txq->sw_nb_free += nb_clean; + start += nb_clean; + if (start == txq->nb_tx_desc) + start = 0; + txq->sw_ntc = start; + } + + if (ntc > start) { + nb_clean = ntc - start; + if (nb_clean > GVE_TX_MAX_FREE_SZ) + nb_clean = GVE_TX_MAX_FREE_SZ; + gve_free_bulk_mbuf(&txq->sw_ring[start], nb_clean); + txq->sw_nb_free += nb_clean; + start += nb_clean; + txq->sw_ntc = start; + } +} + +static inline void +gve_tx_fill_pkt_desc(volatile union gve_tx_desc *desc, struct rte_mbuf *mbuf, + uint8_t desc_cnt, uint16_t len, uint64_t addr) +{ + uint64_t csum_l4 = mbuf->ol_flags & RTE_MBUF_F_TX_L4_MASK; + uint8_t l4_csum_offset = 0; + uint8_t l4_hdr_offset = 0; + + if (mbuf->ol_flags & RTE_MBUF_F_TX_TCP_SEG) + csum_l4 |= RTE_MBUF_F_TX_TCP_CKSUM; + + switch (csum_l4) { + case RTE_MBUF_F_TX_TCP_CKSUM: + l4_csum_offset = offsetof(struct rte_tcp_hdr, cksum); + l4_hdr_offset = mbuf->l2_len + mbuf->l3_len; + break; + case RTE_MBUF_F_TX_UDP_CKSUM: + l4_csum_offset = offsetof(struct rte_udp_hdr, dgram_cksum); + l4_hdr_offset = mbuf->l2_len + mbuf->l3_len; + break; + case RTE_MBUF_F_TX_SCTP_CKSUM: + l4_csum_offset = offsetof(struct rte_sctp_hdr, cksum); + l4_hdr_offset = mbuf->l2_len + mbuf->l3_len; + break; + } + + if (mbuf->ol_flags & RTE_MBUF_F_TX_TCP_SEG) { + desc->pkt.type_flags = GVE_TXD_TSO | GVE_TXF_L4CSUM; + desc->pkt.l4_csum_offset = l4_csum_offset >> 1; + desc->pkt.l4_hdr_offset = l4_hdr_offset >> 1; + } else if (mbuf->ol_flags & RTE_MBUF_F_TX_L4_MASK) { + desc->pkt.type_flags = GVE_TXD_STD | GVE_TXF_L4CSUM; + desc->pkt.l4_csum_offset = l4_csum_offset >> 1; + desc->pkt.l4_hdr_offset = l4_hdr_offset >> 1; + } else { + desc->pkt.type_flags = GVE_TXD_STD; + desc->pkt.l4_csum_offset = 0; + desc->pkt.l4_hdr_offset = 0; + } + desc->pkt.desc_cnt = desc_cnt; + desc->pkt.len = rte_cpu_to_be_16(mbuf->pkt_len); + desc->pkt.seg_len = rte_cpu_to_be_16(len); + desc->pkt.seg_addr = rte_cpu_to_be_64(addr); +} + +static inline void +gve_tx_fill_seg_desc(volatile union gve_tx_desc *desc, uint64_t ol_flags, + union gve_tx_offload tx_offload, + uint16_t len, uint64_t addr) +{ + desc->seg.type_flags = GVE_TXD_SEG; + if (ol_flags & RTE_MBUF_F_TX_TCP_SEG) { + if (ol_flags & RTE_MBUF_F_TX_IPV6) + desc->seg.type_flags |= GVE_TXSF_IPV6; + desc->seg.l3_offset = tx_offload.l2_len >> 1; + desc->seg.mss = rte_cpu_to_be_16(tx_offload.tso_segsz); + } + desc->seg.seg_len = rte_cpu_to_be_16(len); + desc->seg.seg_addr = rte_cpu_to_be_64(addr); +} + +static inline bool +is_fifo_avail(struct gve_tx_queue *txq, uint16_t len) +{ + if (txq->fifo_avail < len) + return false; + /* Don't split segment. */ + if (txq->fifo_head + len > txq->fifo_size && + txq->fifo_size - txq->fifo_head + len > txq->fifo_avail) + return false; + return true; +} +static inline uint64_t +gve_tx_alloc_from_fifo(struct gve_tx_queue *txq, uint16_t tx_id, uint16_t len) +{ + uint32_t head = txq->fifo_head; + uint32_t size = txq->fifo_size; + struct gve_tx_iovec *iov; + uint32_t aligned_head; + uint32_t iov_len = 0; + uint64_t fifo_addr; + + iov = &txq->iov_ring[tx_id]; + + /* Don't split segment */ + if (head + len > size) { + iov_len += (size - head); + head = 0; + } + + fifo_addr = head; + iov_len += len; + iov->iov_base = head; + + /* Re-align to a cacheline for next head */ + head += len; + aligned_head = RTE_ALIGN(head, RTE_CACHE_LINE_SIZE); + iov_len += (aligned_head - head); + iov->iov_len = iov_len; + + if (aligned_head == txq->fifo_size) + aligned_head = 0; + txq->fifo_head = aligned_head; + txq->fifo_avail -= iov_len; + + return fifo_addr; +} + +static inline uint16_t +gve_tx_burst_qpl(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) +{ + union gve_tx_offload tx_offload = {0}; + volatile union gve_tx_desc *txr, *txd; + struct gve_tx_queue *txq = tx_queue; + struct rte_mbuf **sw_ring = txq->sw_ring; + uint16_t mask = txq->nb_tx_desc - 1; + uint16_t tx_id = txq->tx_tail & mask; + uint64_t ol_flags, addr, fifo_addr; + uint32_t tx_tail = txq->tx_tail; + struct rte_mbuf *tx_pkt, *first; + uint16_t sw_id = txq->sw_tail; + uint16_t nb_used, i; + uint16_t nb_tx = 0; + uint32_t hlen; + + txr = txq->tx_desc_ring; + + if (txq->nb_free < txq->free_thresh || txq->fifo_avail == 0) + gve_tx_clean(txq); + + if (txq->sw_nb_free < txq->free_thresh) + gve_tx_clean_swr_qpl(txq); + + for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) { + tx_pkt = *tx_pkts++; + ol_flags = tx_pkt->ol_flags; + + if (txq->sw_nb_free < tx_pkt->nb_segs) { + gve_tx_clean_swr_qpl(txq); + if (txq->sw_nb_free < tx_pkt->nb_segs) + goto end_of_tx; + } + + /* Even for multi-segs, use 1 qpl buf for data */ + nb_used = 1; + if (ol_flags & RTE_MBUF_F_TX_TCP_SEG) + nb_used++; + + if (txq->nb_free < nb_used) + goto end_of_tx; + + tx_offload.l2_len = tx_pkt->l2_len; + tx_offload.l3_len = tx_pkt->l3_len; + tx_offload.l4_len = tx_pkt->l4_len; + tx_offload.tso_segsz = tx_pkt->tso_segsz; + + first = tx_pkt; + txd = &txr[tx_id]; + hlen = ol_flags & RTE_MBUF_F_TX_TCP_SEG ? + (uint32_t)(tx_offload.l2_len + tx_offload.l3_len + tx_offload.l4_len) : + tx_pkt->pkt_len; + + sw_ring[sw_id] = tx_pkt; + if (!is_fifo_avail(txq, hlen)) { + gve_tx_clean(txq); + if (!is_fifo_avail(txq, hlen)) + goto end_of_tx; + } + addr = (uint64_t)(tx_pkt->buf_addr) + tx_pkt->data_off; + fifo_addr = gve_tx_alloc_from_fifo(txq, tx_id, hlen); + + /* For TSO, check if there's enough fifo space for data first */ + if (ol_flags & RTE_MBUF_F_TX_TCP_SEG) { + if (!is_fifo_avail(txq, tx_pkt->pkt_len - hlen)) { + gve_tx_clean(txq); + if (!is_fifo_avail(txq, tx_pkt->pkt_len - hlen)) + goto end_of_tx; + } + } + if (tx_pkt->nb_segs == 1 || ol_flags & RTE_MBUF_F_TX_TCP_SEG) + rte_memcpy((void *)(fifo_addr + txq->fifo_base), (void *)addr, hlen); + else + rte_pktmbuf_read(tx_pkt, 0, hlen, (void *)(fifo_addr + txq->fifo_base)); + gve_tx_fill_pkt_desc(txd, tx_pkt, nb_used, hlen, fifo_addr); + + if (ol_flags & RTE_MBUF_F_TX_TCP_SEG) { + tx_id = (tx_id + 1) & mask; + txd = &txr[tx_id]; + addr = (uint64_t)(tx_pkt->buf_addr) + tx_pkt->data_off + hlen; + fifo_addr = gve_tx_alloc_from_fifo(txq, tx_id, tx_pkt->pkt_len - hlen); + if (tx_pkt->nb_segs == 1) + rte_memcpy((void *)(fifo_addr + txq->fifo_base), (void *)addr, + tx_pkt->pkt_len - hlen); + else + rte_pktmbuf_read(tx_pkt, hlen, tx_pkt->pkt_len - hlen, + (void *)(fifo_addr + txq->fifo_base)); + + gve_tx_fill_seg_desc(txd, ol_flags, tx_offload, + tx_pkt->pkt_len - hlen, fifo_addr); + } + + /* record mbuf in sw_ring for free */ + for (i = 1; i < first->nb_segs; i++) { + sw_id = (sw_id + 1) & mask; + tx_pkt = tx_pkt->next; + sw_ring[sw_id] = tx_pkt; + } + + sw_id = (sw_id + 1) & mask; + tx_id = (tx_id + 1) & mask; + + txq->nb_free -= nb_used; + txq->sw_nb_free -= first->nb_segs; + tx_tail += nb_used; + } + +end_of_tx: + if (nb_tx) { + rte_write32(rte_cpu_to_be_32(tx_tail), txq->qtx_tail); + txq->tx_tail = tx_tail; + txq->sw_tail = sw_id; + } + + return nb_tx; +} + +static inline uint16_t +gve_tx_burst_ra(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) +{ + union gve_tx_offload tx_offload = {0}; + volatile union gve_tx_desc *txr, *txd; + struct gve_tx_queue *txq = tx_queue; + struct rte_mbuf **sw_ring = txq->sw_ring; + uint16_t mask = txq->nb_tx_desc - 1; + uint16_t tx_id = txq->tx_tail & mask; + uint32_t tx_tail = txq->tx_tail; + struct rte_mbuf *tx_pkt, *first; + uint16_t nb_used, hlen, i; + uint64_t ol_flags, addr; + uint16_t nb_tx = 0; + + txr = txq->tx_desc_ring; + + if (txq->nb_free < txq->free_thresh) + gve_tx_clean(txq); + + for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) { + tx_pkt = *tx_pkts++; + ol_flags = tx_pkt->ol_flags; + + nb_used = tx_pkt->nb_segs; + if (ol_flags & RTE_MBUF_F_TX_TCP_SEG) + nb_used++; + + if (txq->nb_free < nb_used) + goto end_of_tx; + + tx_offload.l2_len = tx_pkt->l2_len; + tx_offload.l3_len = tx_pkt->l3_len; + tx_offload.l4_len = tx_pkt->l4_len; + tx_offload.tso_segsz = tx_pkt->tso_segsz; + + first = tx_pkt; + txd = &txr[tx_id]; + + hlen = ol_flags & RTE_MBUF_F_TX_TCP_SEG ? + (uint32_t)(tx_offload.l2_len + tx_offload.l3_len + tx_offload.l4_len) : + tx_pkt->pkt_len; + /* + * if tso, the driver needs to fill 2 descs for 1 mbuf + * so only put this mbuf into the 1st tx entry in sw ring + */ + sw_ring[tx_id] = tx_pkt; + addr = rte_mbuf_data_iova(tx_pkt); + gve_tx_fill_pkt_desc(txd, tx_pkt, nb_used, hlen, addr); + + if (ol_flags & RTE_MBUF_F_TX_TCP_SEG) { + tx_id = (tx_id + 1) & mask; + txd = &txr[tx_id]; + addr = rte_mbuf_data_iova(tx_pkt) + hlen; + gve_tx_fill_seg_desc(txd, ol_flags, tx_offload, + tx_pkt->data_len - hlen, addr); + } + + for (i = 1; i < first->nb_segs; i++) { + tx_id = (tx_id + 1) & mask; + txd = &txr[tx_id]; + tx_pkt = tx_pkt->next; + sw_ring[tx_id] = tx_pkt; + addr = rte_mbuf_data_iova(tx_pkt); + gve_tx_fill_seg_desc(txd, ol_flags, tx_offload, + tx_pkt->data_len, addr); + } + tx_id = (tx_id + 1) & mask; + + txq->nb_free -= nb_used; + tx_tail += nb_used; + } + +end_of_tx: + if (nb_tx) { + rte_write32(rte_cpu_to_be_32(tx_tail), txq->qtx_tail); + txq->tx_tail = tx_tail; + } + + return nb_tx; +} + +uint16_t +gve_tx_burst(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) +{ + struct gve_tx_queue *txq = tx_queue; + + if (txq->is_gqi_qpl) + return gve_tx_burst_qpl(tx_queue, tx_pkts, nb_pkts); + + return gve_tx_burst_ra(tx_queue, tx_pkts, nb_pkts); +} + static inline void gve_reset_txq(struct gve_tx_queue *txq) { From patchwork Fri Jul 29 19:30:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xiaoyun" X-Patchwork-Id: 114458 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D7C69A00C4; Fri, 29 Jul 2022 21:32:01 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 16D2742C80; Fri, 29 Jul 2022 21:31:19 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by mails.dpdk.org (Postfix) with ESMTP id 387C642C6D for ; Fri, 29 Jul 2022 21:31:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1659123077; x=1690659077; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ygJySkLLIsoclOGexHoGWrJk2VPWwtHVKnwuj4qsurw=; b=KEHG+8QSGf7z6uTo01lK+0s/LED2ZBcDywCOgvN8r9Yw3DFrMjnhOAyq HGiMFl9rSuE0pTvSvEfi8rAAFA/dgSqywZKifyVRljDnB22/65SDS7vo1 su4JyJU2kTDw2cWLpHZOU55rRJb7VYAG8v4wh9tK7WqDv3iDmSX9eLCHG 2NwwusJSErl6778w6MigX6Z+UHQD31dgBj74n8WnRR1g6ghU+I9jDfiMP bKxnrj8oNMcA9D3AKpVotlVfWDzjuW0p6JcecTcrBdHwbymUKHutm0AHM O1B7aKmAwRLsknmPP52rFUNCEUvHrM2Uj2nYgqOIGUKS5gheZnGy+NNI6 A==; X-IronPort-AV: E=McAfee;i="6400,9594,10423"; a="268602968" X-IronPort-AV: E=Sophos;i="5.93,201,1654585200"; d="scan'208";a="268602968" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jul 2022 12:31:16 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,201,1654585200"; d="scan'208";a="577059579" Received: from silpixa00399779.ir.intel.com (HELO silpixa00399779.ger.corp.intel.com) ([10.237.223.111]) by orsmga006.jf.intel.com with ESMTP; 29 Jul 2022 12:31:15 -0700 From: Xiaoyun Li To: junfeng.guo@intel.com, qi.z.zhang@intel.com, awogbemila@google.com, bruce.richardson@intel.com Cc: dev@dpdk.org, Xiaoyun Li Subject: [PATCH 08/10] net/gve: add support to get dev info and configure dev Date: Fri, 29 Jul 2022 19:30:40 +0000 Message-Id: <20220729193042.2764633-9-xiaoyun.li@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220729193042.2764633-1-xiaoyun.li@intel.com> References: <20220729193042.2764633-1-xiaoyun.li@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add dev_ops dev_infos_get. Complete dev_configure with RX offloads configuration. Signed-off-by: Xiaoyun Li --- drivers/net/gve/gve.h | 3 ++ drivers/net/gve/gve_ethdev.c | 61 ++++++++++++++++++++++++++++++++++++ 2 files changed, 64 insertions(+) diff --git a/drivers/net/gve/gve.h b/drivers/net/gve/gve.h index 7f4d0e37f3..004e0a75ca 100644 --- a/drivers/net/gve/gve.h +++ b/drivers/net/gve/gve.h @@ -27,6 +27,9 @@ #define GVE_DEFAULT_TX_FREE_THRESH 256 #define GVE_TX_MAX_FREE_SZ 512 +#define GVE_MIN_BUF_SIZE 1024 +#define GVE_MAX_RX_PKTLEN 65535 + /* PTYPEs are always 10 bits. */ #define GVE_NUM_PTYPES 1024 diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c index 5ebe2c30ea..6bc7bf4519 100644 --- a/drivers/net/gve/gve_ethdev.c +++ b/drivers/net/gve/gve_ethdev.c @@ -96,6 +96,14 @@ gve_free_qpls(struct gve_priv *priv) static int gve_dev_configure(__rte_unused struct rte_eth_dev *dev) { + struct gve_priv *priv = dev->data->dev_private; + + if (dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG) + dev->data->dev_conf.rxmode.offloads |= RTE_ETH_RX_OFFLOAD_RSS_HASH; + + if (dev->data->dev_conf.rxmode.offloads & RTE_ETH_RX_OFFLOAD_TCP_LRO) + priv->enable_lsc = 1; + return 0; } @@ -266,6 +274,58 @@ gve_dev_close(struct rte_eth_dev *dev) return err; } +static int +gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) +{ + struct gve_priv *priv = dev->data->dev_private; + + dev_info->device = dev->device; + dev_info->max_mac_addrs = 1; + dev_info->max_rx_queues = priv->max_nb_rxq; + dev_info->max_tx_queues = priv->max_nb_txq; + dev_info->min_rx_bufsize = GVE_MIN_BUF_SIZE; + dev_info->max_rx_pktlen = GVE_MAX_RX_PKTLEN; + + dev_info->rx_offload_capa = 0; + dev_info->tx_offload_capa = + RTE_ETH_TX_OFFLOAD_MULTI_SEGS | + RTE_ETH_TX_OFFLOAD_IPV4_CKSUM | + RTE_ETH_TX_OFFLOAD_UDP_CKSUM | + RTE_ETH_TX_OFFLOAD_TCP_CKSUM | + RTE_ETH_TX_OFFLOAD_SCTP_CKSUM | + RTE_ETH_TX_OFFLOAD_TCP_TSO; + + if (priv->queue_format == GVE_DQO_RDA_FORMAT) + dev_info->rx_offload_capa |= RTE_ETH_RX_OFFLOAD_TCP_LRO; + + dev_info->default_rxconf = (struct rte_eth_rxconf) { + .rx_free_thresh = GVE_DEFAULT_RX_FREE_THRESH, + .rx_drop_en = 0, + .offloads = 0, + }; + + dev_info->default_txconf = (struct rte_eth_txconf) { + .tx_free_thresh = GVE_DEFAULT_TX_FREE_THRESH, + .offloads = 0, + }; + + dev_info->default_rxportconf.ring_size = priv->rx_desc_cnt; + dev_info->rx_desc_lim = (struct rte_eth_desc_lim) { + .nb_max = priv->rx_desc_cnt, + .nb_min = priv->rx_desc_cnt, + .nb_align = 1, + }; + + dev_info->default_txportconf.ring_size = priv->tx_desc_cnt; + dev_info->tx_desc_lim = (struct rte_eth_desc_lim) { + .nb_max = priv->tx_desc_cnt, + .nb_min = priv->tx_desc_cnt, + .nb_align = 1, + }; + + return 0; +} + static int gve_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu) { @@ -299,6 +359,7 @@ static const struct eth_dev_ops gve_eth_dev_ops = { .dev_start = gve_dev_start, .dev_stop = gve_dev_stop, .dev_close = gve_dev_close, + .dev_infos_get = gve_dev_info_get, .rx_queue_setup = gve_rx_queue_setup, .tx_queue_setup = gve_tx_queue_setup, .link_update = gve_link_update, From patchwork Fri Jul 29 19:30:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xiaoyun" X-Patchwork-Id: 114459 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5723FA00C4; Fri, 29 Jul 2022 21:32:08 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E793242C63; Fri, 29 Jul 2022 21:31:21 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by mails.dpdk.org (Postfix) with ESMTP id 1F5DA42C82 for ; Fri, 29 Jul 2022 21:31:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1659123079; x=1690659079; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=3RDISPL1o5kz+ElcTc5DAWFZLKp2mKIlBw6Yo9Wfqzw=; b=bEz9U7D9+kdgervftmlJ2WC91EULOFO8GjZ90SRU30LLOPxNsilXIv5h n28vtA0JuC1Yo25/mMZQEp4AFS2F6HwMlIWOYmLa27YgugQ63wgVbv5OI wo0oC+syyZfNUx7FUpjUOArA6Vr2RONO0czF85OJ+geu7druDvJ5CMShr IWoSYmGHwTE48u6CKO7Au4HkFwo1wx6iAdm8ylIbi20btjv5HkYrejC83 blXyiSSHbqPnRlBA9jwQO/X/pgJiZKc5tqg6tAREukWx7O/X90QIopM1T l0qC7RF06kGaRocp5YmY7YTiy6BqFUWZ+JtvVr4cNnwCwEQ4YxyCHv9sj w==; X-IronPort-AV: E=McAfee;i="6400,9594,10423"; a="268602972" X-IronPort-AV: E=Sophos;i="5.93,201,1654585200"; d="scan'208";a="268602972" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jul 2022 12:31:18 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,201,1654585200"; d="scan'208";a="577059599" Received: from silpixa00399779.ir.intel.com (HELO silpixa00399779.ger.corp.intel.com) ([10.237.223.111]) by orsmga006.jf.intel.com with ESMTP; 29 Jul 2022 12:31:17 -0700 From: Xiaoyun Li To: junfeng.guo@intel.com, qi.z.zhang@intel.com, awogbemila@google.com, bruce.richardson@intel.com Cc: dev@dpdk.org, Xiaoyun Li Subject: [PATCH 09/10] net/gve: add stats support Date: Fri, 29 Jul 2022 19:30:41 +0000 Message-Id: <20220729193042.2764633-10-xiaoyun.li@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220729193042.2764633-1-xiaoyun.li@intel.com> References: <20220729193042.2764633-1-xiaoyun.li@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Update stats add support of dev_ops stats_get/reset. Signed-off-by: Xiaoyun Li --- drivers/net/gve/gve.h | 10 ++++++ drivers/net/gve/gve_ethdev.c | 69 ++++++++++++++++++++++++++++++++++++ drivers/net/gve/gve_rx.c | 15 ++++++-- drivers/net/gve/gve_tx.c | 12 +++++++ 4 files changed, 104 insertions(+), 2 deletions(-) diff --git a/drivers/net/gve/gve.h b/drivers/net/gve/gve.h index 004e0a75ca..e256a2bec2 100644 --- a/drivers/net/gve/gve.h +++ b/drivers/net/gve/gve.h @@ -91,6 +91,10 @@ struct gve_tx_queue { struct gve_queue_page_list *qpl; struct gve_tx_iovec *iov_ring; + /* Stats */ + uint64_t packets; + uint64_t bytes; + uint16_t port_id; uint16_t queue_id; @@ -129,6 +133,12 @@ struct gve_rx_queue { /* only valid for GQI_QPL queue format */ struct gve_queue_page_list *qpl; + /* stats */ + uint64_t no_mbufs; + uint64_t errors; + uint64_t packets; + uint64_t bytes; + struct gve_priv *hw; const struct rte_memzone *qres_mz; struct gve_queue_resources *qres; diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c index 6bc7bf4519..2977df01f1 100644 --- a/drivers/net/gve/gve_ethdev.c +++ b/drivers/net/gve/gve_ethdev.c @@ -326,6 +326,73 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) return 0; } +static int +gve_dev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats) +{ + uint16_t i; + + for (i = 0; i < dev->data->nb_tx_queues; i++) { + struct gve_tx_queue *txq = dev->data->tx_queues[i]; + if (txq == NULL) + continue; + + stats->opackets += txq->packets; + stats->obytes += txq->bytes; + + if (i < RTE_ETHDEV_QUEUE_STAT_CNTRS) { + stats->q_opackets[i] = txq->packets; + stats->q_obytes[i] = txq->bytes; + } + } + + for (i = 0; i < dev->data->nb_rx_queues; i++) { + struct gve_rx_queue *rxq = dev->data->rx_queues[i]; + if (rxq == NULL) + continue; + + stats->ipackets += rxq->packets; + stats->ibytes += rxq->bytes; + stats->ierrors += rxq->errors; + stats->rx_nombuf += rxq->no_mbufs; + + if (i < RTE_ETHDEV_QUEUE_STAT_CNTRS) { + stats->q_ipackets[i] = rxq->packets; + stats->q_ibytes[i] = rxq->bytes; + stats->q_errors[i] = rxq->errors; + } + } + + return 0; +} + +static int +gve_dev_stats_reset(struct rte_eth_dev *dev) +{ + uint16_t i; + + for (i = 0; i < dev->data->nb_tx_queues; i++) { + struct gve_tx_queue *txq = dev->data->tx_queues[i]; + if (txq == NULL) + continue; + + txq->packets = 0; + txq->bytes = 0; + } + + for (i = 0; i < dev->data->nb_rx_queues; i++) { + struct gve_rx_queue *rxq = dev->data->rx_queues[i]; + if (rxq == NULL) + continue; + + rxq->packets = 0; + rxq->bytes = 0; + rxq->no_mbufs = 0; + rxq->errors = 0; + } + + return 0; +} + static int gve_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu) { @@ -363,6 +430,8 @@ static const struct eth_dev_ops gve_eth_dev_ops = { .rx_queue_setup = gve_rx_queue_setup, .tx_queue_setup = gve_tx_queue_setup, .link_update = gve_link_update, + .stats_get = gve_dev_stats_get, + .stats_reset = gve_dev_stats_reset, .mtu_set = gve_dev_mtu_set, }; diff --git a/drivers/net/gve/gve_rx.c b/drivers/net/gve/gve_rx.c index 8f560ae592..3a8a869980 100644 --- a/drivers/net/gve/gve_rx.c +++ b/drivers/net/gve/gve_rx.c @@ -26,8 +26,10 @@ gve_rx_refill(struct gve_rx_queue *rxq) break; rxq->sw_ring[idx + i] = nmb; } - if (i != nb_alloc) + if (i != nb_alloc) { + rxq->no_mbufs += nb_alloc - i; nb_alloc = i; + } } rxq->nb_avail -= nb_alloc; next_avail += nb_alloc; @@ -88,6 +90,7 @@ gve_rx_burst(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) uint16_t rx_id = rxq->rx_tail; struct rte_mbuf *rxe; uint16_t nb_rx, len; + uint64_t bytes = 0; uint64_t addr; rxr = rxq->rx_desc_ring; @@ -97,8 +100,10 @@ gve_rx_burst(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) if (GVE_SEQNO(rxd->flags_seq) != rxq->expected_seqno) break; - if (rxd->flags_seq & GVE_RXF_ERR) + if (rxd->flags_seq & GVE_RXF_ERR) { + rxq->errors++; continue; + } len = rte_be_to_cpu_16(rxd->len) - GVE_RX_PAD; rxe = rxq->sw_ring[rx_id]; @@ -137,6 +142,7 @@ gve_rx_burst(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) rx_id = 0; rx_pkts[nb_rx] = rxe; + bytes += len; } rxq->nb_avail += nb_rx; @@ -145,6 +151,11 @@ gve_rx_burst(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) if (rxq->nb_avail > rxq->free_thresh) gve_rx_refill(rxq); + if (nb_rx) { + rxq->packets += nb_rx; + rxq->bytes += bytes; + } + return nb_rx; } diff --git a/drivers/net/gve/gve_tx.c b/drivers/net/gve/gve_tx.c index 2dc3411672..d99e6eb009 100644 --- a/drivers/net/gve/gve_tx.c +++ b/drivers/net/gve/gve_tx.c @@ -260,6 +260,7 @@ gve_tx_burst_qpl(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) struct rte_mbuf *tx_pkt, *first; uint16_t sw_id = txq->sw_tail; uint16_t nb_used, i; + uint64_t bytes = 0; uint16_t nb_tx = 0; uint32_t hlen; @@ -352,6 +353,8 @@ gve_tx_burst_qpl(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) txq->nb_free -= nb_used; txq->sw_nb_free -= first->nb_segs; tx_tail += nb_used; + + bytes += first->pkt_len; } end_of_tx: @@ -359,6 +362,9 @@ gve_tx_burst_qpl(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) rte_write32(rte_cpu_to_be_32(tx_tail), txq->qtx_tail); txq->tx_tail = tx_tail; txq->sw_tail = sw_id; + + txq->packets += nb_tx; + txq->bytes += bytes; } return nb_tx; @@ -377,6 +383,7 @@ gve_tx_burst_ra(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) struct rte_mbuf *tx_pkt, *first; uint16_t nb_used, hlen, i; uint64_t ol_flags, addr; + uint64_t bytes = 0; uint16_t nb_tx = 0; txr = txq->tx_desc_ring; @@ -435,12 +442,17 @@ gve_tx_burst_ra(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) txq->nb_free -= nb_used; tx_tail += nb_used; + + bytes += first->pkt_len; } end_of_tx: if (nb_tx) { rte_write32(rte_cpu_to_be_32(tx_tail), txq->qtx_tail); txq->tx_tail = tx_tail; + + txq->packets += nb_tx; + txq->bytes += bytes; } return nb_tx; From patchwork Fri Jul 29 19:30:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xiaoyun" X-Patchwork-Id: 114460 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A4D7AA00C4; Fri, 29 Jul 2022 21:32:14 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D1BC342C79; Fri, 29 Jul 2022 21:31:24 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by mails.dpdk.org (Postfix) with ESMTP id E753842C48 for ; Fri, 29 Jul 2022 21:31:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1659123081; x=1690659081; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=iQ6k7vwuSesPlv27XqT9acmhsWe9Gi5yiMRz28EdFiM=; b=XK6PBpCROPtjZZJavhhKXRlxeTfeMUaPbWq46PDSHVfvx5mO3liCcNVh 9Hzai+qxBZSEHY08J/qya/MX44Q/or78N79RCBfexvAt1YGWyg29cRKMx KrAD04nPXCmh8s06H8bRncUd4OLpuwtw9oI208kuCfZbSazJ2bgvV/iDI p8fg+/JS2lkc3hucy0iY0CFIHBplOYrADQOkruUXF6X/3syjGIB2We7x7 d9dxc0Lj8YVdJSqCMQvWgLWmFDUirKUHYySRbXTywcYzEwKi/UbKSW0Gz C4oiQT5bXkKuTS1lyRsO+ouxfJFlmJiddpZ/UUnB29Z4pIcgcfiX/jeoh w==; X-IronPort-AV: E=McAfee;i="6400,9594,10423"; a="268602976" X-IronPort-AV: E=Sophos;i="5.93,201,1654585200"; d="scan'208";a="268602976" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jul 2022 12:31:20 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,201,1654585200"; d="scan'208";a="577059607" Received: from silpixa00399779.ir.intel.com (HELO silpixa00399779.ger.corp.intel.com) ([10.237.223.111]) by orsmga006.jf.intel.com with ESMTP; 29 Jul 2022 12:31:18 -0700 From: Xiaoyun Li To: junfeng.guo@intel.com, qi.z.zhang@intel.com, awogbemila@google.com, bruce.richardson@intel.com Cc: dev@dpdk.org, Xiaoyun Li Subject: [PATCH 10/10] doc: update documentation Date: Fri, 29 Jul 2022 19:30:42 +0000 Message-Id: <20220729193042.2764633-11-xiaoyun.li@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220729193042.2764633-1-xiaoyun.li@intel.com> References: <20220729193042.2764633-1-xiaoyun.li@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Update documentation of GVE PMD and release note. Add Junfeng Guo as GVE PMD maintainer since he'll work on GVE PMD in the future and maintain it and I won't be available for maintaining. Signed-off-by: Xiaoyun Li --- MAINTAINERS | 6 +++ doc/guides/nics/features/gve.ini | 18 +++++++ doc/guides/nics/gve.rst | 65 ++++++++++++++++++++++++++ doc/guides/rel_notes/release_22_11.rst | 4 ++ 4 files changed, 93 insertions(+) create mode 100644 doc/guides/nics/features/gve.ini create mode 100644 doc/guides/nics/gve.rst diff --git a/MAINTAINERS b/MAINTAINERS index 32ffdd1a61..474f41f0de 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -697,6 +697,12 @@ F: drivers/net/enic/ F: doc/guides/nics/enic.rst F: doc/guides/nics/features/enic.ini +Google Virtual Ethernet +M: Junfeng Guo +F: drivers/net/gve/ +F: doc/guides/nics/gve.rst +F: doc/guides/nics/features/gve.ini + Hisilicon hns3 M: Dongdong Liu M: Yisen Zhuang diff --git a/doc/guides/nics/features/gve.ini b/doc/guides/nics/features/gve.ini new file mode 100644 index 0000000000..180408aa80 --- /dev/null +++ b/doc/guides/nics/features/gve.ini @@ -0,0 +1,18 @@ +; +; Supported features of the Google Virtual Ethernet 'gve' poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Speed capabilities = Y +Link status = Y +MTU update = Y +TSO = Y +RSS hash = Y +L4 checksum offload = Y +Basic stats = Y +Stats per queue = Y +Linux = Y +x86-32 = Y +x86-64 = Y +Usage doc = Y diff --git a/doc/guides/nics/gve.rst b/doc/guides/nics/gve.rst new file mode 100644 index 0000000000..310328c8ab --- /dev/null +++ b/doc/guides/nics/gve.rst @@ -0,0 +1,65 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(C) 2022 Intel Corporation. + +GVE poll mode driver +======================= + +The GVE PMD (**librte_net_i40e**) provides poll mode driver support for +Google Virtual Ethernet device. + +The base code is under MIT license and based on GVE kernel driver v1.3.0. +GVE base code files are: + +- gve_adminq.h +- gve_adminq.c +- gve_register.h +- gve_desc.h +- gve_desc_dqo.h + +Please refer to https://github.com/GoogleCloudPlatform/compute-virtual-ethernet-linux/tree/v1.3.0/google/gve +to find the original base code. + +GVE has 3 queue formats: + +- GQI_QPL +- GQI_RDA +- DQO_RDA + +GQI_QPL queue format is queue page list mode. Driver needs to allocate +memory and register this memory as a Queue Page List (QPL) in hardware +(Google Hypervisor/GVE Backend) first. Each queue has its own QPL. +Then Tx needs to copy packets to QPL memory and put this packet's offset +in the QPL memory into hardware descriptors so that hardware can get the +packets data. And Rx needs to read descriptors of offset in QPL to get +QPL address and copy packets from the address to get real packets data. + +GQI_RDA queue format works like usual NICs that driver can put packets' +physical address into hardware descriptors. + +DQO_RDA queue format has submission and completion queue pair for each +Tx/Rx queue. And similar as GQI_RDA, driver can put packets' physical +address into hardware descriptors. + +Please refer to https://www.kernel.org/doc/html/latest/networking/device_drivers/ethernet/google/gve.html +to get more information about GVE queue formats. + +Features and Limitations +------------------------ + +In this release, the GVE PMD provides the basic functionality of packet +reception and transmission. +Supported features of the GVE PMD are: + +- Multiple queues for TX and RX +- Receiver Side Scaling (RSS) +- TSO offload +- Port hardware statistics +- Link state information +- TX multi-segments (Scatter TX) +- Tx UDP/TCP/SCTP Checksum + +Currently, only GQI_QPL and GQI_RDA queue format are supported in PMD. +Jumbo Frame is not supported in PMD for now. It'll be added in the future +DPDK release. +Also, only GQI_QPL queue format is in use on GCP since GQI_RDA hasn't been +released in production. diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst index 8c021cf050..6674f4cf6f 100644 --- a/doc/guides/rel_notes/release_22_11.rst +++ b/doc/guides/rel_notes/release_22_11.rst @@ -55,6 +55,10 @@ New Features Also, make sure to start the actual text at the margin. ======================================================= +* **Added GVE net PMD** + + Added the new ``gve`` net driver for Google Virtual Ethernet devices. + See the :doc:`../nics/gve` NIC guide for more details on this new driver. Removed Items -------------