From patchwork Mon Oct 31 08:33:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xing, Beilei" X-Patchwork-Id: 119342 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E50E2A00C2; Mon, 31 Oct 2022 10:04:28 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5F19D40A89; Mon, 31 Oct 2022 10:04:23 +0100 (CET) Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id BD54E40151 for ; Mon, 31 Oct 2022 10:04:16 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667207056; x=1698743056; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=rHJ3bBIOvoHd6Ga8mO9ToUX7r7T2pgZIr8fA6koX7Pw=; b=SpmzKNpd2b5uNHw/rqm6Rzy9B8TsPBdcYm9r+zudIH+DBk2BEWsCoywH DsJU3QFbqa8Tql74xYam3H/+HBMxxZlwhgUQw3tw80zVM80vGUlPx6mUQ hlzKYiWtYEPRdJL1+Akcw4lgTigElwJ/0hz2yedJccFuKpaQkwaDFzpcr rzSMvud8xkwMUcfnk6EcYsiMHoHvtzvheJLfAIAqCMzHq/NaMK1iqf05i Jw4LjT+5xH7LqqpjIrrjZZQRt7RnkYteG+6PIYGFwF9LT1KYlxpcgc2ir w5n3k9tCoIlQByqVr3TFXzo36rkFRj96U452jMhmo0MVGI1dkol+DTgWK w==; X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="370926263" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="370926263" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Oct 2022 02:04:00 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="878664744" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="878664744" Received: from dpdk-beileix-3.sh.intel.com ([10.67.110.253]) by fmsmga006.fm.intel.com with ESMTP; 31 Oct 2022 02:03:57 -0700 From: beilei.xing@intel.com To: jingjing.wu@intel.com, beilei.xing@intel.com Cc: dev@dpdk.org, Junfeng Guo , Xiao Wang Subject: [PATCH v18 01/18] common/idpf: introduce common library Date: Mon, 31 Oct 2022 08:33:29 +0000 Message-Id: <20221031083346.16558-2-beilei.xing@intel.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20221031083346.16558-1-beilei.xing@intel.com> References: <20221031051556.98549-1-beilei.xing@intel.com> <20221031083346.16558-1-beilei.xing@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Junfeng Guo Introduce common library for IDPF (Infrastructure Data Path Function) PMD. Add base code and OS specific implementation first. Signed-off-by: Beilei Xing Signed-off-by: Xiao Wang Signed-off-by: Junfeng Guo --- drivers/common/idpf/base/idpf_alloc.h | 22 + drivers/common/idpf/base/idpf_common.c | 364 +++ drivers/common/idpf/base/idpf_controlq.c | 691 ++++ drivers/common/idpf/base/idpf_controlq.h | 224 ++ drivers/common/idpf/base/idpf_controlq_api.h | 207 ++ .../common/idpf/base/idpf_controlq_setup.c | 179 + drivers/common/idpf/base/idpf_devids.h | 18 + drivers/common/idpf/base/idpf_lan_pf_regs.h | 134 + drivers/common/idpf/base/idpf_lan_txrx.h | 428 +++ drivers/common/idpf/base/idpf_lan_vf_regs.h | 114 + drivers/common/idpf/base/idpf_osdep.h | 364 +++ drivers/common/idpf/base/idpf_prototype.h | 45 + drivers/common/idpf/base/idpf_type.h | 106 + drivers/common/idpf/base/meson.build | 14 + drivers/common/idpf/base/siov_regs.h | 47 + drivers/common/idpf/base/virtchnl.h | 2866 +++++++++++++++++ drivers/common/idpf/base/virtchnl2.h | 1462 +++++++++ drivers/common/idpf/base/virtchnl2_lan_desc.h | 606 ++++ .../common/idpf/base/virtchnl_inline_ipsec.h | 567 ++++ drivers/common/idpf/meson.build | 4 + drivers/common/idpf/version.map | 12 + drivers/common/meson.build | 1 + 22 files changed, 8475 insertions(+) create mode 100644 drivers/common/idpf/base/idpf_alloc.h create mode 100644 drivers/common/idpf/base/idpf_common.c create mode 100644 drivers/common/idpf/base/idpf_controlq.c create mode 100644 drivers/common/idpf/base/idpf_controlq.h create mode 100644 drivers/common/idpf/base/idpf_controlq_api.h create mode 100644 drivers/common/idpf/base/idpf_controlq_setup.c create mode 100644 drivers/common/idpf/base/idpf_devids.h create mode 100644 drivers/common/idpf/base/idpf_lan_pf_regs.h create mode 100644 drivers/common/idpf/base/idpf_lan_txrx.h create mode 100644 drivers/common/idpf/base/idpf_lan_vf_regs.h create mode 100644 drivers/common/idpf/base/idpf_osdep.h create mode 100644 drivers/common/idpf/base/idpf_prototype.h create mode 100644 drivers/common/idpf/base/idpf_type.h create mode 100644 drivers/common/idpf/base/meson.build create mode 100644 drivers/common/idpf/base/siov_regs.h create mode 100644 drivers/common/idpf/base/virtchnl.h create mode 100644 drivers/common/idpf/base/virtchnl2.h create mode 100644 drivers/common/idpf/base/virtchnl2_lan_desc.h create mode 100644 drivers/common/idpf/base/virtchnl_inline_ipsec.h create mode 100644 drivers/common/idpf/meson.build create mode 100644 drivers/common/idpf/version.map diff --git a/drivers/common/idpf/base/idpf_alloc.h b/drivers/common/idpf/base/idpf_alloc.h new file mode 100644 index 0000000000..bc054851b3 --- /dev/null +++ b/drivers/common/idpf/base/idpf_alloc.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2001-2022 Intel Corporation + */ + +#ifndef _IDPF_ALLOC_H_ +#define _IDPF_ALLOC_H_ + +/* Memory types */ +enum idpf_memset_type { + IDPF_NONDMA_MEM = 0, + IDPF_DMA_MEM +}; + +/* Memcpy types */ +enum idpf_memcpy_type { + IDPF_NONDMA_TO_NONDMA = 0, + IDPF_NONDMA_TO_DMA, + IDPF_DMA_TO_DMA, + IDPF_DMA_TO_NONDMA +}; + +#endif /* _IDPF_ALLOC_H_ */ diff --git a/drivers/common/idpf/base/idpf_common.c b/drivers/common/idpf/base/idpf_common.c new file mode 100644 index 0000000000..1debf129a3 --- /dev/null +++ b/drivers/common/idpf/base/idpf_common.c @@ -0,0 +1,364 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2001-2022 Intel Corporation + */ + +#include "idpf_type.h" +#include "idpf_prototype.h" +#include "virtchnl.h" + + +/** + * idpf_set_mac_type - Sets MAC type + * @hw: pointer to the HW structure + * + * This function sets the mac type of the adapter based on the + * vendor ID and device ID stored in the hw structure. + */ +int idpf_set_mac_type(struct idpf_hw *hw) +{ + int status = 0; + + DEBUGFUNC("Set MAC type\n"); + + if (hw->vendor_id == IDPF_INTEL_VENDOR_ID) { + switch (hw->device_id) { + case IDPF_DEV_ID_PF: + hw->mac.type = IDPF_MAC_PF; + break; + case IDPF_DEV_ID_VF: + hw->mac.type = IDPF_MAC_VF; + break; + default: + hw->mac.type = IDPF_MAC_GENERIC; + break; + } + } else { + status = -ENODEV; + } + + DEBUGOUT2("Setting MAC type found mac: %d, returns: %d\n", + hw->mac.type, status); + return status; +} + +/** + * idpf_init_hw - main initialization routine + * @hw: pointer to the hardware structure + * @ctlq_size: struct to pass ctlq size data + */ +int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size) +{ + struct idpf_ctlq_create_info *q_info; + int status = 0; + struct idpf_ctlq_info *cq = NULL; + + /* Setup initial control queues */ + q_info = (struct idpf_ctlq_create_info *) + idpf_calloc(hw, 2, sizeof(struct idpf_ctlq_create_info)); + if (!q_info) + return -ENOMEM; + + q_info[0].type = IDPF_CTLQ_TYPE_MAILBOX_TX; + q_info[0].buf_size = ctlq_size.asq_buf_size; + q_info[0].len = ctlq_size.asq_ring_size; + q_info[0].id = -1; /* default queue */ + + if (hw->mac.type == IDPF_MAC_PF) { + q_info[0].reg.head = PF_FW_ATQH; + q_info[0].reg.tail = PF_FW_ATQT; + q_info[0].reg.len = PF_FW_ATQLEN; + q_info[0].reg.bah = PF_FW_ATQBAH; + q_info[0].reg.bal = PF_FW_ATQBAL; + q_info[0].reg.len_mask = PF_FW_ATQLEN_ATQLEN_M; + q_info[0].reg.len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M; + q_info[0].reg.head_mask = PF_FW_ATQH_ATQH_M; + } else { + q_info[0].reg.head = VF_ATQH; + q_info[0].reg.tail = VF_ATQT; + q_info[0].reg.len = VF_ATQLEN; + q_info[0].reg.bah = VF_ATQBAH; + q_info[0].reg.bal = VF_ATQBAL; + q_info[0].reg.len_mask = VF_ATQLEN_ATQLEN_M; + q_info[0].reg.len_ena_mask = VF_ATQLEN_ATQENABLE_M; + q_info[0].reg.head_mask = VF_ATQH_ATQH_M; + } + + q_info[1].type = IDPF_CTLQ_TYPE_MAILBOX_RX; + q_info[1].buf_size = ctlq_size.arq_buf_size; + q_info[1].len = ctlq_size.arq_ring_size; + q_info[1].id = -1; /* default queue */ + + if (hw->mac.type == IDPF_MAC_PF) { + q_info[1].reg.head = PF_FW_ARQH; + q_info[1].reg.tail = PF_FW_ARQT; + q_info[1].reg.len = PF_FW_ARQLEN; + q_info[1].reg.bah = PF_FW_ARQBAH; + q_info[1].reg.bal = PF_FW_ARQBAL; + q_info[1].reg.len_mask = PF_FW_ARQLEN_ARQLEN_M; + q_info[1].reg.len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M; + q_info[1].reg.head_mask = PF_FW_ARQH_ARQH_M; + } else { + q_info[1].reg.head = VF_ARQH; + q_info[1].reg.tail = VF_ARQT; + q_info[1].reg.len = VF_ARQLEN; + q_info[1].reg.bah = VF_ARQBAH; + q_info[1].reg.bal = VF_ARQBAL; + q_info[1].reg.len_mask = VF_ARQLEN_ARQLEN_M; + q_info[1].reg.len_ena_mask = VF_ARQLEN_ARQENABLE_M; + q_info[1].reg.head_mask = VF_ARQH_ARQH_M; + } + + status = idpf_ctlq_init(hw, 2, q_info); + if (status) { + /* TODO return error */ + idpf_free(hw, q_info); + return status; + } + + LIST_FOR_EACH_ENTRY(cq, &hw->cq_list_head, idpf_ctlq_info, cq_list) { + if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX) + hw->asq = cq; + else if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX) + hw->arq = cq; + } + + /* TODO hardcode a mac addr for now */ + hw->mac.addr[0] = 0x00; + hw->mac.addr[1] = 0x00; + hw->mac.addr[2] = 0x00; + hw->mac.addr[3] = 0x00; + hw->mac.addr[4] = 0x03; + hw->mac.addr[5] = 0x14; + + return 0; +} + +/** + * idpf_send_msg_to_cp + * @hw: pointer to the hardware structure + * @v_opcode: opcodes for VF-PF communication + * @v_retval: return error code + * @msg: pointer to the msg buffer + * @msglen: msg length + * @cmd_details: pointer to command details + * + * Send message to CP. By default, this message + * is sent asynchronously, i.e. idpf_asq_send_command() does not wait for + * completion before returning. + */ +int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode, + int v_retval, u8 *msg, u16 msglen) +{ + struct idpf_ctlq_msg ctlq_msg = { 0 }; + struct idpf_dma_mem dma_mem = { 0 }; + int status; + + ctlq_msg.opcode = idpf_mbq_opc_send_msg_to_pf; + ctlq_msg.func_id = 0; + ctlq_msg.data_len = msglen; + ctlq_msg.cookie.mbx.chnl_retval = v_retval; + ctlq_msg.cookie.mbx.chnl_opcode = v_opcode; + + if (msglen > 0) { + dma_mem.va = (struct idpf_dma_mem *) + idpf_alloc_dma_mem(hw, &dma_mem, msglen); + if (!dma_mem.va) + return -ENOMEM; + + idpf_memcpy(dma_mem.va, msg, msglen, IDPF_NONDMA_TO_DMA); + ctlq_msg.ctx.indirect.payload = &dma_mem; + } + status = idpf_ctlq_send(hw, hw->asq, 1, &ctlq_msg); + + if (dma_mem.va) + idpf_free_dma_mem(hw, &dma_mem); + + return status; +} + +/** + * idpf_asq_done - check if FW has processed the Admin Send Queue + * @hw: pointer to the hw struct + * + * Returns true if the firmware has processed all descriptors on the + * admin send queue. Returns false if there are still requests pending. + */ +bool idpf_asq_done(struct idpf_hw *hw) +{ + /* AQ designers suggest use of head for better + * timing reliability than DD bit + */ + return rd32(hw, hw->asq->reg.head) == hw->asq->next_to_use; +} + +/** + * idpf_check_asq_alive + * @hw: pointer to the hw struct + * + * Returns true if Queue is enabled else false. + */ +bool idpf_check_asq_alive(struct idpf_hw *hw) +{ + if (hw->asq->reg.len) + return !!(rd32(hw, hw->asq->reg.len) & + PF_FW_ATQLEN_ATQENABLE_M); + + return false; +} + +/** + * idpf_clean_arq_element + * @hw: pointer to the hw struct + * @e: event info from the receive descriptor, includes any buffers + * @pending: number of events that could be left to process + * + * This function cleans one Admin Receive Queue element and returns + * the contents through e. It can also return how many events are + * left to process through 'pending' + */ +int idpf_clean_arq_element(struct idpf_hw *hw, + struct idpf_arq_event_info *e, u16 *pending) +{ + struct idpf_ctlq_msg msg = { 0 }; + int status; + + *pending = 1; + + status = idpf_ctlq_recv(hw->arq, pending, &msg); + + /* ctlq_msg does not align to ctlq_desc, so copy relevant data here */ + e->desc.opcode = msg.opcode; + e->desc.cookie_high = msg.cookie.mbx.chnl_opcode; + e->desc.cookie_low = msg.cookie.mbx.chnl_retval; + e->desc.ret_val = msg.status; + e->desc.datalen = msg.data_len; + if (msg.data_len > 0) { + e->buf_len = msg.data_len; + idpf_memcpy(e->msg_buf, msg.ctx.indirect.payload->va, msg.data_len, + IDPF_DMA_TO_NONDMA); + } + return status; +} + +/** + * idpf_deinit_hw - shutdown routine + * @hw: pointer to the hardware structure + */ +int idpf_deinit_hw(struct idpf_hw *hw) +{ + hw->asq = NULL; + hw->arq = NULL; + + return idpf_ctlq_deinit(hw); +} + +/** + * idpf_reset + * @hw: pointer to the hardware structure + * + * Send a RESET message to the CPF. Does not wait for response from CPF + * as none will be forthcoming. Immediately after calling this function, + * the control queue should be shut down and (optionally) reinitialized. + */ +int idpf_reset(struct idpf_hw *hw) +{ + return idpf_send_msg_to_cp(hw, VIRTCHNL_OP_RESET_VF, + 0, NULL, 0); +} + +/** + * idpf_get_set_rss_lut + * @hw: pointer to the hardware structure + * @vsi_id: vsi fw index + * @pf_lut: for PF table set true, for VSI table set false + * @lut: pointer to the lut buffer provided by the caller + * @lut_size: size of the lut buffer + * @set: set true to set the table, false to get the table + * + * Internal function to get or set RSS look up table + */ +STATIC int idpf_get_set_rss_lut(struct idpf_hw *hw, u16 vsi_id, + bool pf_lut, u8 *lut, u16 lut_size, + bool set) +{ + /* TODO fill out command */ + return 0; +} + +/** + * idpf_get_rss_lut + * @hw: pointer to the hardware structure + * @vsi_id: vsi fw index + * @pf_lut: for PF table set true, for VSI table set false + * @lut: pointer to the lut buffer provided by the caller + * @lut_size: size of the lut buffer + * + * get the RSS lookup table, PF or VSI type + */ +int idpf_get_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut, + u8 *lut, u16 lut_size) +{ + return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, false); +} + +/** + * idpf_set_rss_lut + * @hw: pointer to the hardware structure + * @vsi_id: vsi fw index + * @pf_lut: for PF table set true, for VSI table set false + * @lut: pointer to the lut buffer provided by the caller + * @lut_size: size of the lut buffer + * + * set the RSS lookup table, PF or VSI type + */ +int idpf_set_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut, + u8 *lut, u16 lut_size) +{ + return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, true); +} + +/** + * idpf_get_set_rss_key + * @hw: pointer to the hw struct + * @vsi_id: vsi fw index + * @key: pointer to key info struct + * @set: set true to set the key, false to get the key + * + * get the RSS key per VSI + */ +STATIC int idpf_get_set_rss_key(struct idpf_hw *hw, u16 vsi_id, + struct idpf_get_set_rss_key_data *key, + bool set) +{ + /* TODO fill out command */ + return 0; +} + +/** + * idpf_get_rss_key + * @hw: pointer to the hw struct + * @vsi_id: vsi fw index + * @key: pointer to key info struct + * + */ +int idpf_get_rss_key(struct idpf_hw *hw, u16 vsi_id, + struct idpf_get_set_rss_key_data *key) +{ + return idpf_get_set_rss_key(hw, vsi_id, key, false); +} + +/** + * idpf_set_rss_key + * @hw: pointer to the hw struct + * @vsi_id: vsi fw index + * @key: pointer to key info struct + * + * set the RSS key per VSI + */ +int idpf_set_rss_key(struct idpf_hw *hw, u16 vsi_id, + struct idpf_get_set_rss_key_data *key) +{ + return idpf_get_set_rss_key(hw, vsi_id, key, true); +} + +RTE_LOG_REGISTER_DEFAULT(idpf_common_logger, NOTICE); diff --git a/drivers/common/idpf/base/idpf_controlq.c b/drivers/common/idpf/base/idpf_controlq.c new file mode 100644 index 0000000000..3af81e5a64 --- /dev/null +++ b/drivers/common/idpf/base/idpf_controlq.c @@ -0,0 +1,691 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2001-2022 Intel Corporation + */ + +#include "idpf_controlq.h" + +/** + * idpf_ctlq_setup_regs - initialize control queue registers + * @cq: pointer to the specific control queue + * @q_create_info: structs containing info for each queue to be initialized + */ +static void +idpf_ctlq_setup_regs(struct idpf_ctlq_info *cq, + struct idpf_ctlq_create_info *q_create_info) +{ + /* set head and tail registers in our local struct */ + cq->reg.head = q_create_info->reg.head; + cq->reg.tail = q_create_info->reg.tail; + cq->reg.len = q_create_info->reg.len; + cq->reg.bah = q_create_info->reg.bah; + cq->reg.bal = q_create_info->reg.bal; + cq->reg.len_mask = q_create_info->reg.len_mask; + cq->reg.len_ena_mask = q_create_info->reg.len_ena_mask; + cq->reg.head_mask = q_create_info->reg.head_mask; +} + +/** + * idpf_ctlq_init_regs - Initialize control queue registers + * @hw: pointer to hw struct + * @cq: pointer to the specific Control queue + * @is_rxq: true if receive control queue, false otherwise + * + * Initialize registers. The caller is expected to have already initialized the + * descriptor ring memory and buffer memory + */ +static void idpf_ctlq_init_regs(struct idpf_hw *hw, struct idpf_ctlq_info *cq, + bool is_rxq) +{ + /* Update tail to post pre-allocated buffers for rx queues */ + if (is_rxq) + wr32(hw, cq->reg.tail, (u32)(cq->ring_size - 1)); + + /* For non-Mailbox control queues only TAIL need to be set */ + if (cq->q_id != -1) + return; + + /* Clear Head for both send or receive */ + wr32(hw, cq->reg.head, 0); + + /* set starting point */ + wr32(hw, cq->reg.bal, IDPF_LO_DWORD(cq->desc_ring.pa)); + wr32(hw, cq->reg.bah, IDPF_HI_DWORD(cq->desc_ring.pa)); + wr32(hw, cq->reg.len, (cq->ring_size | cq->reg.len_ena_mask)); +} + +/** + * idpf_ctlq_init_rxq_bufs - populate receive queue descriptors with buf + * @cq: pointer to the specific Control queue + * + * Record the address of the receive queue DMA buffers in the descriptors. + * The buffers must have been previously allocated. + */ +static void idpf_ctlq_init_rxq_bufs(struct idpf_ctlq_info *cq) +{ + int i = 0; + + for (i = 0; i < cq->ring_size; i++) { + struct idpf_ctlq_desc *desc = IDPF_CTLQ_DESC(cq, i); + struct idpf_dma_mem *bi = cq->bi.rx_buff[i]; + + /* No buffer to post to descriptor, continue */ + if (!bi) + continue; + + desc->flags = + CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD); + desc->opcode = 0; + desc->datalen = (__le16)CPU_TO_LE16(bi->size); + desc->ret_val = 0; + desc->cookie_high = 0; + desc->cookie_low = 0; + desc->params.indirect.addr_high = + CPU_TO_LE32(IDPF_HI_DWORD(bi->pa)); + desc->params.indirect.addr_low = + CPU_TO_LE32(IDPF_LO_DWORD(bi->pa)); + desc->params.indirect.param0 = 0; + desc->params.indirect.param1 = 0; + } +} + +/** + * idpf_ctlq_shutdown - shutdown the CQ + * @hw: pointer to hw struct + * @cq: pointer to the specific Control queue + * + * The main shutdown routine for any controq queue + */ +static void idpf_ctlq_shutdown(struct idpf_hw *hw, struct idpf_ctlq_info *cq) +{ + idpf_acquire_lock(&cq->cq_lock); + + if (!cq->ring_size) + goto shutdown_sq_out; + +#ifdef SIMICS_BUILD + wr32(hw, cq->reg.head, 0); + wr32(hw, cq->reg.tail, 0); + wr32(hw, cq->reg.len, 0); + wr32(hw, cq->reg.bal, 0); + wr32(hw, cq->reg.bah, 0); +#endif /* SIMICS_BUILD */ + + /* free ring buffers and the ring itself */ + idpf_ctlq_dealloc_ring_res(hw, cq); + + /* Set ring_size to 0 to indicate uninitialized queue */ + cq->ring_size = 0; + +shutdown_sq_out: + idpf_release_lock(&cq->cq_lock); + idpf_destroy_lock(&cq->cq_lock); +} + +/** + * idpf_ctlq_add - add one control queue + * @hw: pointer to hardware struct + * @qinfo: info for queue to be created + * @cq_out: (output) double pointer to control queue to be created + * + * Allocate and initialize a control queue and add it to the control queue list. + * The cq parameter will be allocated/initialized and passed back to the caller + * if no errors occur. + * + * Note: idpf_ctlq_init must be called prior to any calls to idpf_ctlq_add + */ +int idpf_ctlq_add(struct idpf_hw *hw, + struct idpf_ctlq_create_info *qinfo, + struct idpf_ctlq_info **cq_out) +{ + bool is_rxq = false; + int status = 0; + + if (!qinfo->len || !qinfo->buf_size || + qinfo->len > IDPF_CTLQ_MAX_RING_SIZE || + qinfo->buf_size > IDPF_CTLQ_MAX_BUF_LEN) + return -EINVAL; + + *cq_out = (struct idpf_ctlq_info *) + idpf_calloc(hw, 1, sizeof(struct idpf_ctlq_info)); + if (!(*cq_out)) + return -ENOMEM; + + (*cq_out)->cq_type = qinfo->type; + (*cq_out)->q_id = qinfo->id; + (*cq_out)->buf_size = qinfo->buf_size; + (*cq_out)->ring_size = qinfo->len; + + (*cq_out)->next_to_use = 0; + (*cq_out)->next_to_clean = 0; + (*cq_out)->next_to_post = (*cq_out)->ring_size - 1; + + switch (qinfo->type) { + case IDPF_CTLQ_TYPE_MAILBOX_RX: + is_rxq = true; +#ifdef __KERNEL__ + fallthrough; +#else + /* fallthrough */ +#endif /* __KERNEL__ */ + case IDPF_CTLQ_TYPE_MAILBOX_TX: + status = idpf_ctlq_alloc_ring_res(hw, *cq_out); + break; + default: + status = -EINVAL; + break; + } + + if (status) + goto init_free_q; + + if (is_rxq) { + idpf_ctlq_init_rxq_bufs(*cq_out); + } else { + /* Allocate the array of msg pointers for TX queues */ + (*cq_out)->bi.tx_msg = (struct idpf_ctlq_msg **) + idpf_calloc(hw, qinfo->len, + sizeof(struct idpf_ctlq_msg *)); + if (!(*cq_out)->bi.tx_msg) { + status = -ENOMEM; + goto init_dealloc_q_mem; + } + } + + idpf_ctlq_setup_regs(*cq_out, qinfo); + + idpf_ctlq_init_regs(hw, *cq_out, is_rxq); + + idpf_init_lock(&(*cq_out)->cq_lock); + + LIST_INSERT_HEAD(&hw->cq_list_head, (*cq_out), cq_list); + + return status; + +init_dealloc_q_mem: + /* free ring buffers and the ring itself */ + idpf_ctlq_dealloc_ring_res(hw, *cq_out); +init_free_q: + idpf_free(hw, *cq_out); + + return status; +} + +/** + * idpf_ctlq_remove - deallocate and remove specified control queue + * @hw: pointer to hardware struct + * @cq: pointer to control queue to be removed + */ +void idpf_ctlq_remove(struct idpf_hw *hw, + struct idpf_ctlq_info *cq) +{ + LIST_REMOVE(cq, cq_list); + idpf_ctlq_shutdown(hw, cq); + idpf_free(hw, cq); +} + +/** + * idpf_ctlq_init - main initialization routine for all control queues + * @hw: pointer to hardware struct + * @num_q: number of queues to initialize + * @q_info: array of structs containing info for each queue to be initialized + * + * This initializes any number and any type of control queues. This is an all + * or nothing routine; if one fails, all previously allocated queues will be + * destroyed. This must be called prior to using the individual add/remove + * APIs. + */ +int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q, + struct idpf_ctlq_create_info *q_info) +{ + struct idpf_ctlq_info *cq = NULL, *tmp = NULL; + int ret_code = 0; + int i = 0; + + LIST_INIT(&hw->cq_list_head); + + for (i = 0; i < num_q; i++) { + struct idpf_ctlq_create_info *qinfo = q_info + i; + + ret_code = idpf_ctlq_add(hw, qinfo, &cq); + if (ret_code) + goto init_destroy_qs; + } + + return ret_code; + +init_destroy_qs: + LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head, + idpf_ctlq_info, cq_list) + idpf_ctlq_remove(hw, cq); + + return ret_code; +} + +/** + * idpf_ctlq_deinit - destroy all control queues + * @hw: pointer to hw struct + */ +int idpf_ctlq_deinit(struct idpf_hw *hw) +{ + struct idpf_ctlq_info *cq = NULL, *tmp = NULL; + int ret_code = 0; + + LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head, + idpf_ctlq_info, cq_list) + idpf_ctlq_remove(hw, cq); + + return ret_code; +} + +/** + * idpf_ctlq_send - send command to Control Queue (CTQ) + * @hw: pointer to hw struct + * @cq: handle to control queue struct to send on + * @num_q_msg: number of messages to send on control queue + * @q_msg: pointer to array of queue messages to be sent + * + * The caller is expected to allocate DMAable buffers and pass them to the + * send routine via the q_msg struct / control queue specific data struct. + * The control queue will hold a reference to each send message until + * the completion for that message has been cleaned. + */ +int idpf_ctlq_send(struct idpf_hw *hw, struct idpf_ctlq_info *cq, + u16 num_q_msg, struct idpf_ctlq_msg q_msg[]) +{ + struct idpf_ctlq_desc *desc; + int num_desc_avail = 0; + int status = 0; + int i = 0; + + if (!cq || !cq->ring_size) + return -ENOBUFS; + + idpf_acquire_lock(&cq->cq_lock); + + /* Ensure there are enough descriptors to send all messages */ + num_desc_avail = IDPF_CTLQ_DESC_UNUSED(cq); + if (num_desc_avail == 0 || num_desc_avail < num_q_msg) { + status = -ENOSPC; + goto sq_send_command_out; + } + + for (i = 0; i < num_q_msg; i++) { + struct idpf_ctlq_msg *msg = &q_msg[i]; + u64 msg_cookie; + + desc = IDPF_CTLQ_DESC(cq, cq->next_to_use); + + desc->opcode = CPU_TO_LE16(msg->opcode); + desc->pfid_vfid = CPU_TO_LE16(msg->func_id); + + msg_cookie = *(u64 *)&msg->cookie; + desc->cookie_high = + CPU_TO_LE32(IDPF_HI_DWORD(msg_cookie)); + desc->cookie_low = + CPU_TO_LE32(IDPF_LO_DWORD(msg_cookie)); + + desc->flags = CPU_TO_LE16((msg->host_id & IDPF_HOST_ID_MASK) << + IDPF_CTLQ_FLAG_HOST_ID_S); + if (msg->data_len) { + struct idpf_dma_mem *buff = msg->ctx.indirect.payload; + + desc->datalen |= CPU_TO_LE16(msg->data_len); + desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF); + desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_RD); + + /* Update the address values in the desc with the pa + * value for respective buffer + */ + desc->params.indirect.addr_high = + CPU_TO_LE32(IDPF_HI_DWORD(buff->pa)); + desc->params.indirect.addr_low = + CPU_TO_LE32(IDPF_LO_DWORD(buff->pa)); + + idpf_memcpy(&desc->params, msg->ctx.indirect.context, + IDPF_INDIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA); +#ifdef SIMICS_BUILD + /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf + * need to set peer PF function id in param0 for Simics + */ + if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) { + desc->params.indirect.param0 = + CPU_TO_LE32(msg->func_id); + } +#endif + } else { + idpf_memcpy(&desc->params, msg->ctx.direct, + IDPF_DIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA); +#ifdef SIMICS_BUILD + /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf + * need to set peer PF function id in param0 for Simics + */ + if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) { + desc->params.direct.param0 = + CPU_TO_LE32(msg->func_id); + } +#endif + } + + /* Store buffer info */ + cq->bi.tx_msg[cq->next_to_use] = msg; + + (cq->next_to_use)++; + if (cq->next_to_use == cq->ring_size) + cq->next_to_use = 0; + } + + /* Force memory write to complete before letting hardware + * know that there are new descriptors to fetch. + */ + idpf_wmb(); + + wr32(hw, cq->reg.tail, cq->next_to_use); + +sq_send_command_out: + idpf_release_lock(&cq->cq_lock); + + return status; +} + +/** + * idpf_ctlq_clean_sq - reclaim send descriptors on HW write back for the + * requested queue + * @cq: pointer to the specific Control queue + * @clean_count: (input|output) number of descriptors to clean as input, and + * number of descriptors actually cleaned as output + * @msg_status: (output) pointer to msg pointer array to be populated; needs + * to be allocated by caller + * + * Returns an array of message pointers associated with the cleaned + * descriptors. The pointers are to the original ctlq_msgs sent on the cleaned + * descriptors. The status will be returned for each; any messages that failed + * to send will have a non-zero status. The caller is expected to free original + * ctlq_msgs and free or reuse the DMA buffers. + */ +int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count, + struct idpf_ctlq_msg *msg_status[]) +{ + struct idpf_ctlq_desc *desc; + u16 i = 0, num_to_clean; + u16 ntc, desc_err; + int ret = 0; + + if (!cq || !cq->ring_size) + return -ENOBUFS; + + if (*clean_count == 0) + return 0; + if (*clean_count > cq->ring_size) + return -EINVAL; + + idpf_acquire_lock(&cq->cq_lock); + + ntc = cq->next_to_clean; + + num_to_clean = *clean_count; + + for (i = 0; i < num_to_clean; i++) { + /* Fetch next descriptor and check if marked as done */ + desc = IDPF_CTLQ_DESC(cq, ntc); + if (!(LE16_TO_CPU(desc->flags) & IDPF_CTLQ_FLAG_DD)) + break; + + desc_err = LE16_TO_CPU(desc->ret_val); + if (desc_err) { + /* strip off FW internal code */ + desc_err &= 0xff; + } + + msg_status[i] = cq->bi.tx_msg[ntc]; + msg_status[i]->status = desc_err; + + cq->bi.tx_msg[ntc] = NULL; + + /* Zero out any stale data */ + idpf_memset(desc, 0, sizeof(*desc), IDPF_DMA_MEM); + + ntc++; + if (ntc == cq->ring_size) + ntc = 0; + } + + cq->next_to_clean = ntc; + + idpf_release_lock(&cq->cq_lock); + + /* Return number of descriptors actually cleaned */ + *clean_count = i; + + return ret; +} + +/** + * idpf_ctlq_post_rx_buffs - post buffers to descriptor ring + * @hw: pointer to hw struct + * @cq: pointer to control queue handle + * @buff_count: (input|output) input is number of buffers caller is trying to + * return; output is number of buffers that were not posted + * @buffs: array of pointers to dma mem structs to be given to hardware + * + * Caller uses this function to return DMA buffers to the descriptor ring after + * consuming them; buff_count will be the number of buffers. + * + * Note: this function needs to be called after a receive call even + * if there are no DMA buffers to be returned, i.e. buff_count = 0, + * buffs = NULL to support direct commands + */ +int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw, struct idpf_ctlq_info *cq, + u16 *buff_count, struct idpf_dma_mem **buffs) +{ + struct idpf_ctlq_desc *desc; + u16 ntp = cq->next_to_post; + bool buffs_avail = false; + u16 tbp = ntp + 1; + int status = 0; + int i = 0; + + if (*buff_count > cq->ring_size) + return -EINVAL; + + if (*buff_count > 0) + buffs_avail = true; + + idpf_acquire_lock(&cq->cq_lock); + + if (tbp >= cq->ring_size) + tbp = 0; + + if (tbp == cq->next_to_clean) + /* Nothing to do */ + goto post_buffs_out; + + /* Post buffers for as many as provided or up until the last one used */ + while (ntp != cq->next_to_clean) { + desc = IDPF_CTLQ_DESC(cq, ntp); + + if (cq->bi.rx_buff[ntp]) + goto fill_desc; + if (!buffs_avail) { + /* If the caller hasn't given us any buffers or + * there are none left, search the ring itself + * for an available buffer to move to this + * entry starting at the next entry in the ring + */ + tbp = ntp + 1; + + /* Wrap ring if necessary */ + if (tbp >= cq->ring_size) + tbp = 0; + + while (tbp != cq->next_to_clean) { + if (cq->bi.rx_buff[tbp]) { + cq->bi.rx_buff[ntp] = + cq->bi.rx_buff[tbp]; + cq->bi.rx_buff[tbp] = NULL; + + /* Found a buffer, no need to + * search anymore + */ + break; + } + + /* Wrap ring if necessary */ + tbp++; + if (tbp >= cq->ring_size) + tbp = 0; + } + + if (tbp == cq->next_to_clean) + goto post_buffs_out; + } else { + /* Give back pointer to DMA buffer */ + cq->bi.rx_buff[ntp] = buffs[i]; + i++; + + if (i >= *buff_count) + buffs_avail = false; + } + +fill_desc: + desc->flags = + CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD); + + /* Post buffers to descriptor */ + desc->datalen = CPU_TO_LE16(cq->bi.rx_buff[ntp]->size); + desc->params.indirect.addr_high = + CPU_TO_LE32(IDPF_HI_DWORD(cq->bi.rx_buff[ntp]->pa)); + desc->params.indirect.addr_low = + CPU_TO_LE32(IDPF_LO_DWORD(cq->bi.rx_buff[ntp]->pa)); + + ntp++; + if (ntp == cq->ring_size) + ntp = 0; + } + +post_buffs_out: + /* Only update tail if buffers were actually posted */ + if (cq->next_to_post != ntp) { + if (ntp) + /* Update next_to_post to ntp - 1 since current ntp + * will not have a buffer + */ + cq->next_to_post = ntp - 1; + else + /* Wrap to end of end ring since current ntp is 0 */ + cq->next_to_post = cq->ring_size - 1; + + wr32(hw, cq->reg.tail, cq->next_to_post); + } + + idpf_release_lock(&cq->cq_lock); + + /* return the number of buffers that were not posted */ + *buff_count = *buff_count - i; + + return status; +} + +/** + * idpf_ctlq_recv - receive control queue message call back + * @cq: pointer to control queue handle to receive on + * @num_q_msg: (input|output) input number of messages that should be received; + * output number of messages actually received + * @q_msg: (output) array of received control queue messages on this q; + * needs to be pre-allocated by caller for as many messages as requested + * + * Called by interrupt handler or polling mechanism. Caller is expected + * to free buffers + */ +int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg, + struct idpf_ctlq_msg *q_msg) +{ + u16 num_to_clean, ntc, ret_val, flags; + struct idpf_ctlq_desc *desc; + int ret_code = 0; + u16 i = 0; + + if (!cq || !cq->ring_size) + return -ENOBUFS; + + if (*num_q_msg == 0) + return 0; + else if (*num_q_msg > cq->ring_size) + return -EINVAL; + + /* take the lock before we start messing with the ring */ + idpf_acquire_lock(&cq->cq_lock); + + ntc = cq->next_to_clean; + + num_to_clean = *num_q_msg; + + for (i = 0; i < num_to_clean; i++) { + u64 msg_cookie; + + /* Fetch next descriptor and check if marked as done */ + desc = IDPF_CTLQ_DESC(cq, ntc); + flags = LE16_TO_CPU(desc->flags); + + if (!(flags & IDPF_CTLQ_FLAG_DD)) + break; + + ret_val = LE16_TO_CPU(desc->ret_val); + + q_msg[i].vmvf_type = (flags & + (IDPF_CTLQ_FLAG_FTYPE_VM | + IDPF_CTLQ_FLAG_FTYPE_PF)) >> + IDPF_CTLQ_FLAG_FTYPE_S; + + if (flags & IDPF_CTLQ_FLAG_ERR) + ret_code = -EBADMSG; + + msg_cookie = (u64)LE32_TO_CPU(desc->cookie_high) << 32; + msg_cookie |= (u64)LE32_TO_CPU(desc->cookie_low); + idpf_memcpy(&q_msg[i].cookie, &msg_cookie, sizeof(u64), + IDPF_NONDMA_TO_NONDMA); + + q_msg[i].opcode = LE16_TO_CPU(desc->opcode); + q_msg[i].data_len = LE16_TO_CPU(desc->datalen); + q_msg[i].status = ret_val; + + if (desc->datalen) { + idpf_memcpy(q_msg[i].ctx.indirect.context, + &desc->params.indirect, + IDPF_INDIRECT_CTX_SIZE, + IDPF_DMA_TO_NONDMA); + + /* Assign pointer to dma buffer to ctlq_msg array + * to be given to upper layer + */ + q_msg[i].ctx.indirect.payload = cq->bi.rx_buff[ntc]; + + /* Zero out pointer to DMA buffer info; + * will be repopulated by post buffers API + */ + cq->bi.rx_buff[ntc] = NULL; + } else { + idpf_memcpy(q_msg[i].ctx.direct, + desc->params.raw, + IDPF_DIRECT_CTX_SIZE, + IDPF_DMA_TO_NONDMA); + } + + /* Zero out stale data in descriptor */ + idpf_memset(desc, 0, sizeof(struct idpf_ctlq_desc), + IDPF_DMA_MEM); + + ntc++; + if (ntc == cq->ring_size) + ntc = 0; + }; + + cq->next_to_clean = ntc; + + idpf_release_lock(&cq->cq_lock); + + *num_q_msg = i; + if (*num_q_msg == 0) + ret_code = -ENOMSG; + + return ret_code; +} diff --git a/drivers/common/idpf/base/idpf_controlq.h b/drivers/common/idpf/base/idpf_controlq.h new file mode 100644 index 0000000000..e7b0d803b3 --- /dev/null +++ b/drivers/common/idpf/base/idpf_controlq.h @@ -0,0 +1,224 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2001-2022 Intel Corporation + */ + +#ifndef _IDPF_CONTROLQ_H_ +#define _IDPF_CONTROLQ_H_ + +#ifdef __KERNEL__ +#include +#endif + +#ifndef __KERNEL__ +#include "idpf_osdep.h" +#include "idpf_alloc.h" +#endif +#include "idpf_controlq_api.h" + +/* Maximum buffer lengths for all control queue types */ +#define IDPF_CTLQ_MAX_RING_SIZE 1024 +#define IDPF_CTLQ_MAX_BUF_LEN 4096 + +#define IDPF_CTLQ_DESC(R, i) \ + (&(((struct idpf_ctlq_desc *)((R)->desc_ring.va))[i])) + +#define IDPF_CTLQ_DESC_UNUSED(R) \ + ((u16)((((R)->next_to_clean > (R)->next_to_use) ? 0 : (R)->ring_size) + \ + (R)->next_to_clean - (R)->next_to_use - 1)) + +#ifndef __KERNEL__ +/* Data type manipulation macros. */ +#define IDPF_HI_DWORD(x) ((u32)((((x) >> 16) >> 16) & 0xFFFFFFFF)) +#define IDPF_LO_DWORD(x) ((u32)((x) & 0xFFFFFFFF)) +#define IDPF_HI_WORD(x) ((u16)(((x) >> 16) & 0xFFFF)) +#define IDPF_LO_WORD(x) ((u16)((x) & 0xFFFF)) + +#endif +/* Control Queue default settings */ +#define IDPF_CTRL_SQ_CMD_TIMEOUT 250 /* msecs */ + +struct idpf_ctlq_desc { + __le16 flags; + __le16 opcode; + __le16 datalen; /* 0 for direct commands */ + union { + __le16 ret_val; + __le16 pfid_vfid; +#define IDPF_CTLQ_DESC_VF_ID_S 0 +#ifdef SIMICS_BUILD +#define IDPF_CTLQ_DESC_VF_ID_M (0x3FF << IDPF_CTLQ_DESC_VF_ID_S) +#define IDPF_CTLQ_DESC_PF_ID_S 10 +#define IDPF_CTLQ_DESC_PF_ID_M (0x3F << IDPF_CTLQ_DESC_PF_ID_S) +#else +#define IDPF_CTLQ_DESC_VF_ID_M (0x7FF << IDPF_CTLQ_DESC_VF_ID_S) +#define IDPF_CTLQ_DESC_PF_ID_S 11 +#define IDPF_CTLQ_DESC_PF_ID_M (0x1F << IDPF_CTLQ_DESC_PF_ID_S) +#endif + }; + __le32 cookie_high; + __le32 cookie_low; + union { + struct { + __le32 param0; + __le32 param1; + __le32 param2; + __le32 param3; + } direct; + struct { + __le32 param0; + __le32 param1; + __le32 addr_high; + __le32 addr_low; + } indirect; + u8 raw[16]; + } params; +}; + +/* Flags sub-structure + * |0 |1 |2 |3 |4 |5 |6 |7 |8 |9 |10 |11 |12 |13 |14 |15 | + * |DD |CMP|ERR| * RSV * |FTYPE | *RSV* |RD |VFC|BUF| HOST_ID | + */ +/* command flags and offsets */ +#define IDPF_CTLQ_FLAG_DD_S 0 +#define IDPF_CTLQ_FLAG_CMP_S 1 +#define IDPF_CTLQ_FLAG_ERR_S 2 +#define IDPF_CTLQ_FLAG_FTYPE_S 6 +#define IDPF_CTLQ_FLAG_RD_S 10 +#define IDPF_CTLQ_FLAG_VFC_S 11 +#define IDPF_CTLQ_FLAG_BUF_S 12 +#define IDPF_CTLQ_FLAG_HOST_ID_S 13 + +#define IDPF_CTLQ_FLAG_DD BIT(IDPF_CTLQ_FLAG_DD_S) /* 0x1 */ +#define IDPF_CTLQ_FLAG_CMP BIT(IDPF_CTLQ_FLAG_CMP_S) /* 0x2 */ +#define IDPF_CTLQ_FLAG_ERR BIT(IDPF_CTLQ_FLAG_ERR_S) /* 0x4 */ +#define IDPF_CTLQ_FLAG_FTYPE_VM BIT(IDPF_CTLQ_FLAG_FTYPE_S) /* 0x40 */ +#define IDPF_CTLQ_FLAG_FTYPE_PF BIT(IDPF_CTLQ_FLAG_FTYPE_S + 1) /* 0x80 */ +#define IDPF_CTLQ_FLAG_RD BIT(IDPF_CTLQ_FLAG_RD_S) /* 0x400 */ +#define IDPF_CTLQ_FLAG_VFC BIT(IDPF_CTLQ_FLAG_VFC_S) /* 0x800 */ +#define IDPF_CTLQ_FLAG_BUF BIT(IDPF_CTLQ_FLAG_BUF_S) /* 0x1000 */ + +/* Host ID is a special field that has 3b and not a 1b flag */ +#define IDPF_CTLQ_FLAG_HOST_ID_M MAKE_MASK(0x7000UL, IDPF_CTLQ_FLAG_HOST_ID_S) + +struct idpf_mbxq_desc { + u8 pad[8]; /* CTLQ flags/opcode/len/retval fields */ + u32 chnl_opcode; /* avoid confusion with desc->opcode */ + u32 chnl_retval; /* ditto for desc->retval */ + u32 pf_vf_id; /* used by CP when sending to PF */ +}; + +enum idpf_mac_type { + IDPF_MAC_UNKNOWN = 0, + IDPF_MAC_PF, + IDPF_MAC_VF, + IDPF_MAC_GENERIC +}; + +#define ETH_ALEN 6 + +struct idpf_mac_info { + enum idpf_mac_type type; + u8 addr[ETH_ALEN]; + u8 perm_addr[ETH_ALEN]; +}; + +#define IDPF_AQ_LINK_UP 0x1 + +/* PCI bus types */ +enum idpf_bus_type { + idpf_bus_type_unknown = 0, + idpf_bus_type_pci, + idpf_bus_type_pcix, + idpf_bus_type_pci_express, + idpf_bus_type_reserved +}; + +/* PCI bus speeds */ +enum idpf_bus_speed { + idpf_bus_speed_unknown = 0, + idpf_bus_speed_33 = 33, + idpf_bus_speed_66 = 66, + idpf_bus_speed_100 = 100, + idpf_bus_speed_120 = 120, + idpf_bus_speed_133 = 133, + idpf_bus_speed_2500 = 2500, + idpf_bus_speed_5000 = 5000, + idpf_bus_speed_8000 = 8000, + idpf_bus_speed_reserved +}; + +/* PCI bus widths */ +enum idpf_bus_width { + idpf_bus_width_unknown = 0, + idpf_bus_width_pcie_x1 = 1, + idpf_bus_width_pcie_x2 = 2, + idpf_bus_width_pcie_x4 = 4, + idpf_bus_width_pcie_x8 = 8, + idpf_bus_width_32 = 32, + idpf_bus_width_64 = 64, + idpf_bus_width_reserved +}; + +/* Bus parameters */ +struct idpf_bus_info { + enum idpf_bus_speed speed; + enum idpf_bus_width width; + enum idpf_bus_type type; + + u16 func; + u16 device; + u16 lan_id; + u16 bus_id; +}; + +/* Function specific capabilities */ +struct idpf_hw_func_caps { + u32 num_alloc_vfs; + u32 vf_base_id; +}; + +/* Define the APF hardware struct to replace other control structs as needed + * Align to ctlq_hw_info + */ +struct idpf_hw { + /* Some part of BAR0 address space is not mapped by the LAN driver. + * This results in 2 regions of BAR0 to be mapped by LAN driver which + * will have its own base hardware address when mapped. + */ + u8 *hw_addr; + u8 *hw_addr_region2; + u64 hw_addr_len; + u64 hw_addr_region2_len; + + void *back; + + /* control queue - send and receive */ + struct idpf_ctlq_info *asq; + struct idpf_ctlq_info *arq; + + /* subsystem structs */ + struct idpf_mac_info mac; + struct idpf_bus_info bus; + struct idpf_hw_func_caps func_caps; + + /* pci info */ + u16 device_id; + u16 vendor_id; + u16 subsystem_device_id; + u16 subsystem_vendor_id; + u8 revision_id; + bool adapter_stopped; + + LIST_HEAD_TYPE(list_head, idpf_ctlq_info) cq_list_head; +}; + +int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw, + struct idpf_ctlq_info *cq); + +void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq); + +/* prototype for functions used for dynamic memory allocation */ +void *idpf_alloc_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem, + u64 size); +void idpf_free_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem); +#endif /* _IDPF_CONTROLQ_H_ */ diff --git a/drivers/common/idpf/base/idpf_controlq_api.h b/drivers/common/idpf/base/idpf_controlq_api.h new file mode 100644 index 0000000000..2d79c7f4dd --- /dev/null +++ b/drivers/common/idpf/base/idpf_controlq_api.h @@ -0,0 +1,207 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2001-2022 Intel Corporation + */ + +#ifndef _IDPF_CONTROLQ_API_H_ +#define _IDPF_CONTROLQ_API_H_ + +#ifdef __KERNEL__ +#include "idpf_mem.h" +#else /* !__KERNEL__ */ +#include "idpf_osdep.h" +#endif /* !__KERNEL__ */ + +struct idpf_hw; + +/* Used for queue init, response and events */ +enum idpf_ctlq_type { + IDPF_CTLQ_TYPE_MAILBOX_TX = 0, + IDPF_CTLQ_TYPE_MAILBOX_RX = 1, + IDPF_CTLQ_TYPE_CONFIG_TX = 2, + IDPF_CTLQ_TYPE_CONFIG_RX = 3, + IDPF_CTLQ_TYPE_EVENT_RX = 4, + IDPF_CTLQ_TYPE_RDMA_TX = 5, + IDPF_CTLQ_TYPE_RDMA_RX = 6, + IDPF_CTLQ_TYPE_RDMA_COMPL = 7 +}; + +/* + * Generic Control Queue Structures + */ + +struct idpf_ctlq_reg { + /* used for queue tracking */ + u32 head; + u32 tail; + /* Below applies only to default mb (if present) */ + u32 len; + u32 bah; + u32 bal; + u32 len_mask; + u32 len_ena_mask; + u32 head_mask; +}; + +/* Generic queue msg structure */ +struct idpf_ctlq_msg { + u8 vmvf_type; /* represents the source of the message on recv */ +#define IDPF_VMVF_TYPE_VF 0 +#define IDPF_VMVF_TYPE_VM 1 +#define IDPF_VMVF_TYPE_PF 2 + u8 host_id; + /* 3b field used only when sending a message to peer - to be used in + * combination with target func_id to route the message + */ +#define IDPF_HOST_ID_MASK 0x7 + + u16 opcode; + u16 data_len; /* data_len = 0 when no payload is attached */ + union { + u16 func_id; /* when sending a message */ + u16 status; /* when receiving a message */ + }; + union { + struct { + u32 chnl_retval; + u32 chnl_opcode; + } mbx; + } cookie; + union { +#define IDPF_DIRECT_CTX_SIZE 16 +#define IDPF_INDIRECT_CTX_SIZE 8 + /* 16 bytes of context can be provided or 8 bytes of context + * plus the address of a DMA buffer + */ + u8 direct[IDPF_DIRECT_CTX_SIZE]; + struct { + u8 context[IDPF_INDIRECT_CTX_SIZE]; + struct idpf_dma_mem *payload; + } indirect; + } ctx; +}; + +/* Generic queue info structures */ +/* MB, CONFIG and EVENT q do not have extended info */ +struct idpf_ctlq_create_info { + enum idpf_ctlq_type type; + int id; /* absolute queue offset passed as input + * -1 for default mailbox if present + */ + u16 len; /* Queue length passed as input */ + u16 buf_size; /* buffer size passed as input */ + u64 base_address; /* output, HPA of the Queue start */ + struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */ + + int ext_info_size; + void *ext_info; /* Specific to q type */ +}; + +/* Control Queue information */ +struct idpf_ctlq_info { + LIST_ENTRY_TYPE(idpf_ctlq_info) cq_list; + + enum idpf_ctlq_type cq_type; + int q_id; + idpf_lock cq_lock; /* queue lock + * idpf_lock is defined in OSdep.h + */ + /* used for interrupt processing */ + u16 next_to_use; + u16 next_to_clean; + u16 next_to_post; /* starting descriptor to post buffers + * to after recev + */ + + struct idpf_dma_mem desc_ring; /* descriptor ring memory + * idpf_dma_mem is defined in OSdep.h + */ + union { + struct idpf_dma_mem **rx_buff; + struct idpf_ctlq_msg **tx_msg; + } bi; + + u16 buf_size; /* queue buffer size */ + u16 ring_size; /* Number of descriptors */ + struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */ +}; + +/* PF/VF mailbox commands */ +enum idpf_mbx_opc { + /* idpf_mbq_opc_send_msg_to_pf: + * usage: used by PF or VF to send a message to its CPF + * target: RX queue and function ID of parent PF taken from HW + */ + idpf_mbq_opc_send_msg_to_pf = 0x0801, + + /* idpf_mbq_opc_send_msg_to_vf: + * usage: used by PF to send message to a VF + * target: VF control queue ID must be specified in descriptor + */ + idpf_mbq_opc_send_msg_to_vf = 0x0802, + + /* idpf_mbq_opc_send_msg_to_peer_pf: + * usage: used by any function to send message to any peer PF + * target: RX queue and host of parent PF taken from HW + */ + idpf_mbq_opc_send_msg_to_peer_pf = 0x0803, + + /* idpf_mbq_opc_send_msg_to_peer_drv: + * usage: used by any function to send message to any peer driver + * target: RX queue and target host must be specific in descriptor + */ + idpf_mbq_opc_send_msg_to_peer_drv = 0x0804, +}; + +/* + * API supported for control queue management + */ + +/* Will init all required q including default mb. "q_info" is an array of + * create_info structs equal to the number of control queues to be created. + */ +__rte_internal +int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q, + struct idpf_ctlq_create_info *q_info); + +/* Allocate and initialize a single control queue, which will be added to the + * control queue list; returns a handle to the created control queue + */ +int idpf_ctlq_add(struct idpf_hw *hw, + struct idpf_ctlq_create_info *qinfo, + struct idpf_ctlq_info **cq); + +/* Deinitialize and deallocate a single control queue */ +void idpf_ctlq_remove(struct idpf_hw *hw, + struct idpf_ctlq_info *cq); + +/* Sends messages to HW and will also free the buffer*/ +__rte_internal +int idpf_ctlq_send(struct idpf_hw *hw, + struct idpf_ctlq_info *cq, + u16 num_q_msg, + struct idpf_ctlq_msg q_msg[]); + +/* Receives messages and called by interrupt handler/polling + * initiated by app/process. Also caller is supposed to free the buffers + */ +__rte_internal +int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg, + struct idpf_ctlq_msg *q_msg); + +/* Reclaims send descriptors on HW write back */ +__rte_internal +int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count, + struct idpf_ctlq_msg *msg_status[]); + +/* Indicate RX buffers are done being processed */ +__rte_internal +int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw, + struct idpf_ctlq_info *cq, + u16 *buff_count, + struct idpf_dma_mem **buffs); + +/* Will destroy all q including the default mb */ +__rte_internal +int idpf_ctlq_deinit(struct idpf_hw *hw); + +#endif /* _IDPF_CONTROLQ_API_H_ */ diff --git a/drivers/common/idpf/base/idpf_controlq_setup.c b/drivers/common/idpf/base/idpf_controlq_setup.c new file mode 100644 index 0000000000..3a272b1f8d --- /dev/null +++ b/drivers/common/idpf/base/idpf_controlq_setup.c @@ -0,0 +1,179 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2001-2022 Intel Corporation + */ + + +#include "idpf_controlq.h" + + +/** + * idpf_ctlq_alloc_desc_ring - Allocate Control Queue (CQ) rings + * @hw: pointer to hw struct + * @cq: pointer to the specific Control queue + */ +static int +idpf_ctlq_alloc_desc_ring(struct idpf_hw *hw, + struct idpf_ctlq_info *cq) +{ + size_t size = cq->ring_size * sizeof(struct idpf_ctlq_desc); + + cq->desc_ring.va = idpf_alloc_dma_mem(hw, &cq->desc_ring, size); + if (!cq->desc_ring.va) + return -ENOMEM; + + return 0; +} + +/** + * idpf_ctlq_alloc_bufs - Allocate Control Queue (CQ) buffers + * @hw: pointer to hw struct + * @cq: pointer to the specific Control queue + * + * Allocate the buffer head for all control queues, and if it's a receive + * queue, allocate DMA buffers + */ +static int idpf_ctlq_alloc_bufs(struct idpf_hw *hw, + struct idpf_ctlq_info *cq) +{ + int i = 0; + + /* Do not allocate DMA buffers for transmit queues */ + if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX) + return 0; + + /* We'll be allocating the buffer info memory first, then we can + * allocate the mapped buffers for the event processing + */ + cq->bi.rx_buff = (struct idpf_dma_mem **) + idpf_calloc(hw, cq->ring_size, + sizeof(struct idpf_dma_mem *)); + if (!cq->bi.rx_buff) + return -ENOMEM; + + /* allocate the mapped buffers (except for the last one) */ + for (i = 0; i < cq->ring_size - 1; i++) { + struct idpf_dma_mem *bi; + int num = 1; /* number of idpf_dma_mem to be allocated */ + + cq->bi.rx_buff[i] = (struct idpf_dma_mem *)idpf_calloc(hw, num, + sizeof(struct idpf_dma_mem)); + if (!cq->bi.rx_buff[i]) + goto unwind_alloc_cq_bufs; + + bi = cq->bi.rx_buff[i]; + + bi->va = idpf_alloc_dma_mem(hw, bi, cq->buf_size); + if (!bi->va) { + /* unwind will not free the failed entry */ + idpf_free(hw, cq->bi.rx_buff[i]); + goto unwind_alloc_cq_bufs; + } + } + + return 0; + +unwind_alloc_cq_bufs: + /* don't try to free the one that failed... */ + i--; + for (; i >= 0; i--) { + idpf_free_dma_mem(hw, cq->bi.rx_buff[i]); + idpf_free(hw, cq->bi.rx_buff[i]); + } + idpf_free(hw, cq->bi.rx_buff); + + return -ENOMEM; +} + +/** + * idpf_ctlq_free_desc_ring - Free Control Queue (CQ) rings + * @hw: pointer to hw struct + * @cq: pointer to the specific Control queue + * + * This assumes the posted send buffers have already been cleaned + * and de-allocated + */ +static void idpf_ctlq_free_desc_ring(struct idpf_hw *hw, + struct idpf_ctlq_info *cq) +{ + idpf_free_dma_mem(hw, &cq->desc_ring); +} + +/** + * idpf_ctlq_free_bufs - Free CQ buffer info elements + * @hw: pointer to hw struct + * @cq: pointer to the specific Control queue + * + * Free the DMA buffers for RX queues, and DMA buffer header for both RX and TX + * queues. The upper layers are expected to manage freeing of TX DMA buffers + */ +static void idpf_ctlq_free_bufs(struct idpf_hw *hw, struct idpf_ctlq_info *cq) +{ + void *bi; + + if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX) { + int i; + + /* free DMA buffers for rx queues*/ + for (i = 0; i < cq->ring_size; i++) { + if (cq->bi.rx_buff[i]) { + idpf_free_dma_mem(hw, cq->bi.rx_buff[i]); + idpf_free(hw, cq->bi.rx_buff[i]); + } + } + + bi = (void *)cq->bi.rx_buff; + } else { + bi = (void *)cq->bi.tx_msg; + } + + /* free the buffer header */ + idpf_free(hw, bi); +} + +/** + * idpf_ctlq_dealloc_ring_res - Free memory allocated for control queue + * @hw: pointer to hw struct + * @cq: pointer to the specific Control queue + * + * Free the memory used by the ring, buffers and other related structures + */ +void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq) +{ + /* free ring buffers and the ring itself */ + idpf_ctlq_free_bufs(hw, cq); + idpf_ctlq_free_desc_ring(hw, cq); +} + +/** + * idpf_ctlq_alloc_ring_res - allocate memory for descriptor ring and bufs + * @hw: pointer to hw struct + * @cq: pointer to control queue struct + * + * Do *NOT* hold the lock when calling this as the memory allocation routines + * called are not going to be atomic context safe + */ +int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq) +{ + int ret_code; + + /* verify input for valid configuration */ + if (!cq->ring_size || !cq->buf_size) + return -EINVAL; + + /* allocate the ring memory */ + ret_code = idpf_ctlq_alloc_desc_ring(hw, cq); + if (ret_code) + return ret_code; + + /* allocate buffers in the rings */ + ret_code = idpf_ctlq_alloc_bufs(hw, cq); + if (ret_code) + goto idpf_init_cq_free_ring; + + /* success! */ + return 0; + +idpf_init_cq_free_ring: + idpf_free_dma_mem(hw, &cq->desc_ring); + return ret_code; +} diff --git a/drivers/common/idpf/base/idpf_devids.h b/drivers/common/idpf/base/idpf_devids.h new file mode 100644 index 0000000000..a91eb4e02a --- /dev/null +++ b/drivers/common/idpf/base/idpf_devids.h @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2001-2022 Intel Corporation + */ + +#ifndef _IDPF_DEVIDS_H_ +#define _IDPF_DEVIDS_H_ + +/* Vendor ID */ +#define IDPF_INTEL_VENDOR_ID 0x8086 + +/* Device IDs */ +#define IDPF_DEV_ID_PF 0x1452 +#define IDPF_DEV_ID_VF 0x1889 + + + + +#endif /* _IDPF_DEVIDS_H_ */ diff --git a/drivers/common/idpf/base/idpf_lan_pf_regs.h b/drivers/common/idpf/base/idpf_lan_pf_regs.h new file mode 100644 index 0000000000..3df2347bd7 --- /dev/null +++ b/drivers/common/idpf/base/idpf_lan_pf_regs.h @@ -0,0 +1,134 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2001-2022 Intel Corporation + */ + +#ifndef _IDPF_LAN_PF_REGS_H_ +#define _IDPF_LAN_PF_REGS_H_ + + +/* Receive queues */ +#define PF_QRX_BASE 0x00000000 +#define PF_QRX_TAIL(_QRX) (PF_QRX_BASE + (((_QRX) * 0x1000))) +#define PF_QRX_BUFFQ_BASE 0x03000000 +#define PF_QRX_BUFFQ_TAIL(_QRX) (PF_QRX_BUFFQ_BASE + (((_QRX) * 0x1000))) + +/* Transmit queues */ +#define PF_QTX_BASE 0x05000000 +#define PF_QTX_COMM_DBELL(_DBQM) (PF_QTX_BASE + ((_DBQM) * 0x1000)) + + +/* Control(PF Mailbox) Queue */ +#define PF_FW_BASE 0x08400000 + +#define PF_FW_ARQBAL (PF_FW_BASE) +#define PF_FW_ARQBAH (PF_FW_BASE + 0x4) +#define PF_FW_ARQLEN (PF_FW_BASE + 0x8) +#define PF_FW_ARQLEN_ARQLEN_S 0 +#define PF_FW_ARQLEN_ARQLEN_M MAKEMASK(0x1FFF, PF_FW_ARQLEN_ARQLEN_S) +#define PF_FW_ARQLEN_ARQVFE_S 28 +#define PF_FW_ARQLEN_ARQVFE_M BIT(PF_FW_ARQLEN_ARQVFE_S) +#define PF_FW_ARQLEN_ARQOVFL_S 29 +#define PF_FW_ARQLEN_ARQOVFL_M BIT(PF_FW_ARQLEN_ARQOVFL_S) +#define PF_FW_ARQLEN_ARQCRIT_S 30 +#define PF_FW_ARQLEN_ARQCRIT_M BIT(PF_FW_ARQLEN_ARQCRIT_S) +#define PF_FW_ARQLEN_ARQENABLE_S 31 +#define PF_FW_ARQLEN_ARQENABLE_M BIT(PF_FW_ARQLEN_ARQENABLE_S) +#define PF_FW_ARQH (PF_FW_BASE + 0xC) +#define PF_FW_ARQH_ARQH_S 0 +#define PF_FW_ARQH_ARQH_M MAKEMASK(0x1FFF, PF_FW_ARQH_ARQH_S) +#define PF_FW_ARQT (PF_FW_BASE + 0x10) + +#define PF_FW_ATQBAL (PF_FW_BASE + 0x14) +#define PF_FW_ATQBAH (PF_FW_BASE + 0x18) +#define PF_FW_ATQLEN (PF_FW_BASE + 0x1C) +#define PF_FW_ATQLEN_ATQLEN_S 0 +#define PF_FW_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, PF_FW_ATQLEN_ATQLEN_S) +#define PF_FW_ATQLEN_ATQVFE_S 28 +#define PF_FW_ATQLEN_ATQVFE_M BIT(PF_FW_ATQLEN_ATQVFE_S) +#define PF_FW_ATQLEN_ATQOVFL_S 29 +#define PF_FW_ATQLEN_ATQOVFL_M BIT(PF_FW_ATQLEN_ATQOVFL_S) +#define PF_FW_ATQLEN_ATQCRIT_S 30 +#define PF_FW_ATQLEN_ATQCRIT_M BIT(PF_FW_ATQLEN_ATQCRIT_S) +#define PF_FW_ATQLEN_ATQENABLE_S 31 +#define PF_FW_ATQLEN_ATQENABLE_M BIT(PF_FW_ATQLEN_ATQENABLE_S) +#define PF_FW_ATQH (PF_FW_BASE + 0x20) +#define PF_FW_ATQH_ATQH_S 0 +#define PF_FW_ATQH_ATQH_M MAKEMASK(0x3FF, PF_FW_ATQH_ATQH_S) +#define PF_FW_ATQT (PF_FW_BASE + 0x24) + +/* Interrupts */ +#define PF_GLINT_BASE 0x08900000 +#define PF_GLINT_DYN_CTL(_INT) (PF_GLINT_BASE + ((_INT) * 0x1000)) +#define PF_GLINT_DYN_CTL_INTENA_S 0 +#define PF_GLINT_DYN_CTL_INTENA_M BIT(PF_GLINT_DYN_CTL_INTENA_S) +#define PF_GLINT_DYN_CTL_CLEARPBA_S 1 +#define PF_GLINT_DYN_CTL_CLEARPBA_M BIT(PF_GLINT_DYN_CTL_CLEARPBA_S) +#define PF_GLINT_DYN_CTL_SWINT_TRIG_S 2 +#define PF_GLINT_DYN_CTL_SWINT_TRIG_M BIT(PF_GLINT_DYN_CTL_SWINT_TRIG_S) +#define PF_GLINT_DYN_CTL_ITR_INDX_S 3 +#define PF_GLINT_DYN_CTL_ITR_INDX_M MAKEMASK(0x3, PF_GLINT_DYN_CTL_ITR_INDX_S) +#define PF_GLINT_DYN_CTL_INTERVAL_S 5 +#define PF_GLINT_DYN_CTL_INTERVAL_M BIT(PF_GLINT_DYN_CTL_INTERVAL_S) +#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S 24 +#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S) +#define PF_GLINT_DYN_CTL_SW_ITR_INDX_S 25 +#define PF_GLINT_DYN_CTL_SW_ITR_INDX_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_S) +#define PF_GLINT_DYN_CTL_WB_ON_ITR_S 30 +#define PF_GLINT_DYN_CTL_WB_ON_ITR_M BIT(PF_GLINT_DYN_CTL_WB_ON_ITR_S) +#define PF_GLINT_DYN_CTL_INTENA_MSK_S 31 +#define PF_GLINT_DYN_CTL_INTENA_MSK_M BIT(PF_GLINT_DYN_CTL_INTENA_MSK_S) +#define PF_GLINT_ITR_V2(_i, _reg_start) (((_i) * 4) + (_reg_start)) +#define PF_GLINT_ITR(_i, _INT) (PF_GLINT_BASE + (((_i) + 1) * 4) + ((_INT) * 0x1000)) +#define PF_GLINT_ITR_MAX_INDEX 2 +#define PF_GLINT_ITR_INTERVAL_S 0 +#define PF_GLINT_ITR_INTERVAL_M MAKEMASK(0xFFF, PF_GLINT_ITR_INTERVAL_S) + +/* Timesync registers */ +#define PF_TIMESYNC_BASE 0x08404000 +#define PF_GLTSYN_CMD_SYNC (PF_TIMESYNC_BASE) +#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_S 0 +#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_M MAKEMASK(0x3, PF_GLTSYN_CMD_SYNC_EXEC_CMD_S) +#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_S 2 +#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_M BIT(PF_GLTSYN_CMD_SYNC_SHTIME_EN_S) +#define PF_GLTSYN_SHTIME_0 (PF_TIMESYNC_BASE + 0x4) +#define PF_GLTSYN_SHTIME_L (PF_TIMESYNC_BASE + 0x8) +#define PF_GLTSYN_SHTIME_H (PF_TIMESYNC_BASE + 0xC) +#define PF_GLTSYN_ART_L (PF_TIMESYNC_BASE + 0x10) +#define PF_GLTSYN_ART_H (PF_TIMESYNC_BASE + 0x14) + +/* Generic registers */ +#define PF_INT_DIR_OICR_ENA 0x08406000 +#define PF_INT_DIR_OICR_ENA_S 0 +#define PF_INT_DIR_OICR_ENA_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_ENA_S) +#define PF_INT_DIR_OICR 0x08406004 +#define PF_INT_DIR_OICR_TSYN_EVNT 0 +#define PF_INT_DIR_OICR_PHY_TS_0 BIT(1) +#define PF_INT_DIR_OICR_PHY_TS_1 BIT(2) +#define PF_INT_DIR_OICR_CAUSE 0x08406008 +#define PF_INT_DIR_OICR_CAUSE_CAUSE_S 0 +#define PF_INT_DIR_OICR_CAUSE_CAUSE_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_CAUSE_CAUSE_S) +#define PF_INT_PBA_CLEAR 0x0840600C + +#define PF_FUNC_RID 0x08406010 +#define PF_FUNC_RID_FUNCTION_NUMBER_S 0 +#define PF_FUNC_RID_FUNCTION_NUMBER_M MAKEMASK(0x7, PF_FUNC_RID_FUNCTION_NUMBER_S) +#define PF_FUNC_RID_DEVICE_NUMBER_S 3 +#define PF_FUNC_RID_DEVICE_NUMBER_M MAKEMASK(0x1F, PF_FUNC_RID_DEVICE_NUMBER_S) +#define PF_FUNC_RID_BUS_NUMBER_S 8 +#define PF_FUNC_RID_BUS_NUMBER_M MAKEMASK(0xFF, PF_FUNC_RID_BUS_NUMBER_S) + +/* Reset registers */ +#define PFGEN_RTRIG 0x08407000 +#define PFGEN_RTRIG_CORER_S 0 +#define PFGEN_RTRIG_CORER_M BIT(0) +#define PFGEN_RTRIG_LINKR_S 1 +#define PFGEN_RTRIG_LINKR_M BIT(1) +#define PFGEN_RTRIG_IMCR_S 2 +#define PFGEN_RTRIG_IMCR_M BIT(2) +#define PFGEN_RSTAT 0x08407008 /* PFR Status */ +#define PFGEN_RSTAT_PFR_STATE_S 0 +#define PFGEN_RSTAT_PFR_STATE_M MAKEMASK(0x3, PFGEN_RSTAT_PFR_STATE_S) +#define PFGEN_CTRL 0x0840700C +#define PFGEN_CTRL_PFSWR BIT(0) + +#endif diff --git a/drivers/common/idpf/base/idpf_lan_txrx.h b/drivers/common/idpf/base/idpf_lan_txrx.h new file mode 100644 index 0000000000..98484b267c --- /dev/null +++ b/drivers/common/idpf/base/idpf_lan_txrx.h @@ -0,0 +1,428 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2001-2022 Intel Corporation + */ + +#ifndef _IDPF_LAN_TXRX_H_ +#define _IDPF_LAN_TXRX_H_ +#ifndef __KERNEL__ +#include "idpf_osdep.h" +#endif + +enum idpf_rss_hash { + /* Values 0 - 28 are reserved for future use */ + IDPF_HASH_INVALID = 0, + IDPF_HASH_NONF_UNICAST_IPV4_UDP = 29, + IDPF_HASH_NONF_MULTICAST_IPV4_UDP, + IDPF_HASH_NONF_IPV4_UDP, + IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK, + IDPF_HASH_NONF_IPV4_TCP, + IDPF_HASH_NONF_IPV4_SCTP, + IDPF_HASH_NONF_IPV4_OTHER, + IDPF_HASH_FRAG_IPV4, + /* Values 37-38 are reserved */ + IDPF_HASH_NONF_UNICAST_IPV6_UDP = 39, + IDPF_HASH_NONF_MULTICAST_IPV6_UDP, + IDPF_HASH_NONF_IPV6_UDP, + IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK, + IDPF_HASH_NONF_IPV6_TCP, + IDPF_HASH_NONF_IPV6_SCTP, + IDPF_HASH_NONF_IPV6_OTHER, + IDPF_HASH_FRAG_IPV6, + IDPF_HASH_NONF_RSVD47, + IDPF_HASH_NONF_FCOE_OX, + IDPF_HASH_NONF_FCOE_RX, + IDPF_HASH_NONF_FCOE_OTHER, + /* Values 51-62 are reserved */ + IDPF_HASH_L2_PAYLOAD = 63, + IDPF_HASH_MAX +}; + +/* Supported RSS offloads */ +#define IDPF_DEFAULT_RSS_HASH ( \ + BIT_ULL(IDPF_HASH_NONF_IPV4_UDP) | \ + BIT_ULL(IDPF_HASH_NONF_IPV4_SCTP) | \ + BIT_ULL(IDPF_HASH_NONF_IPV4_TCP) | \ + BIT_ULL(IDPF_HASH_NONF_IPV4_OTHER) | \ + BIT_ULL(IDPF_HASH_FRAG_IPV4) | \ + BIT_ULL(IDPF_HASH_NONF_IPV6_UDP) | \ + BIT_ULL(IDPF_HASH_NONF_IPV6_TCP) | \ + BIT_ULL(IDPF_HASH_NONF_IPV6_SCTP) | \ + BIT_ULL(IDPF_HASH_NONF_IPV6_OTHER) | \ + BIT_ULL(IDPF_HASH_FRAG_IPV6) | \ + BIT_ULL(IDPF_HASH_L2_PAYLOAD)) + + /* TODO: Wrap below comment under internal flag + * Below 6 pcktypes are not supported by FVL or older products + * They are supported by FPK and future products + */ +#define IDPF_DEFAULT_RSS_HASH_EXPANDED (IDPF_DEFAULT_RSS_HASH | \ + BIT_ULL(IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK) | \ + BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV4_UDP) | \ + BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV4_UDP) | \ + BIT_ULL(IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK) | \ + BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV6_UDP) | \ + BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV6_UDP)) + +/* For idpf_splitq_base_tx_compl_desc */ +#define IDPF_TXD_COMPLQ_GEN_S 15 +#define IDPF_TXD_COMPLQ_GEN_M BIT_ULL(IDPF_TXD_COMPLQ_GEN_S) +#define IDPF_TXD_COMPLQ_COMPL_TYPE_S 11 +#define IDPF_TXD_COMPLQ_COMPL_TYPE_M \ + MAKEMASK(0x7UL, IDPF_TXD_COMPLQ_COMPL_TYPE_S) +#define IDPF_TXD_COMPLQ_QID_S 0 +#define IDPF_TXD_COMPLQ_QID_M MAKEMASK(0x3FFUL, IDPF_TXD_COMPLQ_QID_S) + +/* For base mode TX descriptors */ + +#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S 23 +#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_M BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S) +#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_S 19 +#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_M \ + (0xFULL << IDPF_TXD_CTX_QW0_TUNN_DECTTL_S) +#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_S 12 +#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_M \ + (0X7FULL << IDPF_TXD_CTX_QW0_TUNN_NATLEN_S) +#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S 11 +#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M \ + BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S) +#define IDPF_TXD_CTX_EIP_NOINC_IPID_CONST \ + IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M +#define IDPF_TXD_CTX_QW0_TUNN_NATT_S 9 +#define IDPF_TXD_CTX_QW0_TUNN_NATT_M (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S) +#define IDPF_TXD_CTX_UDP_TUNNELING BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_NATT_S) +#define IDPF_TXD_CTX_GRE_TUNNELING (0x2ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S) +#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S 2 +#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_M \ + (0x3FULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S) +#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S 0 +#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_M \ + (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S) + +#define IDPF_TXD_CTX_QW1_MSS_S 50 +#define IDPF_TXD_CTX_QW1_MSS_M \ + MAKEMASK(0x3FFFULL, IDPF_TXD_CTX_QW1_MSS_S) +#define IDPF_TXD_CTX_QW1_TSO_LEN_S 30 +#define IDPF_TXD_CTX_QW1_TSO_LEN_M \ + MAKEMASK(0x3FFFFULL, IDPF_TXD_CTX_QW1_TSO_LEN_S) +#define IDPF_TXD_CTX_QW1_CMD_S 4 +#define IDPF_TXD_CTX_QW1_CMD_M \ + MAKEMASK(0xFFFUL, IDPF_TXD_CTX_QW1_CMD_S) +#define IDPF_TXD_CTX_QW1_DTYPE_S 0 +#define IDPF_TXD_CTX_QW1_DTYPE_M \ + MAKEMASK(0xFUL, IDPF_TXD_CTX_QW1_DTYPE_S) +#define IDPF_TXD_QW1_L2TAG1_S 48 +#define IDPF_TXD_QW1_L2TAG1_M \ + MAKEMASK(0xFFFFULL, IDPF_TXD_QW1_L2TAG1_S) +#define IDPF_TXD_QW1_TX_BUF_SZ_S 34 +#define IDPF_TXD_QW1_TX_BUF_SZ_M \ + MAKEMASK(0x3FFFULL, IDPF_TXD_QW1_TX_BUF_SZ_S) +#define IDPF_TXD_QW1_OFFSET_S 16 +#define IDPF_TXD_QW1_OFFSET_M \ + MAKEMASK(0x3FFFFULL, IDPF_TXD_QW1_OFFSET_S) +#define IDPF_TXD_QW1_CMD_S 4 +#define IDPF_TXD_QW1_CMD_M MAKEMASK(0xFFFUL, IDPF_TXD_QW1_CMD_S) +#define IDPF_TXD_QW1_DTYPE_S 0 +#define IDPF_TXD_QW1_DTYPE_M MAKEMASK(0xFUL, IDPF_TXD_QW1_DTYPE_S) + +/* TX Completion Descriptor Completion Types */ +#define IDPF_TXD_COMPLT_ITR_FLUSH 0 +#define IDPF_TXD_COMPLT_RULE_MISS 1 +#define IDPF_TXD_COMPLT_RS 2 +#define IDPF_TXD_COMPLT_REINJECTED 3 +#define IDPF_TXD_COMPLT_RE 4 +#define IDPF_TXD_COMPLT_SW_MARKER 5 + +enum idpf_tx_desc_dtype_value { + IDPF_TX_DESC_DTYPE_DATA = 0, + IDPF_TX_DESC_DTYPE_CTX = 1, + IDPF_TX_DESC_DTYPE_REINJECT_CTX = 2, + IDPF_TX_DESC_DTYPE_FLEX_DATA = 3, + IDPF_TX_DESC_DTYPE_FLEX_CTX = 4, + IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX = 5, + IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 = 6, + IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 = 7, + IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX = 8, + IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_TSO_CTX = 9, + IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_CTX = 10, + IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX = 11, + IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE = 12, + IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_TSO_CTX = 13, + IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_CTX = 14, + /* DESC_DONE - HW has completed write-back of descriptor */ + IDPF_TX_DESC_DTYPE_DESC_DONE = 15, +}; + +enum idpf_tx_ctx_desc_cmd_bits { + IDPF_TX_CTX_DESC_TSO = 0x01, + IDPF_TX_CTX_DESC_TSYN = 0x02, + IDPF_TX_CTX_DESC_IL2TAG2 = 0x04, + IDPF_TX_CTX_DESC_RSVD = 0x08, + IDPF_TX_CTX_DESC_SWTCH_NOTAG = 0x00, + IDPF_TX_CTX_DESC_SWTCH_UPLINK = 0x10, + IDPF_TX_CTX_DESC_SWTCH_LOCAL = 0x20, + IDPF_TX_CTX_DESC_SWTCH_VSI = 0x30, + IDPF_TX_CTX_DESC_FILT_AU_EN = 0x40, + IDPF_TX_CTX_DESC_FILT_AU_EVICT = 0x80, + IDPF_TX_CTX_DESC_RSVD1 = 0xF00 +}; + +enum idpf_tx_desc_len_fields { + /* Note: These are predefined bit offsets */ + IDPF_TX_DESC_LEN_MACLEN_S = 0, /* 7 BITS */ + IDPF_TX_DESC_LEN_IPLEN_S = 7, /* 7 BITS */ + IDPF_TX_DESC_LEN_L4_LEN_S = 14 /* 4 BITS */ +}; + +#define IDPF_TXD_QW1_MACLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_MACLEN_S) +#define IDPF_TXD_QW1_IPLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_IPLEN_S) +#define IDPF_TXD_QW1_L4LEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S) +#define IDPF_TXD_QW1_FCLEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S) + +enum idpf_tx_base_desc_cmd_bits { + IDPF_TX_DESC_CMD_EOP = 0x0001, + IDPF_TX_DESC_CMD_RS = 0x0002, + /* only on VFs else RSVD */ + IDPF_TX_DESC_CMD_ICRC = 0x0004, + IDPF_TX_DESC_CMD_IL2TAG1 = 0x0008, + IDPF_TX_DESC_CMD_RSVD1 = 0x0010, + IDPF_TX_DESC_CMD_IIPT_NONIP = 0x0000, /* 2 BITS */ + IDPF_TX_DESC_CMD_IIPT_IPV6 = 0x0020, /* 2 BITS */ + IDPF_TX_DESC_CMD_IIPT_IPV4 = 0x0040, /* 2 BITS */ + IDPF_TX_DESC_CMD_IIPT_IPV4_CSUM = 0x0060, /* 2 BITS */ + IDPF_TX_DESC_CMD_RSVD2 = 0x0080, + IDPF_TX_DESC_CMD_L4T_EOFT_UNK = 0x0000, /* 2 BITS */ + IDPF_TX_DESC_CMD_L4T_EOFT_TCP = 0x0100, /* 2 BITS */ + IDPF_TX_DESC_CMD_L4T_EOFT_SCTP = 0x0200, /* 2 BITS */ + IDPF_TX_DESC_CMD_L4T_EOFT_UDP = 0x0300, /* 2 BITS */ + IDPF_TX_DESC_CMD_RSVD3 = 0x0400, + IDPF_TX_DESC_CMD_RSVD4 = 0x0800, +}; + +/* Transmit descriptors */ +/* splitq tx buf, singleq tx buf and singleq compl desc */ +struct idpf_base_tx_desc { + __le64 buf_addr; /* Address of descriptor's data buf */ + __le64 qw1; /* type_cmd_offset_bsz_l2tag1 */ +};/* read used with buffer queues*/ + +struct idpf_splitq_tx_compl_desc { + /* qid=[10:0] comptype=[13:11] rsvd=[14] gen=[15] */ + __le16 qid_comptype_gen; + union { + __le16 q_head; /* Queue head */ + __le16 compl_tag; /* Completion tag */ + } q_head_compl_tag; + u32 rsvd; + +};/* writeback used with completion queues*/ + +/* Context descriptors */ +struct idpf_base_tx_ctx_desc { + struct { + __le32 tunneling_params; + __le16 l2tag2; + __le16 rsvd1; + } qw0; + __le64 qw1; /* type_cmd_tlen_mss/rt_hint */ +}; + +/* Common cmd field defines for all desc except Flex Flow Scheduler (0x0C) */ +enum idpf_tx_flex_desc_cmd_bits { + IDPF_TX_FLEX_DESC_CMD_EOP = 0x01, + IDPF_TX_FLEX_DESC_CMD_RS = 0x02, + IDPF_TX_FLEX_DESC_CMD_RE = 0x04, + IDPF_TX_FLEX_DESC_CMD_IL2TAG1 = 0x08, + IDPF_TX_FLEX_DESC_CMD_DUMMY = 0x10, + IDPF_TX_FLEX_DESC_CMD_CS_EN = 0x20, + IDPF_TX_FLEX_DESC_CMD_FILT_AU_EN = 0x40, + IDPF_TX_FLEX_DESC_CMD_FILT_AU_EVICT = 0x80, +}; + +struct idpf_flex_tx_desc { + __le64 buf_addr; /* Packet buffer address */ + struct { + __le16 cmd_dtype; +#define IDPF_FLEX_TXD_QW1_DTYPE_S 0 +#define IDPF_FLEX_TXD_QW1_DTYPE_M \ + MAKEMASK(0x1FUL, IDPF_FLEX_TXD_QW1_DTYPE_S) +#define IDPF_FLEX_TXD_QW1_CMD_S 5 +#define IDPF_FLEX_TXD_QW1_CMD_M MAKEMASK(0x7FFUL, IDPF_TXD_QW1_CMD_S) + union { + /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_DATA_(0x03) */ + u8 raw[4]; + + /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 (0x06) */ + struct { + __le16 l2tag1; + u8 flex; + u8 tsync; + } tsync; + + /* DTYPE=IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 (0x07) */ + struct { + __le16 l2tag1; + __le16 l2tag2; + } l2tags; + } flex; + __le16 buf_size; + } qw1; +}; + +struct idpf_flex_tx_sched_desc { + __le64 buf_addr; /* Packet buffer address */ + + /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE_16B (0x0C) */ + struct { + u8 cmd_dtype; +#define IDPF_TXD_FLEX_FLOW_DTYPE_M 0x1F +#define IDPF_TXD_FLEX_FLOW_CMD_EOP 0x20 +#define IDPF_TXD_FLEX_FLOW_CMD_CS_EN 0x40 +#define IDPF_TXD_FLEX_FLOW_CMD_RE 0x80 + + u8 rsvd[3]; + + __le16 compl_tag; + __le16 rxr_bufsize; +#define IDPF_TXD_FLEX_FLOW_RXR 0x4000 +#define IDPF_TXD_FLEX_FLOW_BUFSIZE_M 0x3FFF + } qw1; +}; + +/* Common cmd fields for all flex context descriptors + * Note: these defines already account for the 5 bit dtype in the cmd_dtype + * field + */ +enum idpf_tx_flex_ctx_desc_cmd_bits { + IDPF_TX_FLEX_CTX_DESC_CMD_TSO = 0x0020, + IDPF_TX_FLEX_CTX_DESC_CMD_TSYN_EN = 0x0040, + IDPF_TX_FLEX_CTX_DESC_CMD_L2TAG2 = 0x0080, + IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_UPLNK = 0x0200, /* 2 bits */ + IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_LOCAL = 0x0400, /* 2 bits */ + IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_TARGETVSI = 0x0600, /* 2 bits */ +}; + +/* Standard flex descriptor TSO context quad word */ +struct idpf_flex_tx_tso_ctx_qw { + __le32 flex_tlen; +#define IDPF_TXD_FLEX_CTX_TLEN_M 0x3FFFF +#define IDPF_TXD_FLEX_TSO_CTX_FLEX_S 24 + __le16 mss_rt; +#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF + u8 hdr_len; + u8 flex; +}; + +union idpf_flex_tx_ctx_desc { + /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_CTX (0x04) */ + struct { + u8 qw0_flex[8]; + struct { + __le16 cmd_dtype; + __le16 l2tag1; + u8 qw1_flex[4]; + } qw1; + } gen; + + /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX (0x05) */ + struct { + struct idpf_flex_tx_tso_ctx_qw qw0; + struct { + __le16 cmd_dtype; + u8 flex[6]; + } qw1; + } tso; + + /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX (0x08) */ + struct { + struct idpf_flex_tx_tso_ctx_qw qw0; + struct { + __le16 cmd_dtype; + __le16 l2tag2; + u8 flex0; + u8 ptag; + u8 flex1[2]; + } qw1; + } tso_l2tag2_ptag; + + /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX (0x0B) */ + struct { + u8 qw0_flex[8]; + struct { + __le16 cmd_dtype; + __le16 l2tag2; + u8 flex[4]; + } qw1; + } l2tag2; + + /* DTYPE = IDPF_TX_DESC_DTYPE_REINJECT_CTX (0x02) */ + struct { + struct { + __le32 sa_domain; +#define IDPF_TXD_FLEX_CTX_SA_DOM_M 0xFFFF +#define IDPF_TXD_FLEX_CTX_SA_DOM_VAL 0x10000 + __le32 sa_idx; +#define IDPF_TXD_FLEX_CTX_SAIDX_M 0x1FFFFF + } qw0; + struct { + __le16 cmd_dtype; + __le16 txr2comp; +#define IDPF_TXD_FLEX_CTX_TXR2COMP 0x1 + __le16 miss_txq_comp_tag; + __le16 miss_txq_id; + } qw1; + } reinjection_pkt; +}; + +/* Host Split Context Descriptors */ +struct idpf_flex_tx_hs_ctx_desc { + union { + struct { + __le32 host_fnum_tlen; +#define IDPF_TXD_FLEX_CTX_TLEN_S 0 +/* see IDPF_TXD_FLEX_CTX_TLEN_M for mask definition */ +#define IDPF_TXD_FLEX_CTX_FNUM_S 18 +#define IDPF_TXD_FLEX_CTX_FNUM_M 0x7FF +#define IDPF_TXD_FLEX_CTX_HOST_S 29 +#define IDPF_TXD_FLEX_CTX_HOST_M 0x7 + __le16 ftype_mss_rt; +#define IDPF_TXD_FLEX_CTX_MSS_RT_0 0 +#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF +#define IDPF_TXD_FLEX_CTX_FTYPE_S 14 +#define IDPF_TXD_FLEX_CTX_FTYPE_VF MAKEMASK(0x0, IDPF_TXD_FLEX_CTX_FTYPE_S) +#define IDPF_TXD_FLEX_CTX_FTYPE_VDEV MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_FTYPE_S) +#define IDPF_TXD_FLEX_CTX_FTYPE_PF MAKEMASK(0x2, IDPF_TXD_FLEX_CTX_FTYPE_S) + u8 hdr_len; + u8 ptag; + } tso; + struct { + u8 flex0[2]; + __le16 host_fnum_ftype; + u8 flex1[3]; + u8 ptag; + } no_tso; + } qw0; + + __le64 qw1_cmd_dtype; +#define IDPF_TXD_FLEX_CTX_QW1_PASID_S 16 +#define IDPF_TXD_FLEX_CTX_QW1_PASID_M 0xFFFFF +#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID_S 36 +#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID \ + MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_PASID_VALID_S) +#define IDPF_TXD_FLEX_CTX_QW1_TPH_S 37 +#define IDPF_TXD_FLEX_CTX_QW1_TPH \ + MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_TPH_S) +#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_S 38 +#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_M 0xF +/* The following are only valid for DTYPE = 0x09 and DTYPE = 0x0A */ +#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_S 42 +#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_M 0x1FFFFF +#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S 63 +#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VALID \ + MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S) +/* The following are only valid for DTYPE = 0x0D and DTYPE = 0x0E */ +#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_S 48 +#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_M 0xFF +#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_S 56 +#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_M 0xFF +}; +#endif /* _IDPF_LAN_TXRX_H_ */ diff --git a/drivers/common/idpf/base/idpf_lan_vf_regs.h b/drivers/common/idpf/base/idpf_lan_vf_regs.h new file mode 100644 index 0000000000..9cd4f757d9 --- /dev/null +++ b/drivers/common/idpf/base/idpf_lan_vf_regs.h @@ -0,0 +1,114 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2001-2022 Intel Corporation + */ + +#ifndef _IDPF_LAN_VF_REGS_H_ +#define _IDPF_LAN_VF_REGS_H_ + + +/* Reset */ +#define VFGEN_RSTAT 0x00008800 +#define VFGEN_RSTAT_VFR_STATE_S 0 +#define VFGEN_RSTAT_VFR_STATE_M MAKEMASK(0x3, VFGEN_RSTAT_VFR_STATE_S) + +/* Control(VF Mailbox) Queue */ +#define VF_BASE 0x00006000 + +#define VF_ATQBAL (VF_BASE + 0x1C00) +#define VF_ATQBAH (VF_BASE + 0x1800) +#define VF_ATQLEN (VF_BASE + 0x0800) +#define VF_ATQLEN_ATQLEN_S 0 +#define VF_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, VF_ATQLEN_ATQLEN_S) +#define VF_ATQLEN_ATQVFE_S 28 +#define VF_ATQLEN_ATQVFE_M BIT(VF_ATQLEN_ATQVFE_S) +#define VF_ATQLEN_ATQOVFL_S 29 +#define VF_ATQLEN_ATQOVFL_M BIT(VF_ATQLEN_ATQOVFL_S) +#define VF_ATQLEN_ATQCRIT_S 30 +#define VF_ATQLEN_ATQCRIT_M BIT(VF_ATQLEN_ATQCRIT_S) +#define VF_ATQLEN_ATQENABLE_S 31 +#define VF_ATQLEN_ATQENABLE_M BIT(VF_ATQLEN_ATQENABLE_S) +#define VF_ATQH (VF_BASE + 0x0400) +#define VF_ATQH_ATQH_S 0 +#define VF_ATQH_ATQH_M MAKEMASK(0x3FF, VF_ATQH_ATQH_S) +#define VF_ATQT (VF_BASE + 0x2400) + +#define VF_ARQBAL (VF_BASE + 0x0C00) +#define VF_ARQBAH (VF_BASE) +#define VF_ARQLEN (VF_BASE + 0x2000) +#define VF_ARQLEN_ARQLEN_S 0 +#define VF_ARQLEN_ARQLEN_M MAKEMASK(0x3FF, VF_ARQLEN_ARQLEN_S) +#define VF_ARQLEN_ARQVFE_S 28 +#define VF_ARQLEN_ARQVFE_M BIT(VF_ARQLEN_ARQVFE_S) +#define VF_ARQLEN_ARQOVFL_S 29 +#define VF_ARQLEN_ARQOVFL_M BIT(VF_ARQLEN_ARQOVFL_S) +#define VF_ARQLEN_ARQCRIT_S 30 +#define VF_ARQLEN_ARQCRIT_M BIT(VF_ARQLEN_ARQCRIT_S) +#define VF_ARQLEN_ARQENABLE_S 31 +#define VF_ARQLEN_ARQENABLE_M BIT(VF_ARQLEN_ARQENABLE_S) +#define VF_ARQH (VF_BASE + 0x1400) +#define VF_ARQH_ARQH_S 0 +#define VF_ARQH_ARQH_M MAKEMASK(0x1FFF, VF_ARQH_ARQH_S) +#define VF_ARQT (VF_BASE + 0x1000) + +/* Transmit queues */ +#define VF_QTX_TAIL_BASE 0x00000000 +#define VF_QTX_TAIL(_QTX) (VF_QTX_TAIL_BASE + (_QTX) * 0x4) +#define VF_QTX_TAIL_EXT_BASE 0x00040000 +#define VF_QTX_TAIL_EXT(_QTX) (VF_QTX_TAIL_EXT_BASE + ((_QTX) * 4)) + +/* Receive queues */ +#define VF_QRX_TAIL_BASE 0x00002000 +#define VF_QRX_TAIL(_QRX) (VF_QRX_TAIL_BASE + ((_QRX) * 4)) +#define VF_QRX_TAIL_EXT_BASE 0x00050000 +#define VF_QRX_TAIL_EXT(_QRX) (VF_QRX_TAIL_EXT_BASE + ((_QRX) * 4)) +#define VF_QRXB_TAIL_BASE 0x00060000 +#define VF_QRXB_TAIL(_QRX) (VF_QRXB_TAIL_BASE + ((_QRX) * 4)) + +/* Interrupts */ +#define VF_INT_DYN_CTL0 0x00005C00 +#define VF_INT_DYN_CTL0_INTENA_S 0 +#define VF_INT_DYN_CTL0_INTENA_M BIT(VF_INT_DYN_CTL0_INTENA_S) +#define VF_INT_DYN_CTL0_ITR_INDX_S 3 +#define VF_INT_DYN_CTL0_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTL0_ITR_INDX_S) +#define VF_INT_DYN_CTLN(_INT) (0x00003800 + ((_INT) * 4)) +#define VF_INT_DYN_CTLN_EXT(_INT) (0x00070000 + ((_INT) * 4)) +#define VF_INT_DYN_CTLN_INTENA_S 0 +#define VF_INT_DYN_CTLN_INTENA_M BIT(VF_INT_DYN_CTLN_INTENA_S) +#define VF_INT_DYN_CTLN_CLEARPBA_S 1 +#define VF_INT_DYN_CTLN_CLEARPBA_M BIT(VF_INT_DYN_CTLN_CLEARPBA_S) +#define VF_INT_DYN_CTLN_SWINT_TRIG_S 2 +#define VF_INT_DYN_CTLN_SWINT_TRIG_M BIT(VF_INT_DYN_CTLN_SWINT_TRIG_S) +#define VF_INT_DYN_CTLN_ITR_INDX_S 3 +#define VF_INT_DYN_CTLN_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTLN_ITR_INDX_S) +#define VF_INT_DYN_CTLN_INTERVAL_S 5 +#define VF_INT_DYN_CTLN_INTERVAL_M BIT(VF_INT_DYN_CTLN_INTERVAL_S) +#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S 24 +#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S) +#define VF_INT_DYN_CTLN_SW_ITR_INDX_S 25 +#define VF_INT_DYN_CTLN_SW_ITR_INDX_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_S) +#define VF_INT_DYN_CTLN_WB_ON_ITR_S 30 +#define VF_INT_DYN_CTLN_WB_ON_ITR_M BIT(VF_INT_DYN_CTLN_WB_ON_ITR_S) +#define VF_INT_DYN_CTLN_INTENA_MSK_S 31 +#define VF_INT_DYN_CTLN_INTENA_MSK_M BIT(VF_INT_DYN_CTLN_INTENA_MSK_S) +#define VF_INT_ITR0(_i) (0x00004C00 + ((_i) * 4)) +#define VF_INT_ITRN_V2(_i, _reg_start) ((_reg_start) + (((_i)) * 4)) +#define VF_INT_ITRN(_i, _INT) (0x00002800 + ((_i) * 4) + ((_INT) * 0x40)) +#define VF_INT_ITRN_64(_i, _INT) (0x00002C00 + ((_i) * 4) + ((_INT) * 0x100)) +#define VF_INT_ITRN_2K(_i, _INT) (0x00072000 + ((_i) * 4) + ((_INT) * 0x100)) +#define VF_INT_ITRN_MAX_INDEX 2 +#define VF_INT_ITRN_INTERVAL_S 0 +#define VF_INT_ITRN_INTERVAL_M MAKEMASK(0xFFF, VF_INT_ITRN_INTERVAL_S) +#define VF_INT_PBA_CLEAR 0x00008900 + +#define VF_INT_ICR0_ENA1 0x00005000 +#define VF_INT_ICR0_ENA1_ADMINQ_S 30 +#define VF_INT_ICR0_ENA1_ADMINQ_M BIT(VF_INT_ICR0_ENA1_ADMINQ_S) +#define VF_INT_ICR0_ENA1_RSVD_S 31 +#define VF_INT_ICR01 0x00004800 +#define VF_QF_HENA(_i) (0x0000C400 + ((_i) * 4)) +#define VF_QF_HENA_MAX_INDX 1 +#define VF_QF_HKEY(_i) (0x0000CC00 + ((_i) * 4)) +#define VF_QF_HKEY_MAX_INDX 12 +#define VF_QF_HLUT(_i) (0x0000D000 + ((_i) * 4)) +#define VF_QF_HLUT_MAX_INDX 15 +#endif diff --git a/drivers/common/idpf/base/idpf_osdep.h b/drivers/common/idpf/base/idpf_osdep.h new file mode 100644 index 0000000000..99ae9cf60a --- /dev/null +++ b/drivers/common/idpf/base/idpf_osdep.h @@ -0,0 +1,364 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2001-2022 Intel Corporation + */ + +#ifndef _IDPF_OSDEP_H_ +#define _IDPF_OSDEP_H_ + +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define INLINE inline +#define STATIC static + +typedef uint8_t u8; +typedef int8_t s8; +typedef uint16_t u16; +typedef int16_t s16; +typedef uint32_t u32; +typedef int32_t s32; +typedef uint64_t u64; +typedef uint64_t s64; + +typedef struct idpf_lock idpf_lock; + +#define __iomem +#define hw_dbg(hw, S, A...) do {} while (0) +#define upper_32_bits(n) ((u32)(((n) >> 16) >> 16)) +#define lower_32_bits(n) ((u32)(n)) +#define low_16_bits(x) ((x) & 0xFFFF) +#define high_16_bits(x) (((x) & 0xFFFF0000) >> 16) + +#ifndef ETH_ADDR_LEN +#define ETH_ADDR_LEN 6 +#endif + +#ifndef __le16 +#define __le16 uint16_t +#endif +#ifndef __le32 +#define __le32 uint32_t +#endif +#ifndef __le64 +#define __le64 uint64_t +#endif +#ifndef __be16 +#define __be16 uint16_t +#endif +#ifndef __be32 +#define __be32 uint32_t +#endif +#ifndef __be64 +#define __be64 uint64_t +#endif + +#ifndef BIT_ULL +#define BIT_ULL(a) RTE_BIT64(a) +#endif + +#ifndef BIT +#define BIT(a) RTE_BIT32(a) +#endif + +#define FALSE 0 +#define TRUE 1 +#define false 0 +#define true 1 + +/* Avoid macro redefinition warning on Windows */ +#ifdef RTE_EXEC_ENV_WINDOWS +#ifdef min +#undef min +#endif +#ifdef max +#undef max +#endif +#endif + +#define min(a, b) RTE_MIN(a, b) +#define max(a, b) RTE_MAX(a, b) + +#define ARRAY_SIZE(arr) RTE_DIM(arr) +#define FIELD_SIZEOF(t, f) (sizeof(((t *)0)->(f))) +#define MAKEMASK(m, s) ((m) << (s)) + +extern int idpf_common_logger; + +#define DEBUGOUT(S) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S) +#define DEBUGOUT2(S, A...) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S, ##A) +#define DEBUGFUNC(F) DEBUGOUT(F "\n") + +#define idpf_debug(h, m, s, ...) \ + do { \ + if (((m) & (h)->debug_mask)) \ + PMD_DRV_LOG_RAW(DEBUG, "idpf %02x.%x " s, \ + (h)->bus.device, (h)->bus.func, \ + ##__VA_ARGS__); \ + } while (0) + +#define idpf_info(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args) +#define idpf_warn(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args) +#define idpf_debug_array(hw, type, rowsize, groupsize, buf, len) \ + do { \ + struct idpf_hw *hw_l = hw; \ + u16 len_l = len; \ + u8 *buf_l = buf; \ + int i; \ + for (i = 0; i < len_l; i += 8) \ + idpf_debug(hw_l, type, \ + "0x%04X 0x%016"PRIx64"\n", \ + i, *((u64 *)((buf_l) + i))); \ + } while (0) +#define idpf_snprintf snprintf +#ifndef SNPRINTF +#define SNPRINTF idpf_snprintf +#endif + +#define IDPF_PCI_REG(reg) rte_read32(reg) +#define IDPF_PCI_REG_ADDR(a, reg) \ + ((volatile uint32_t *)((char *)(a)->hw_addr + (reg))) +#define IDPF_PCI_REG64(reg) rte_read64(reg) +#define IDPF_PCI_REG_ADDR64(a, reg) \ + ((volatile uint64_t *)((char *)(a)->hw_addr + (reg))) + +#define idpf_wmb() rte_io_wmb() +#define idpf_rmb() rte_io_rmb() +#define idpf_mb() rte_io_mb() + +static inline uint32_t idpf_read_addr(volatile void *addr) +{ + return rte_le_to_cpu_32(IDPF_PCI_REG(addr)); +} + +static inline uint64_t idpf_read_addr64(volatile void *addr) +{ + return rte_le_to_cpu_64(IDPF_PCI_REG64(addr)); +} + +#define IDPF_PCI_REG_WRITE(reg, value) \ + rte_write32((rte_cpu_to_le_32(value)), reg) + +#define IDPF_PCI_REG_WRITE64(reg, value) \ + rte_write64((rte_cpu_to_le_64(value)), reg) + +#define IDPF_READ_REG(hw, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((hw), (reg))) +#define IDPF_WRITE_REG(hw, reg, value) \ + IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((hw), (reg)), (value)) + +#define rd32(a, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((a), (reg))) +#define wr32(a, reg, value) \ + IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((a), (reg)), (value)) +#define div64_long(n, d) ((n) / (d)) +#define rd64(a, reg) idpf_read_addr64(IDPF_PCI_REG_ADDR64((a), (reg))) + +#define BITS_PER_BYTE 8 + +/* memory allocation tracking */ +struct idpf_dma_mem { + void *va; + u64 pa; + u32 size; + const void *zone; +} __rte_packed; + +struct idpf_virt_mem { + void *va; + u32 size; +} __rte_packed; + +#define idpf_malloc(h, s) rte_zmalloc(NULL, s, 0) +#define idpf_calloc(h, c, s) rte_zmalloc(NULL, (c) * (s), 0) +#define idpf_free(h, m) rte_free(m) + +#define idpf_memset(a, b, c, d) memset((a), (b), (c)) +#define idpf_memcpy(a, b, c, d) rte_memcpy((a), (b), (c)) +#define idpf_memdup(a, b, c, d) rte_memcpy(idpf_malloc(a, c), b, c) + +#define CPU_TO_BE16(o) rte_cpu_to_be_16(o) +#define CPU_TO_BE32(o) rte_cpu_to_be_32(o) +#define CPU_TO_BE64(o) rte_cpu_to_be_64(o) +#define CPU_TO_LE16(o) rte_cpu_to_le_16(o) +#define CPU_TO_LE32(s) rte_cpu_to_le_32(s) +#define CPU_TO_LE64(h) rte_cpu_to_le_64(h) +#define LE16_TO_CPU(a) rte_le_to_cpu_16(a) +#define LE32_TO_CPU(c) rte_le_to_cpu_32(c) +#define LE64_TO_CPU(k) rte_le_to_cpu_64(k) + +#define NTOHS(a) rte_be_to_cpu_16(a) +#define NTOHL(a) rte_be_to_cpu_32(a) +#define HTONS(a) rte_cpu_to_be_16(a) +#define HTONL(a) rte_cpu_to_be_32(a) + +/* SW spinlock */ +struct idpf_lock { + rte_spinlock_t spinlock; +}; + +static inline void +idpf_init_lock(struct idpf_lock *sp) +{ + rte_spinlock_init(&sp->spinlock); +} + +static inline void +idpf_acquire_lock(struct idpf_lock *sp) +{ + rte_spinlock_lock(&sp->spinlock); +} + +static inline void +idpf_release_lock(struct idpf_lock *sp) +{ + rte_spinlock_unlock(&sp->spinlock); +} + +static inline void +idpf_destroy_lock(__rte_unused struct idpf_lock *sp) +{ +} + +struct idpf_hw; + +static inline void * +idpf_alloc_dma_mem(__rte_unused struct idpf_hw *hw, + struct idpf_dma_mem *mem, u64 size) +{ + const struct rte_memzone *mz = NULL; + char z_name[RTE_MEMZONE_NAMESIZE]; + + if (!mem) + return NULL; + + snprintf(z_name, sizeof(z_name), "idpf_dma_%"PRIu64, rte_rand()); + mz = rte_memzone_reserve_aligned(z_name, size, SOCKET_ID_ANY, + RTE_MEMZONE_IOVA_CONTIG, RTE_PGSIZE_4K); + if (!mz) + return NULL; + + mem->size = size; + mem->va = mz->addr; + mem->pa = mz->iova; + mem->zone = (const void *)mz; + memset(mem->va, 0, size); + + return mem->va; +} + +static inline void +idpf_free_dma_mem(__rte_unused struct idpf_hw *hw, + struct idpf_dma_mem *mem) +{ + rte_memzone_free((const struct rte_memzone *)mem->zone); + mem->size = 0; + mem->va = NULL; + mem->pa = 0; +} + +static inline u8 +idpf_hweight8(u32 num) +{ + u8 bits = 0; + u32 i; + + for (i = 0; i < 8; i++) { + bits += (u8)(num & 0x1); + num >>= 1; + } + + return bits; +} + +static inline u8 +idpf_hweight32(u32 num) +{ + u8 bits = 0; + u32 i; + + for (i = 0; i < 32; i++) { + bits += (u8)(num & 0x1); + num >>= 1; + } + + return bits; +} + +#define DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d)) +#define DELAY(x) rte_delay_us(x) +#define idpf_usec_delay(x) rte_delay_us(x) +#define idpf_msec_delay(x, y) rte_delay_us(1000 * (x)) +#define udelay(x) DELAY(x) +#define msleep(x) DELAY(1000 * (x)) +#define usleep_range(min, max) msleep(DIV_ROUND_UP(min, 1000)) + +#ifndef IDPF_DBG_TRACE +#define IDPF_DBG_TRACE BIT_ULL(0) +#endif + +#ifndef DIVIDE_AND_ROUND_UP +#define DIVIDE_AND_ROUND_UP(a, b) (((a) + (b) - 1) / (b)) +#endif + +#ifndef IDPF_INTEL_VENDOR_ID +#define IDPF_INTEL_VENDOR_ID 0x8086 +#endif + +#ifndef IS_UNICAST_ETHER_ADDR +#define IS_UNICAST_ETHER_ADDR(addr) \ + ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 0)) +#endif + +#ifndef IS_MULTICAST_ETHER_ADDR +#define IS_MULTICAST_ETHER_ADDR(addr) \ + ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 1)) +#endif + +#ifndef IS_BROADCAST_ETHER_ADDR +/* Check whether an address is broadcast. */ +#define IS_BROADCAST_ETHER_ADDR(addr) \ + ((bool)((((u16 *)(addr))[0] == ((u16)0xffff)))) +#endif + +#ifndef IS_ZERO_ETHER_ADDR +#define IS_ZERO_ETHER_ADDR(addr) \ + (((bool)((((u16 *)(addr))[0] == ((u16)0x0)))) && \ + ((bool)((((u16 *)(addr))[1] == ((u16)0x0)))) && \ + ((bool)((((u16 *)(addr))[2] == ((u16)0x0))))) +#endif + +#ifndef LIST_HEAD_TYPE +#define LIST_HEAD_TYPE(list_name, type) LIST_HEAD(list_name, type) +#endif + +#ifndef LIST_ENTRY_TYPE +#define LIST_ENTRY_TYPE(type) LIST_ENTRY(type) +#endif + +#ifndef LIST_FOR_EACH_ENTRY_SAFE +#define LIST_FOR_EACH_ENTRY_SAFE(pos, temp, head, entry_type, list) \ + LIST_FOREACH(pos, head, list) + +#endif + +#ifndef LIST_FOR_EACH_ENTRY +#define LIST_FOR_EACH_ENTRY(pos, head, entry_type, list) \ + LIST_FOREACH(pos, head, list) + +#endif + +#endif /* _IDPF_OSDEP_H_ */ diff --git a/drivers/common/idpf/base/idpf_prototype.h b/drivers/common/idpf/base/idpf_prototype.h new file mode 100644 index 0000000000..529b62212d --- /dev/null +++ b/drivers/common/idpf/base/idpf_prototype.h @@ -0,0 +1,45 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2001-2022 Intel Corporation + */ + +#ifndef _IDPF_PROTOTYPE_H_ +#define _IDPF_PROTOTYPE_H_ + +/* Include generic macros and types first */ +#include "idpf_osdep.h" +#include "idpf_controlq.h" +#include "idpf_type.h" +#include "idpf_alloc.h" +#include "idpf_devids.h" +#include "idpf_controlq_api.h" +#include "idpf_lan_pf_regs.h" +#include "idpf_lan_vf_regs.h" +#include "idpf_lan_txrx.h" +#include "virtchnl.h" + +#define APF + +int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size); +int idpf_deinit_hw(struct idpf_hw *hw); + +int idpf_clean_arq_element(struct idpf_hw *hw, + struct idpf_arq_event_info *e, + u16 *events_pending); +bool idpf_asq_done(struct idpf_hw *hw); +bool idpf_check_asq_alive(struct idpf_hw *hw); + +int idpf_get_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut, + u8 *lut, u16 lut_size); +int idpf_set_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut, + u8 *lut, u16 lut_size); +int idpf_get_rss_key(struct idpf_hw *hw, u16 seid, + struct idpf_get_set_rss_key_data *key); +int idpf_set_rss_key(struct idpf_hw *hw, u16 seid, + struct idpf_get_set_rss_key_data *key); + +int idpf_set_mac_type(struct idpf_hw *hw); + +int idpf_reset(struct idpf_hw *hw); +int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode, + int v_retval, u8 *msg, u16 msglen); +#endif /* _IDPF_PROTOTYPE_H_ */ diff --git a/drivers/common/idpf/base/idpf_type.h b/drivers/common/idpf/base/idpf_type.h new file mode 100644 index 0000000000..3b46536287 --- /dev/null +++ b/drivers/common/idpf/base/idpf_type.h @@ -0,0 +1,106 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2001-2022 Intel Corporation + */ + +#ifndef _IDPF_TYPE_H_ +#define _IDPF_TYPE_H_ + +#include "idpf_controlq.h" + +#define UNREFERENCED_XPARAMETER +#define UNREFERENCED_1PARAMETER(_p) +#define UNREFERENCED_2PARAMETER(_p, _q) +#define UNREFERENCED_3PARAMETER(_p, _q, _r) +#define UNREFERENCED_4PARAMETER(_p, _q, _r, _s) +#define UNREFERENCED_5PARAMETER(_p, _q, _r, _s, _t) + +#define MAKEMASK(m, s) ((m) << (s)) + +struct idpf_eth_stats { + u64 rx_bytes; /* gorc */ + u64 rx_unicast; /* uprc */ + u64 rx_multicast; /* mprc */ + u64 rx_broadcast; /* bprc */ + u64 rx_discards; /* rdpc */ + u64 rx_unknown_protocol; /* rupp */ + u64 tx_bytes; /* gotc */ + u64 tx_unicast; /* uptc */ + u64 tx_multicast; /* mptc */ + u64 tx_broadcast; /* bptc */ + u64 tx_discards; /* tdpc */ + u64 tx_errors; /* tepc */ +}; + +/* Statistics collected by the MAC */ +struct idpf_hw_port_stats { + /* eth stats collected by the port */ + struct idpf_eth_stats eth; + + /* additional port specific stats */ + u64 tx_dropped_link_down; /* tdold */ + u64 crc_errors; /* crcerrs */ + u64 illegal_bytes; /* illerrc */ + u64 error_bytes; /* errbc */ + u64 mac_local_faults; /* mlfc */ + u64 mac_remote_faults; /* mrfc */ + u64 rx_length_errors; /* rlec */ + u64 link_xon_rx; /* lxonrxc */ + u64 link_xoff_rx; /* lxoffrxc */ + u64 priority_xon_rx[8]; /* pxonrxc[8] */ + u64 priority_xoff_rx[8]; /* pxoffrxc[8] */ + u64 link_xon_tx; /* lxontxc */ + u64 link_xoff_tx; /* lxofftxc */ + u64 priority_xon_tx[8]; /* pxontxc[8] */ + u64 priority_xoff_tx[8]; /* pxofftxc[8] */ + u64 priority_xon_2_xoff[8]; /* pxon2offc[8] */ + u64 rx_size_64; /* prc64 */ + u64 rx_size_127; /* prc127 */ + u64 rx_size_255; /* prc255 */ + u64 rx_size_511; /* prc511 */ + u64 rx_size_1023; /* prc1023 */ + u64 rx_size_1522; /* prc1522 */ + u64 rx_size_big; /* prc9522 */ + u64 rx_undersize; /* ruc */ + u64 rx_fragments; /* rfc */ + u64 rx_oversize; /* roc */ + u64 rx_jabber; /* rjc */ + u64 tx_size_64; /* ptc64 */ + u64 tx_size_127; /* ptc127 */ + u64 tx_size_255; /* ptc255 */ + u64 tx_size_511; /* ptc511 */ + u64 tx_size_1023; /* ptc1023 */ + u64 tx_size_1522; /* ptc1522 */ + u64 tx_size_big; /* ptc9522 */ + u64 mac_short_packet_dropped; /* mspdc */ + u64 checksum_error; /* xec */ +}; +/* Static buffer size to initialize control queue */ +struct idpf_ctlq_size { + u16 asq_buf_size; + u16 asq_ring_size; + u16 arq_buf_size; + u16 arq_ring_size; +}; + +/* Temporary definition to compile - TBD if needed */ +struct idpf_arq_event_info { + struct idpf_ctlq_desc desc; + u16 msg_len; + u16 buf_len; + u8 *msg_buf; +}; + +struct idpf_get_set_rss_key_data { + u8 standard_rss_key[0x28]; + u8 extended_hash_key[0xc]; +}; + +struct idpf_aq_get_phy_abilities_resp { + __le32 phy_type; +}; + +struct idpf_filter_program_desc { + __le32 qid; +}; + +#endif /* _IDPF_TYPE_H_ */ diff --git a/drivers/common/idpf/base/meson.build b/drivers/common/idpf/base/meson.build new file mode 100644 index 0000000000..ce2b5346e4 --- /dev/null +++ b/drivers/common/idpf/base/meson.build @@ -0,0 +1,14 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2022 Intel Corporation + +sources = files( + 'idpf_common.c', + 'idpf_controlq.c', + 'idpf_controlq_setup.c', +) + +cflags += ['-Wno-unused-value'] +cflags += ['-Wno-unused-variable'] +cflags += ['-Wno-unused-parameter'] +cflags += ['-Wno-implicit-fallthrough'] +cflags += ['-Wno-strict-aliasing'] \ No newline at end of file diff --git a/drivers/common/idpf/base/siov_regs.h b/drivers/common/idpf/base/siov_regs.h new file mode 100644 index 0000000000..3ac4f8f177 --- /dev/null +++ b/drivers/common/idpf/base/siov_regs.h @@ -0,0 +1,47 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2001-2022 Intel Corporation + */ +#ifndef _SIOV_REGS_H_ +#define _SIOV_REGS_H_ +#define VDEV_MBX_START 0x20000 /* Begin at 128KB */ +#define VDEV_MBX_ATQBAL (VDEV_MBX_START + 0x0000) +#define VDEV_MBX_ATQBAH (VDEV_MBX_START + 0x0004) +#define VDEV_MBX_ATQLEN (VDEV_MBX_START + 0x0008) +#define VDEV_MBX_ATQH (VDEV_MBX_START + 0x000C) +#define VDEV_MBX_ATQT (VDEV_MBX_START + 0x0010) +#define VDEV_MBX_ARQBAL (VDEV_MBX_START + 0x0014) +#define VDEV_MBX_ARQBAH (VDEV_MBX_START + 0x0018) +#define VDEV_MBX_ARQLEN (VDEV_MBX_START + 0x001C) +#define VDEV_MBX_ARQH (VDEV_MBX_START + 0x0020) +#define VDEV_MBX_ARQT (VDEV_MBX_START + 0x0024) +#define VDEV_GET_RSTAT 0x21000 /* 132KB for RSTAT */ + +/* Begin at offset after 1MB (after 256 4k pages) */ +#define VDEV_QRX_TAIL_START 0x100000 +#define VDEV_QRX_TAIL(_i) (VDEV_QRX_TAIL_START + ((_i) * 0x1000)) /* 2k Rx queues */ + +/* Begin at offset of 9MB for Rx buffer queue tail register pages */ +#define VDEV_QRX_BUFQ_TAIL_START 0x900000 +/* 2k Rx buffer queues */ +#define VDEV_QRX_BUFQ_TAIL(_i) (VDEV_QRX_BUFQ_TAIL_START + ((_i) * 0x1000)) + +/* Begin at offset of 17MB for 2k Tx queues */ +#define VDEV_QTX_TAIL_START 0x1100000 +#define VDEV_QTX_TAIL(_i) (VDEV_QTX_TAIL_START + ((_i) * 0x1000)) /* 2k Tx queues */ + +/* Begin at offset of 25MB for 2k Tx completion queues */ +#define VDEV_QTX_COMPL_TAIL_START 0x1900000 +/* 2k Tx completion queues */ +#define VDEV_QTX_COMPL_TAIL(_i) (VDEV_QTX_COMPL_TAIL_START + ((_i) * 0x1000)) + +#define VDEV_INT_DYN_CTL01 0x2100000 /* Begin at offset 33MB */ + +/* Begin at offset of 33MB + 4k to accommodate CTL01 register */ +#define VDEV_INT_DYN_START (VDEV_INT_DYN_CTL01 + 0x1000) +#define VDEV_INT_DYN_CTL(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000)) +#define VDEV_INT_ITR_0(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x04) +#define VDEV_INT_ITR_1(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x08) +#define VDEV_INT_ITR_2(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x0C) + +/* Next offset to begin at 42MB (0x2A00000) */ +#endif /* _SIOV_REGS_H_ */ diff --git a/drivers/common/idpf/base/virtchnl.h b/drivers/common/idpf/base/virtchnl.h new file mode 100644 index 0000000000..ea798e3971 --- /dev/null +++ b/drivers/common/idpf/base/virtchnl.h @@ -0,0 +1,2866 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2001-2022 Intel Corporation + */ + +#ifndef _VIRTCHNL_H_ +#define _VIRTCHNL_H_ + +/* Description: + * This header file describes the Virtual Function (VF) - Physical Function + * (PF) communication protocol used by the drivers for all devices starting + * from our 40G product line + * + * Admin queue buffer usage: + * desc->opcode is always aqc_opc_send_msg_to_pf + * flags, retval, datalen, and data addr are all used normally. + * The Firmware copies the cookie fields when sending messages between the + * PF and VF, but uses all other fields internally. Due to this limitation, + * we must send all messages as "indirect", i.e. using an external buffer. + * + * All the VSI indexes are relative to the VF. Each VF can have maximum of + * three VSIs. All the queue indexes are relative to the VSI. Each VF can + * have a maximum of sixteen queues for all of its VSIs. + * + * The PF is required to return a status code in v_retval for all messages + * except RESET_VF, which does not require any response. The returned value + * is of virtchnl_status_code type, defined here. + * + * In general, VF driver initialization should roughly follow the order of + * these opcodes. The VF driver must first validate the API version of the + * PF driver, then request a reset, then get resources, then configure + * queues and interrupts. After these operations are complete, the VF + * driver may start its queues, optionally add MAC and VLAN filters, and + * process traffic. + */ + +/* START GENERIC DEFINES + * Need to ensure the following enums and defines hold the same meaning and + * value in current and future projects + */ + +#define VIRTCHNL_ETH_LENGTH_OF_ADDRESS 6 + +/* These macros are used to generate compilation errors if a structure/union + * is not exactly the correct length. It gives a divide by zero error if the + * structure/union is not of the correct size, otherwise it creates an enum + * that is never used. + */ +#define VIRTCHNL_CHECK_STRUCT_LEN(n, X) enum virtchnl_static_assert_enum_##X \ + { virtchnl_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) } +#define VIRTCHNL_CHECK_UNION_LEN(n, X) enum virtchnl_static_asset_enum_##X \ + { virtchnl_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) } + + +/* Error Codes + * Note that many older versions of various iAVF drivers convert the reported + * status code directly into an iavf_status enumeration. For this reason, it + * is important that the values of these enumerations line up. + */ +enum virtchnl_status_code { + VIRTCHNL_STATUS_SUCCESS = 0, + VIRTCHNL_STATUS_ERR_PARAM = -5, + VIRTCHNL_STATUS_ERR_NO_MEMORY = -18, + VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH = -38, + VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR = -39, + VIRTCHNL_STATUS_ERR_INVALID_VF_ID = -40, + VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR = -53, + VIRTCHNL_STATUS_ERR_NOT_SUPPORTED = -64, +}; + +/* Backward compatibility */ +#define VIRTCHNL_ERR_PARAM VIRTCHNL_STATUS_ERR_PARAM +#define VIRTCHNL_STATUS_NOT_SUPPORTED VIRTCHNL_STATUS_ERR_NOT_SUPPORTED + +#define VIRTCHNL_LINK_SPEED_2_5GB_SHIFT 0x0 +#define VIRTCHNL_LINK_SPEED_100MB_SHIFT 0x1 +#define VIRTCHNL_LINK_SPEED_1000MB_SHIFT 0x2 +#define VIRTCHNL_LINK_SPEED_10GB_SHIFT 0x3 +#define VIRTCHNL_LINK_SPEED_40GB_SHIFT 0x4 +#define VIRTCHNL_LINK_SPEED_20GB_SHIFT 0x5 +#define VIRTCHNL_LINK_SPEED_25GB_SHIFT 0x6 +#define VIRTCHNL_LINK_SPEED_5GB_SHIFT 0x7 + +enum virtchnl_link_speed { + VIRTCHNL_LINK_SPEED_UNKNOWN = 0, + VIRTCHNL_LINK_SPEED_100MB = BIT(VIRTCHNL_LINK_SPEED_100MB_SHIFT), + VIRTCHNL_LINK_SPEED_1GB = BIT(VIRTCHNL_LINK_SPEED_1000MB_SHIFT), + VIRTCHNL_LINK_SPEED_10GB = BIT(VIRTCHNL_LINK_SPEED_10GB_SHIFT), + VIRTCHNL_LINK_SPEED_40GB = BIT(VIRTCHNL_LINK_SPEED_40GB_SHIFT), + VIRTCHNL_LINK_SPEED_20GB = BIT(VIRTCHNL_LINK_SPEED_20GB_SHIFT), + VIRTCHNL_LINK_SPEED_25GB = BIT(VIRTCHNL_LINK_SPEED_25GB_SHIFT), + VIRTCHNL_LINK_SPEED_2_5GB = BIT(VIRTCHNL_LINK_SPEED_2_5GB_SHIFT), + VIRTCHNL_LINK_SPEED_5GB = BIT(VIRTCHNL_LINK_SPEED_5GB_SHIFT), +}; + +/* for hsplit_0 field of Rx HMC context */ +/* deprecated with AVF 1.0 */ +enum virtchnl_rx_hsplit { + VIRTCHNL_RX_HSPLIT_NO_SPLIT = 0, + VIRTCHNL_RX_HSPLIT_SPLIT_L2 = 1, + VIRTCHNL_RX_HSPLIT_SPLIT_IP = 2, + VIRTCHNL_RX_HSPLIT_SPLIT_TCP_UDP = 4, + VIRTCHNL_RX_HSPLIT_SPLIT_SCTP = 8, +}; + +enum virtchnl_bw_limit_type { + VIRTCHNL_BW_SHAPER = 0, +}; +/* END GENERIC DEFINES */ + +/* Opcodes for VF-PF communication. These are placed in the v_opcode field + * of the virtchnl_msg structure. + */ +enum virtchnl_ops { +/* The PF sends status change events to VFs using + * the VIRTCHNL_OP_EVENT opcode. + * VFs send requests to the PF using the other ops. + * Use of "advanced opcode" features must be negotiated as part of capabilities + * exchange and are not considered part of base mode feature set. + * + */ + VIRTCHNL_OP_UNKNOWN = 0, + VIRTCHNL_OP_VERSION = 1, /* must ALWAYS be 1 */ + VIRTCHNL_OP_RESET_VF = 2, + VIRTCHNL_OP_GET_VF_RESOURCES = 3, + VIRTCHNL_OP_CONFIG_TX_QUEUE = 4, + VIRTCHNL_OP_CONFIG_RX_QUEUE = 5, + VIRTCHNL_OP_CONFIG_VSI_QUEUES = 6, + VIRTCHNL_OP_CONFIG_IRQ_MAP = 7, + VIRTCHNL_OP_ENABLE_QUEUES = 8, + VIRTCHNL_OP_DISABLE_QUEUES = 9, + VIRTCHNL_OP_ADD_ETH_ADDR = 10, + VIRTCHNL_OP_DEL_ETH_ADDR = 11, + VIRTCHNL_OP_ADD_VLAN = 12, + VIRTCHNL_OP_DEL_VLAN = 13, + VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE = 14, + VIRTCHNL_OP_GET_STATS = 15, + VIRTCHNL_OP_RSVD = 16, + VIRTCHNL_OP_EVENT = 17, /* must ALWAYS be 17 */ + /* opcode 19 is reserved */ + /* opcodes 20, 21, and 22 are reserved */ + VIRTCHNL_OP_CONFIG_RSS_KEY = 23, + VIRTCHNL_OP_CONFIG_RSS_LUT = 24, + VIRTCHNL_OP_GET_RSS_HENA_CAPS = 25, + VIRTCHNL_OP_SET_RSS_HENA = 26, + VIRTCHNL_OP_ENABLE_VLAN_STRIPPING = 27, + VIRTCHNL_OP_DISABLE_VLAN_STRIPPING = 28, + VIRTCHNL_OP_REQUEST_QUEUES = 29, + VIRTCHNL_OP_ENABLE_CHANNELS = 30, + VIRTCHNL_OP_DISABLE_CHANNELS = 31, + VIRTCHNL_OP_ADD_CLOUD_FILTER = 32, + VIRTCHNL_OP_DEL_CLOUD_FILTER = 33, + /* opcode 34 is reserved */ + /* opcodes 38, 39, 40, 41, 42 and 43 are reserved */ + /* opcode 44 is reserved */ + VIRTCHNL_OP_ADD_RSS_CFG = 45, + VIRTCHNL_OP_DEL_RSS_CFG = 46, + VIRTCHNL_OP_ADD_FDIR_FILTER = 47, + VIRTCHNL_OP_DEL_FDIR_FILTER = 48, + VIRTCHNL_OP_GET_MAX_RSS_QREGION = 50, + VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS = 51, + VIRTCHNL_OP_ADD_VLAN_V2 = 52, + VIRTCHNL_OP_DEL_VLAN_V2 = 53, + VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 = 54, + VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 = 55, + VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 = 56, + VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2 = 57, + VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2 = 58, + VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 = 59, + VIRTCHNL_OP_1588_PTP_GET_CAPS = 60, + VIRTCHNL_OP_1588_PTP_GET_TIME = 61, + VIRTCHNL_OP_1588_PTP_SET_TIME = 62, + VIRTCHNL_OP_1588_PTP_ADJ_TIME = 63, + VIRTCHNL_OP_1588_PTP_ADJ_FREQ = 64, + VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP = 65, + VIRTCHNL_OP_GET_QOS_CAPS = 66, + VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP = 67, + VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS = 68, + VIRTCHNL_OP_1588_PTP_SET_PIN_CFG = 69, + VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP = 70, + VIRTCHNL_OP_ENABLE_QUEUES_V2 = 107, + VIRTCHNL_OP_DISABLE_QUEUES_V2 = 108, + VIRTCHNL_OP_MAP_QUEUE_VECTOR = 111, + VIRTCHNL_OP_CONFIG_QUEUE_BW = 112, + VIRTCHNL_OP_CONFIG_QUANTA = 113, + VIRTCHNL_OP_MAX, +}; + +static inline const char *virtchnl_op_str(enum virtchnl_ops v_opcode) +{ + switch (v_opcode) { + case VIRTCHNL_OP_UNKNOWN: + return "VIRTCHNL_OP_UNKNOWN"; + case VIRTCHNL_OP_VERSION: + return "VIRTCHNL_OP_VERSION"; + case VIRTCHNL_OP_RESET_VF: + return "VIRTCHNL_OP_RESET_VF"; + case VIRTCHNL_OP_GET_VF_RESOURCES: + return "VIRTCHNL_OP_GET_VF_RESOURCES"; + case VIRTCHNL_OP_CONFIG_TX_QUEUE: + return "VIRTCHNL_OP_CONFIG_TX_QUEUE"; + case VIRTCHNL_OP_CONFIG_RX_QUEUE: + return "VIRTCHNL_OP_CONFIG_RX_QUEUE"; + case VIRTCHNL_OP_CONFIG_VSI_QUEUES: + return "VIRTCHNL_OP_CONFIG_VSI_QUEUES"; + case VIRTCHNL_OP_CONFIG_IRQ_MAP: + return "VIRTCHNL_OP_CONFIG_IRQ_MAP"; + case VIRTCHNL_OP_ENABLE_QUEUES: + return "VIRTCHNL_OP_ENABLE_QUEUES"; + case VIRTCHNL_OP_DISABLE_QUEUES: + return "VIRTCHNL_OP_DISABLE_QUEUES"; + case VIRTCHNL_OP_ADD_ETH_ADDR: + return "VIRTCHNL_OP_ADD_ETH_ADDR"; + case VIRTCHNL_OP_DEL_ETH_ADDR: + return "VIRTCHNL_OP_DEL_ETH_ADDR"; + case VIRTCHNL_OP_ADD_VLAN: + return "VIRTCHNL_OP_ADD_VLAN"; + case VIRTCHNL_OP_DEL_VLAN: + return "VIRTCHNL_OP_DEL_VLAN"; + case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE: + return "VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE"; + case VIRTCHNL_OP_GET_STATS: + return "VIRTCHNL_OP_GET_STATS"; + case VIRTCHNL_OP_RSVD: + return "VIRTCHNL_OP_RSVD"; + case VIRTCHNL_OP_EVENT: + return "VIRTCHNL_OP_EVENT"; + case VIRTCHNL_OP_CONFIG_RSS_KEY: + return "VIRTCHNL_OP_CONFIG_RSS_KEY"; + case VIRTCHNL_OP_CONFIG_RSS_LUT: + return "VIRTCHNL_OP_CONFIG_RSS_LUT"; + case VIRTCHNL_OP_GET_RSS_HENA_CAPS: + return "VIRTCHNL_OP_GET_RSS_HENA_CAPS"; + case VIRTCHNL_OP_SET_RSS_HENA: + return "VIRTCHNL_OP_SET_RSS_HENA"; + case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING: + return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING"; + case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING: + return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING"; + case VIRTCHNL_OP_REQUEST_QUEUES: + return "VIRTCHNL_OP_REQUEST_QUEUES"; + case VIRTCHNL_OP_ENABLE_CHANNELS: + return "VIRTCHNL_OP_ENABLE_CHANNELS"; + case VIRTCHNL_OP_DISABLE_CHANNELS: + return "VIRTCHNL_OP_DISABLE_CHANNELS"; + case VIRTCHNL_OP_ADD_CLOUD_FILTER: + return "VIRTCHNL_OP_ADD_CLOUD_FILTER"; + case VIRTCHNL_OP_DEL_CLOUD_FILTER: + return "VIRTCHNL_OP_DEL_CLOUD_FILTER"; + case VIRTCHNL_OP_ADD_RSS_CFG: + return "VIRTCHNL_OP_ADD_RSS_CFG"; + case VIRTCHNL_OP_DEL_RSS_CFG: + return "VIRTCHNL_OP_DEL_RSS_CFG"; + case VIRTCHNL_OP_ADD_FDIR_FILTER: + return "VIRTCHNL_OP_ADD_FDIR_FILTER"; + case VIRTCHNL_OP_DEL_FDIR_FILTER: + return "VIRTCHNL_OP_DEL_FDIR_FILTER"; + case VIRTCHNL_OP_GET_MAX_RSS_QREGION: + return "VIRTCHNL_OP_GET_MAX_RSS_QREGION"; + case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS: + return "VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS"; + case VIRTCHNL_OP_ADD_VLAN_V2: + return "VIRTCHNL_OP_ADD_VLAN_V2"; + case VIRTCHNL_OP_DEL_VLAN_V2: + return "VIRTCHNL_OP_DEL_VLAN_V2"; + case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2: + return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2"; + case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2: + return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2"; + case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2: + return "VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2"; + case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2: + return "VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2"; + case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2: + return "VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2"; + case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2: + return "VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2"; + case VIRTCHNL_OP_1588_PTP_GET_CAPS: + return "VIRTCHNL_OP_1588_PTP_GET_CAPS"; + case VIRTCHNL_OP_1588_PTP_GET_TIME: + return "VIRTCHNL_OP_1588_PTP_GET_TIME"; + case VIRTCHNL_OP_1588_PTP_SET_TIME: + return "VIRTCHNL_OP_1588_PTP_SET_TIME"; + case VIRTCHNL_OP_1588_PTP_ADJ_TIME: + return "VIRTCHNL_OP_1588_PTP_ADJ_TIME"; + case VIRTCHNL_OP_1588_PTP_ADJ_FREQ: + return "VIRTCHNL_OP_1588_PTP_ADJ_FREQ"; + case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP: + return "VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP"; + case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS: + return "VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS"; + case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG: + return "VIRTCHNL_OP_1588_PTP_SET_PIN_CFG"; + case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP: + return "VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP"; + case VIRTCHNL_OP_ENABLE_QUEUES_V2: + return "VIRTCHNL_OP_ENABLE_QUEUES_V2"; + case VIRTCHNL_OP_DISABLE_QUEUES_V2: + return "VIRTCHNL_OP_DISABLE_QUEUES_V2"; + case VIRTCHNL_OP_MAP_QUEUE_VECTOR: + return "VIRTCHNL_OP_MAP_QUEUE_VECTOR"; + case VIRTCHNL_OP_MAX: + return "VIRTCHNL_OP_MAX"; + default: + return "Unsupported (update virtchnl.h)"; + } +} + +static inline const char *virtchnl_stat_str(enum virtchnl_status_code v_status) +{ + switch (v_status) { + case VIRTCHNL_STATUS_SUCCESS: + return "VIRTCHNL_STATUS_SUCCESS"; + case VIRTCHNL_STATUS_ERR_PARAM: + return "VIRTCHNL_STATUS_ERR_PARAM"; + case VIRTCHNL_STATUS_ERR_NO_MEMORY: + return "VIRTCHNL_STATUS_ERR_NO_MEMORY"; + case VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH: + return "VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH"; + case VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR: + return "VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR"; + case VIRTCHNL_STATUS_ERR_INVALID_VF_ID: + return "VIRTCHNL_STATUS_ERR_INVALID_VF_ID"; + case VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR: + return "VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR"; + case VIRTCHNL_STATUS_ERR_NOT_SUPPORTED: + return "VIRTCHNL_STATUS_ERR_NOT_SUPPORTED"; + default: + return "Unknown status code (update virtchnl.h)"; + } +} + +/* Virtual channel message descriptor. This overlays the admin queue + * descriptor. All other data is passed in external buffers. + */ + +struct virtchnl_msg { + u8 pad[8]; /* AQ flags/opcode/len/retval fields */ + + /* avoid confusion with desc->opcode */ + enum virtchnl_ops v_opcode; + + /* ditto for desc->retval */ + enum virtchnl_status_code v_retval; + u32 vfid; /* used by PF when sending to VF */ +}; + +VIRTCHNL_CHECK_STRUCT_LEN(20, virtchnl_msg); + +/* Message descriptions and data structures. */ + +/* VIRTCHNL_OP_VERSION + * VF posts its version number to the PF. PF responds with its version number + * in the same format, along with a return code. + * Reply from PF has its major/minor versions also in param0 and param1. + * If there is a major version mismatch, then the VF cannot operate. + * If there is a minor version mismatch, then the VF can operate but should + * add a warning to the system log. + * + * This enum element MUST always be specified as == 1, regardless of other + * changes in the API. The PF must always respond to this message without + * error regardless of version mismatch. + */ +#define VIRTCHNL_VERSION_MAJOR 1 +#define VIRTCHNL_VERSION_MINOR 1 +#define VIRTCHNL_VERSION_MAJOR_2 2 +#define VIRTCHNL_VERSION_MINOR_0 0 +#define VIRTCHNL_VERSION_MINOR_NO_VF_CAPS 0 + +struct virtchnl_version_info { + u32 major; + u32 minor; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_version_info); + +#define VF_IS_V10(_ver) (((_ver)->major == 1) && ((_ver)->minor == 0)) +#define VF_IS_V11(_ver) (((_ver)->major == 1) && ((_ver)->minor == 1)) +#define VF_IS_V20(_ver) (((_ver)->major == 2) && ((_ver)->minor == 0)) + +/* VIRTCHNL_OP_RESET_VF + * VF sends this request to PF with no parameters + * PF does NOT respond! VF driver must delay then poll VFGEN_RSTAT register + * until reset completion is indicated. The admin queue must be reinitialized + * after this operation. + * + * When reset is complete, PF must ensure that all queues in all VSIs associated + * with the VF are stopped, all queue configurations in the HMC are set to 0, + * and all MAC and VLAN filters (except the default MAC address) on all VSIs + * are cleared. + */ + +/* VSI types that use VIRTCHNL interface for VF-PF communication. VSI_SRIOV + * vsi_type should always be 6 for backward compatibility. Add other fields + * as needed. + */ +enum virtchnl_vsi_type { + VIRTCHNL_VSI_TYPE_INVALID = 0, + VIRTCHNL_VSI_SRIOV = 6, +}; + +/* VIRTCHNL_OP_GET_VF_RESOURCES + * Version 1.0 VF sends this request to PF with no parameters + * Version 1.1 VF sends this request to PF with u32 bitmap of its capabilities + * PF responds with an indirect message containing + * virtchnl_vf_resource and one or more + * virtchnl_vsi_resource structures. + */ + +struct virtchnl_vsi_resource { + u16 vsi_id; + u16 num_queue_pairs; + + /* see enum virtchnl_vsi_type */ + s32 vsi_type; + u16 qset_handle; + u8 default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS]; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vsi_resource); + +/* VF capability flags + * VIRTCHNL_VF_OFFLOAD_L2 flag is inclusive of base mode L2 offloads including + * TX/RX Checksum offloading and TSO for non-tunnelled packets. + */ +#define VIRTCHNL_VF_OFFLOAD_L2 BIT(0) +#define VIRTCHNL_VF_OFFLOAD_IWARP BIT(1) +#define VIRTCHNL_VF_CAP_RDMA VIRTCHNL_VF_OFFLOAD_IWARP +#define VIRTCHNL_VF_OFFLOAD_RSS_AQ BIT(3) +#define VIRTCHNL_VF_OFFLOAD_RSS_REG BIT(4) +#define VIRTCHNL_VF_OFFLOAD_WB_ON_ITR BIT(5) +#define VIRTCHNL_VF_OFFLOAD_REQ_QUEUES BIT(6) +/* used to negotiate communicating link speeds in Mbps */ +#define VIRTCHNL_VF_CAP_ADV_LINK_SPEED BIT(7) + /* BIT(8) is reserved */ +#define VIRTCHNL_VF_LARGE_NUM_QPAIRS BIT(9) +#define VIRTCHNL_VF_OFFLOAD_CRC BIT(10) +#define VIRTCHNL_VF_OFFLOAD_VLAN_V2 BIT(15) +#define VIRTCHNL_VF_OFFLOAD_VLAN BIT(16) +#define VIRTCHNL_VF_OFFLOAD_RX_POLLING BIT(17) +#define VIRTCHNL_VF_OFFLOAD_RSS_PCTYPE_V2 BIT(18) +#define VIRTCHNL_VF_OFFLOAD_RSS_PF BIT(19) +#define VIRTCHNL_VF_OFFLOAD_ENCAP BIT(20) +#define VIRTCHNL_VF_OFFLOAD_ENCAP_CSUM BIT(21) +#define VIRTCHNL_VF_OFFLOAD_RX_ENCAP_CSUM BIT(22) +#define VIRTCHNL_VF_OFFLOAD_ADQ BIT(23) +#define VIRTCHNL_VF_OFFLOAD_ADQ_V2 BIT(24) +#define VIRTCHNL_VF_OFFLOAD_USO BIT(25) + /* BIT(26) is reserved */ +#define VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF BIT(27) +#define VIRTCHNL_VF_OFFLOAD_FDIR_PF BIT(28) +#define VIRTCHNL_VF_OFFLOAD_QOS BIT(29) + /* BIT(30) is reserved */ +#define VIRTCHNL_VF_CAP_PTP BIT(31) + +#define VF_BASE_MODE_OFFLOADS (VIRTCHNL_VF_OFFLOAD_L2 | \ + VIRTCHNL_VF_OFFLOAD_VLAN | \ + VIRTCHNL_VF_OFFLOAD_RSS_PF) + +struct virtchnl_vf_resource { + u16 num_vsis; + u16 num_queue_pairs; + u16 max_vectors; + u16 max_mtu; + + u32 vf_cap_flags; + u32 rss_key_size; + u32 rss_lut_size; + + struct virtchnl_vsi_resource vsi_res[1]; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_vf_resource); + +/* VIRTCHNL_OP_CONFIG_TX_QUEUE + * VF sends this message to set up parameters for one TX queue. + * External data buffer contains one instance of virtchnl_txq_info. + * PF configures requested queue and returns a status code. + */ + +/* Tx queue config info */ +struct virtchnl_txq_info { + u16 vsi_id; + u16 queue_id; + u16 ring_len; /* number of descriptors, multiple of 8 */ + u16 headwb_enabled; /* deprecated with AVF 1.0 */ + u64 dma_ring_addr; + u64 dma_headwb_addr; /* deprecated with AVF 1.0 */ +}; + +VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_txq_info); + +/* RX descriptor IDs (range from 0 to 63) */ +enum virtchnl_rx_desc_ids { + VIRTCHNL_RXDID_0_16B_BASE = 0, + VIRTCHNL_RXDID_1_32B_BASE = 1, + VIRTCHNL_RXDID_2_FLEX_SQ_NIC = 2, + VIRTCHNL_RXDID_3_FLEX_SQ_SW = 3, + VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB = 4, + VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL = 5, + VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2 = 6, + VIRTCHNL_RXDID_7_HW_RSVD = 7, + /* 8 through 15 are reserved */ + VIRTCHNL_RXDID_16_COMMS_GENERIC = 16, + VIRTCHNL_RXDID_17_COMMS_AUX_VLAN = 17, + VIRTCHNL_RXDID_18_COMMS_AUX_IPV4 = 18, + VIRTCHNL_RXDID_19_COMMS_AUX_IPV6 = 19, + VIRTCHNL_RXDID_20_COMMS_AUX_FLOW = 20, + VIRTCHNL_RXDID_21_COMMS_AUX_TCP = 21, + /* 22 through 63 are reserved */ +}; + +/* RX descriptor ID bitmasks */ +enum virtchnl_rx_desc_id_bitmasks { + VIRTCHNL_RXDID_0_16B_BASE_M = BIT(VIRTCHNL_RXDID_0_16B_BASE), + VIRTCHNL_RXDID_1_32B_BASE_M = BIT(VIRTCHNL_RXDID_1_32B_BASE), + VIRTCHNL_RXDID_2_FLEX_SQ_NIC_M = BIT(VIRTCHNL_RXDID_2_FLEX_SQ_NIC), + VIRTCHNL_RXDID_3_FLEX_SQ_SW_M = BIT(VIRTCHNL_RXDID_3_FLEX_SQ_SW), + VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB_M = BIT(VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB), + VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL_M = BIT(VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL), + VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2_M = BIT(VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2), + VIRTCHNL_RXDID_7_HW_RSVD_M = BIT(VIRTCHNL_RXDID_7_HW_RSVD), + /* 9 through 15 are reserved */ + VIRTCHNL_RXDID_16_COMMS_GENERIC_M = BIT(VIRTCHNL_RXDID_16_COMMS_GENERIC), + VIRTCHNL_RXDID_17_COMMS_AUX_VLAN_M = BIT(VIRTCHNL_RXDID_17_COMMS_AUX_VLAN), + VIRTCHNL_RXDID_18_COMMS_AUX_IPV4_M = BIT(VIRTCHNL_RXDID_18_COMMS_AUX_IPV4), + VIRTCHNL_RXDID_19_COMMS_AUX_IPV6_M = BIT(VIRTCHNL_RXDID_19_COMMS_AUX_IPV6), + VIRTCHNL_RXDID_20_COMMS_AUX_FLOW_M = BIT(VIRTCHNL_RXDID_20_COMMS_AUX_FLOW), + VIRTCHNL_RXDID_21_COMMS_AUX_TCP_M = BIT(VIRTCHNL_RXDID_21_COMMS_AUX_TCP), + /* 22 through 63 are reserved */ +}; + +/* virtchnl_rxq_info_flags + * + * Definition of bits in the flags field of the virtchnl_rxq_info structure. + */ +enum virtchnl_rxq_info_flags { + /* If the VIRTCHNL_PTP_RX_TSTAMP bit of the flag field is set, this is + * a request to enable Rx timestamp. Other flag bits are currently + * reserved and they may be extended in the future. + */ + VIRTCHNL_PTP_RX_TSTAMP = BIT(0), +}; + +/* VIRTCHNL_OP_CONFIG_RX_QUEUE + * VF sends this message to set up parameters for one RX queue. + * External data buffer contains one instance of virtchnl_rxq_info. + * PF configures requested queue and returns a status code. The + * crc_disable flag disables CRC stripping on the VF. Setting + * the crc_disable flag to 1 will disable CRC stripping for each + * queue in the VF where the flag is set. The VIRTCHNL_VF_OFFLOAD_CRC + * offload must have been set prior to sending this info or the PF + * will ignore the request. This flag should be set the same for + * all of the queues for a VF. + */ + +/* Rx queue config info */ +struct virtchnl_rxq_info { + u16 vsi_id; + u16 queue_id; + u32 ring_len; /* number of descriptors, multiple of 32 */ + u16 hdr_size; + u16 splithdr_enabled; /* deprecated with AVF 1.0 */ + u32 databuffer_size; + u32 max_pkt_size; + u8 crc_disable; + u8 pad1[3]; + u64 dma_ring_addr; + + /* see enum virtchnl_rx_hsplit; deprecated with AVF 1.0 */ + s32 rx_split_pos; + u32 pad2; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_rxq_info); + +/* VIRTCHNL_OP_CONFIG_VSI_QUEUES + * VF sends this message to set parameters for active TX and RX queues + * associated with the specified VSI. + * PF configures queues and returns status. + * If the number of queues specified is greater than the number of queues + * associated with the VSI, an error is returned and no queues are configured. + * NOTE: The VF is not required to configure all queues in a single request. + * It may send multiple messages. PF drivers must correctly handle all VF + * requests. + */ +struct virtchnl_queue_pair_info { + /* NOTE: vsi_id and queue_id should be identical for both queues. */ + struct virtchnl_txq_info txq; + struct virtchnl_rxq_info rxq; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(64, virtchnl_queue_pair_info); + +struct virtchnl_vsi_queue_config_info { + u16 vsi_id; + u16 num_queue_pairs; + u32 pad; + struct virtchnl_queue_pair_info qpair[1]; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_vsi_queue_config_info); + +/* VIRTCHNL_OP_REQUEST_QUEUES + * VF sends this message to request the PF to allocate additional queues to + * this VF. Each VF gets a guaranteed number of queues on init but asking for + * additional queues must be negotiated. This is a best effort request as it + * is possible the PF does not have enough queues left to support the request. + * If the PF cannot support the number requested it will respond with the + * maximum number it is able to support. If the request is successful, PF will + * then reset the VF to institute required changes. + */ + +/* VF resource request */ +struct virtchnl_vf_res_request { + u16 num_queue_pairs; +}; + +/* VIRTCHNL_OP_CONFIG_IRQ_MAP + * VF uses this message to map vectors to queues. + * The rxq_map and txq_map fields are bitmaps used to indicate which queues + * are to be associated with the specified vector. + * The "other" causes are always mapped to vector 0. The VF may not request + * that vector 0 be used for traffic. + * PF configures interrupt mapping and returns status. + * NOTE: due to hardware requirements, all active queues (both TX and RX) + * should be mapped to interrupts, even if the driver intends to operate + * only in polling mode. In this case the interrupt may be disabled, but + * the ITR timer will still run to trigger writebacks. + */ +struct virtchnl_vector_map { + u16 vsi_id; + u16 vector_id; + u16 rxq_map; + u16 txq_map; + u16 rxitr_idx; + u16 txitr_idx; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_vector_map); + +struct virtchnl_irq_map_info { + u16 num_vectors; + struct virtchnl_vector_map vecmap[1]; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(14, virtchnl_irq_map_info); + +/* VIRTCHNL_OP_ENABLE_QUEUES + * VIRTCHNL_OP_DISABLE_QUEUES + * VF sends these message to enable or disable TX/RX queue pairs. + * The queues fields are bitmaps indicating which queues to act upon. + * (Currently, we only support 16 queues per VF, but we make the field + * u32 to allow for expansion.) + * PF performs requested action and returns status. + * NOTE: The VF is not required to enable/disable all queues in a single + * request. It may send multiple messages. + * PF drivers must correctly handle all VF requests. + */ +struct virtchnl_queue_select { + u16 vsi_id; + u16 pad; + u32 rx_queues; + u32 tx_queues; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_select); + +/* VIRTCHNL_OP_GET_MAX_RSS_QREGION + * + * if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in VIRTCHNL_OP_GET_VF_RESOURCES + * then this op must be supported. + * + * VF sends this message in order to query the max RSS queue region + * size supported by PF, when VIRTCHNL_VF_LARGE_NUM_QPAIRS is enabled. + * This information should be used when configuring the RSS LUT and/or + * configuring queue region based filters. + * + * The maximum RSS queue region is 2^qregion_width. So, a qregion_width + * of 6 would inform the VF that the PF supports a maximum RSS queue region + * of 64. + * + * A queue region represents a range of queues that can be used to configure + * a RSS LUT. For example, if a VF is given 64 queues, but only a max queue + * region size of 16 (i.e. 2^qregion_width = 16) then it will only be able + * to configure the RSS LUT with queue indices from 0 to 15. However, other + * filters can be used to direct packets to queues >15 via specifying a queue + * base/offset and queue region width. + */ +struct virtchnl_max_rss_qregion { + u16 vport_id; + u16 qregion_width; + u8 pad[4]; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_max_rss_qregion); + +/* VIRTCHNL_OP_ADD_ETH_ADDR + * VF sends this message in order to add one or more unicast or multicast + * address filters for the specified VSI. + * PF adds the filters and returns status. + */ + +/* VIRTCHNL_OP_DEL_ETH_ADDR + * VF sends this message in order to remove one or more unicast or multicast + * filters for the specified VSI. + * PF removes the filters and returns status. + */ + +/* VIRTCHNL_ETHER_ADDR_LEGACY + * Prior to adding the @type member to virtchnl_ether_addr, there were 2 pad + * bytes. Moving forward all VF drivers should not set type to + * VIRTCHNL_ETHER_ADDR_LEGACY. This is only here to not break previous/legacy + * behavior. The control plane function (i.e. PF) can use a best effort method + * of tracking the primary/device unicast in this case, but there is no + * guarantee and functionality depends on the implementation of the PF. + */ + +/* VIRTCHNL_ETHER_ADDR_PRIMARY + * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_PRIMARY for the + * primary/device unicast MAC address filter for VIRTCHNL_OP_ADD_ETH_ADDR and + * VIRTCHNL_OP_DEL_ETH_ADDR. This allows for the underlying control plane + * function (i.e. PF) to accurately track and use this MAC address for + * displaying on the host and for VM/function reset. + */ + +/* VIRTCHNL_ETHER_ADDR_EXTRA + * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_EXTRA for any extra + * unicast and/or multicast filters that are being added/deleted via + * VIRTCHNL_OP_DEL_ETH_ADDR/VIRTCHNL_OP_ADD_ETH_ADDR respectively. + */ +struct virtchnl_ether_addr { + u8 addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS]; + u8 type; +#define VIRTCHNL_ETHER_ADDR_LEGACY 0 +#define VIRTCHNL_ETHER_ADDR_PRIMARY 1 +#define VIRTCHNL_ETHER_ADDR_EXTRA 2 +#define VIRTCHNL_ETHER_ADDR_TYPE_MASK 3 /* first two bits of type are valid */ + u8 pad; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_ether_addr); + +struct virtchnl_ether_addr_list { + u16 vsi_id; + u16 num_elements; + struct virtchnl_ether_addr list[1]; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_ether_addr_list); + +/* VIRTCHNL_OP_ADD_VLAN + * VF sends this message to add one or more VLAN tag filters for receives. + * PF adds the filters and returns status. + * If a port VLAN is configured by the PF, this operation will return an + * error to the VF. + */ + +/* VIRTCHNL_OP_DEL_VLAN + * VF sends this message to remove one or more VLAN tag filters for receives. + * PF removes the filters and returns status. + * If a port VLAN is configured by the PF, this operation will return an + * error to the VF. + */ + +struct virtchnl_vlan_filter_list { + u16 vsi_id; + u16 num_elements; + u16 vlan_id[1]; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_vlan_filter_list); + +/* This enum is used for all of the VIRTCHNL_VF_OFFLOAD_VLAN_V2_CAPS related + * structures and opcodes. + * + * VIRTCHNL_VLAN_UNSUPPORTED - This field is not supported and if a VF driver + * populates it the PF should return VIRTCHNL_STATUS_ERR_NOT_SUPPORTED. + * + * VIRTCHNL_VLAN_ETHERTYPE_8100 - This field supports 0x8100 ethertype. + * VIRTCHNL_VLAN_ETHERTYPE_88A8 - This field supports 0x88A8 ethertype. + * VIRTCHNL_VLAN_ETHERTYPE_9100 - This field supports 0x9100 ethertype. + * + * VIRTCHNL_VLAN_ETHERTYPE_AND - Used when multiple ethertypes can be supported + * by the PF concurrently. For example, if the PF can support + * VIRTCHNL_VLAN_ETHERTYPE_8100 AND VIRTCHNL_VLAN_ETHERTYPE_88A8 filters it + * would OR the following bits: + * + * VIRTHCNL_VLAN_ETHERTYPE_8100 | + * VIRTCHNL_VLAN_ETHERTYPE_88A8 | + * VIRTCHNL_VLAN_ETHERTYPE_AND; + * + * The VF would interpret this as VLAN filtering can be supported on both 0x8100 + * and 0x88A8 VLAN ethertypes. + * + * VIRTCHNL_ETHERTYPE_XOR - Used when only a single ethertype can be supported + * by the PF concurrently. For example if the PF can support + * VIRTCHNL_VLAN_ETHERTYPE_8100 XOR VIRTCHNL_VLAN_ETHERTYPE_88A8 stripping + * offload it would OR the following bits: + * + * VIRTCHNL_VLAN_ETHERTYPE_8100 | + * VIRTCHNL_VLAN_ETHERTYPE_88A8 | + * VIRTCHNL_VLAN_ETHERTYPE_XOR; + * + * The VF would interpret this as VLAN stripping can be supported on either + * 0x8100 or 0x88a8 VLAN ethertypes. So when requesting VLAN stripping via + * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 the specified ethertype will override + * the previously set value. + * + * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 - Used to tell the VF to insert and/or + * strip the VLAN tag using the L2TAG1 field of the Tx/Rx descriptors. + * + * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to insert hardware + * offloaded VLAN tags using the L2TAG2 field of the Tx descriptor. + * + * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to strip hardware + * offloaded VLAN tags using the L2TAG2_2 field of the Rx descriptor. + * + * VIRTCHNL_VLAN_PRIO - This field supports VLAN priority bits. This is used for + * VLAN filtering if the underlying PF supports it. + * + * VIRTCHNL_VLAN_TOGGLE_ALLOWED - This field is used to say whether a + * certain VLAN capability can be toggled. For example if the underlying PF/CP + * allows the VF to toggle VLAN filtering, stripping, and/or insertion it should + * set this bit along with the supported ethertypes. + */ +enum virtchnl_vlan_support { + VIRTCHNL_VLAN_UNSUPPORTED = 0, + VIRTCHNL_VLAN_ETHERTYPE_8100 = 0x00000001, + VIRTCHNL_VLAN_ETHERTYPE_88A8 = 0x00000002, + VIRTCHNL_VLAN_ETHERTYPE_9100 = 0x00000004, + VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 = 0x00000100, + VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 = 0x00000200, + VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2_2 = 0x00000400, + VIRTCHNL_VLAN_PRIO = 0x01000000, + VIRTCHNL_VLAN_FILTER_MASK = 0x10000000, + VIRTCHNL_VLAN_ETHERTYPE_AND = 0x20000000, + VIRTCHNL_VLAN_ETHERTYPE_XOR = 0x40000000, + VIRTCHNL_VLAN_TOGGLE = 0x80000000 +}; + +/* This structure is used as part of the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS + * for filtering, insertion, and stripping capabilities. + * + * If only outer capabilities are supported (for filtering, insertion, and/or + * stripping) then this refers to the outer most or single VLAN from the VF's + * perspective. + * + * If only inner capabilities are supported (for filtering, insertion, and/or + * stripping) then this refers to the outer most or single VLAN from the VF's + * perspective. Functionally this is the same as if only outer capabilities are + * supported. The VF driver is just forced to use the inner fields when + * adding/deleting filters and enabling/disabling offloads (if supported). + * + * If both outer and inner capabilities are supported (for filtering, insertion, + * and/or stripping) then outer refers to the outer most or single VLAN and + * inner refers to the second VLAN, if it exists, in the packet. + * + * There is no support for tunneled VLAN offloads, so outer or inner are never + * referring to a tunneled packet from the VF's perspective. + */ +struct virtchnl_vlan_supported_caps { + u32 outer; + u32 inner; +}; + +/* The PF populates these fields based on the supported VLAN filtering. If a + * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will + * reject any VIRTCHNL_OP_ADD_VLAN_V2 or VIRTCHNL_OP_DEL_VLAN_V2 messages using + * the unsupported fields. + * + * Also, a VF is only allowed to toggle its VLAN filtering setting if the + * VIRTCHNL_VLAN_TOGGLE bit is set. + * + * The ethertype(s) specified in the ethertype_init field are the ethertypes + * enabled for VLAN filtering. VLAN filtering in this case refers to the outer + * most VLAN from the VF's perspective. If both inner and outer filtering are + * allowed then ethertype_init only refers to the outer most VLAN as only + * VLAN ethertype supported for inner VLAN filtering is + * VIRTCHNL_VLAN_ETHERTYPE_8100. By default, inner VLAN filtering is disabled + * when both inner and outer filtering are allowed. + * + * The max_filters field tells the VF how many VLAN filters it's allowed to have + * at any one time. If it exceeds this amount and tries to add another filter, + * then the request will be rejected by the PF. To prevent failures, the VF + * should keep track of how many VLAN filters it has added and not attempt to + * add more than max_filters. + */ +struct virtchnl_vlan_filtering_caps { + struct virtchnl_vlan_supported_caps filtering_support; + u32 ethertype_init; + u16 max_filters; + u8 pad[2]; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_filtering_caps); + +/* This enum is used for the virtchnl_vlan_offload_caps structure to specify + * if the PF supports a different ethertype for stripping and insertion. + * + * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION - The ethertype(s) specified + * for stripping affect the ethertype(s) specified for insertion and visa versa + * as well. If the VF tries to configure VLAN stripping via + * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 with VIRTCHNL_VLAN_ETHERTYPE_8100 then + * that will be the ethertype for both stripping and insertion. + * + * VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED - The ethertype(s) specified for + * stripping do not affect the ethertype(s) specified for insertion and visa + * versa. + */ +enum virtchnl_vlan_ethertype_match { + VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION = 0, + VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED = 1, +}; + +/* The PF populates these fields based on the supported VLAN offloads. If a + * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will + * reject any VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 or + * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 messages using the unsupported fields. + * + * Also, a VF is only allowed to toggle its VLAN offload setting if the + * VIRTCHNL_VLAN_TOGGLE_ALLOWED bit is set. + * + * The VF driver needs to be aware of how the tags are stripped by hardware and + * inserted by the VF driver based on the level of offload support. The PF will + * populate these fields based on where the VLAN tags are expected to be + * offloaded via the VIRTHCNL_VLAN_TAG_LOCATION_* bits. The VF will need to + * interpret these fields. See the definition of the + * VIRTCHNL_VLAN_TAG_LOCATION_* bits above the virtchnl_vlan_support + * enumeration. + */ +struct virtchnl_vlan_offload_caps { + struct virtchnl_vlan_supported_caps stripping_support; + struct virtchnl_vlan_supported_caps insertion_support; + u32 ethertype_init; + u8 ethertype_match; + u8 pad[3]; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_vlan_offload_caps); + +/* VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS + * VF sends this message to determine its VLAN capabilities. + * + * PF will mark which capabilities it supports based on hardware support and + * current configuration. For example, if a port VLAN is configured the PF will + * not allow outer VLAN filtering, stripping, or insertion to be configured so + * it will block these features from the VF. + * + * The VF will need to cross reference its capabilities with the PFs + * capabilities in the response message from the PF to determine the VLAN + * support. + */ +struct virtchnl_vlan_caps { + struct virtchnl_vlan_filtering_caps filtering; + struct virtchnl_vlan_offload_caps offloads; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_caps); + +struct virtchnl_vlan { + u16 tci; /* tci[15:13] = PCP and tci[11:0] = VID */ + u16 tci_mask; /* only valid if VIRTCHNL_VLAN_FILTER_MASK set in + * filtering caps + */ + u16 tpid; /* 0x8100, 0x88a8, etc. and only type(s) set in + * filtering caps. Note that tpid here does not refer to + * VIRTCHNL_VLAN_ETHERTYPE_*, but it refers to the + * actual 2-byte VLAN TPID + */ + u8 pad[2]; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_vlan); + +struct virtchnl_vlan_filter { + struct virtchnl_vlan inner; + struct virtchnl_vlan outer; + u8 pad[16]; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(32, virtchnl_vlan_filter); + +/* VIRTCHNL_OP_ADD_VLAN_V2 + * VIRTCHNL_OP_DEL_VLAN_V2 + * + * VF sends these messages to add/del one or more VLAN tag filters for Rx + * traffic. + * + * The PF attempts to add the filters and returns status. + * + * The VF should only ever attempt to add/del virtchnl_vlan_filter(s) using the + * supported fields negotiated via VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS. + */ +struct virtchnl_vlan_filter_list_v2 { + u16 vport_id; + u16 num_elements; + u8 pad[4]; + struct virtchnl_vlan_filter filters[1]; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_filter_list_v2); + +/* VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 + * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 + * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 + * VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2 + * + * VF sends this message to enable or disable VLAN stripping or insertion. It + * also needs to specify an ethertype. The VF knows which VLAN ethertypes are + * allowed and whether or not it's allowed to enable/disable the specific + * offload via the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to + * parse the virtchnl_vlan_caps.offloads fields to determine which offload + * messages are allowed. + * + * For example, if the PF populates the virtchnl_vlan_caps.offloads in the + * following manner the VF will be allowed to enable and/or disable 0x8100 inner + * VLAN insertion and/or stripping via the opcodes listed above. Inner in this + * case means the outer most or single VLAN from the VF's perspective. This is + * because no outer offloads are supported. See the comments above the + * virtchnl_vlan_supported_caps structure for more details. + * + * virtchnl_vlan_caps.offloads.stripping_support.inner = + * VIRTCHNL_VLAN_TOGGLE | + * VIRTCHNL_VLAN_ETHERTYPE_8100; + * + * virtchnl_vlan_caps.offloads.insertion_support.inner = + * VIRTCHNL_VLAN_TOGGLE | + * VIRTCHNL_VLAN_ETHERTYPE_8100; + * + * In order to enable inner (again note that in this case inner is the outer + * most or single VLAN from the VF's perspective) VLAN stripping for 0x8100 + * VLANs, the VF would populate the virtchnl_vlan_setting structure in the + * following manner and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message. + * + * virtchnl_vlan_setting.inner_ethertype_setting = + * VIRTCHNL_VLAN_ETHERTYPE_8100; + * + * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on + * initialization. + * + * The reason that VLAN TPID(s) are not being used for the + * outer_ethertype_setting and inner_ethertype_setting fields is because it's + * possible a device could support VLAN insertion and/or stripping offload on + * multiple ethertypes concurrently, so this method allows a VF to request + * multiple ethertypes in one message using the virtchnl_vlan_support + * enumeration. + * + * For example, if the PF populates the virtchnl_vlan_caps.offloads in the + * following manner the VF will be allowed to enable 0x8100 and 0x88a8 outer + * VLAN insertion and stripping simultaneously. The + * virtchnl_vlan_caps.offloads.ethertype_match field will also have to be + * populated based on what the PF can support. + * + * virtchnl_vlan_caps.offloads.stripping_support.outer = + * VIRTCHNL_VLAN_TOGGLE | + * VIRTCHNL_VLAN_ETHERTYPE_8100 | + * VIRTCHNL_VLAN_ETHERTYPE_88A8 | + * VIRTCHNL_VLAN_ETHERTYPE_AND; + * + * virtchnl_vlan_caps.offloads.insertion_support.outer = + * VIRTCHNL_VLAN_TOGGLE | + * VIRTCHNL_VLAN_ETHERTYPE_8100 | + * VIRTCHNL_VLAN_ETHERTYPE_88A8 | + * VIRTCHNL_VLAN_ETHERTYPE_AND; + * + * In order to enable outer VLAN stripping for 0x8100 and 0x88a8 VLANs, the VF + * would populate the virthcnl_vlan_offload_structure in the following manner + * and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message. + * + * virtchnl_vlan_setting.outer_ethertype_setting = + * VIRTHCNL_VLAN_ETHERTYPE_8100 | + * VIRTHCNL_VLAN_ETHERTYPE_88A8; + * + * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on + * initialization. + * + * There is also the case where a PF and the underlying hardware can support + * VLAN offloads on multiple ethertypes, but not concurrently. For example, if + * the PF populates the virtchnl_vlan_caps.offloads in the following manner the + * VF will be allowed to enable and/or disable 0x8100 XOR 0x88a8 outer VLAN + * offloads. The ethertypes must match for stripping and insertion. + * + * virtchnl_vlan_caps.offloads.stripping_support.outer = + * VIRTCHNL_VLAN_TOGGLE | + * VIRTCHNL_VLAN_ETHERTYPE_8100 | + * VIRTCHNL_VLAN_ETHERTYPE_88A8 | + * VIRTCHNL_VLAN_ETHERTYPE_XOR; + * + * virtchnl_vlan_caps.offloads.insertion_support.outer = + * VIRTCHNL_VLAN_TOGGLE | + * VIRTCHNL_VLAN_ETHERTYPE_8100 | + * VIRTCHNL_VLAN_ETHERTYPE_88A8 | + * VIRTCHNL_VLAN_ETHERTYPE_XOR; + * + * virtchnl_vlan_caps.offloads.ethertype_match = + * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION; + * + * In order to enable outer VLAN stripping for 0x88a8 VLANs, the VF would + * populate the virtchnl_vlan_setting structure in the following manner and send + * the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2. Also, this will change the + * ethertype for VLAN insertion if it's enabled. So, for completeness, a + * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 with the same ethertype should be sent. + * + * virtchnl_vlan_setting.outer_ethertype_setting = VIRTHCNL_VLAN_ETHERTYPE_88A8; + * + * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on + * initialization. + * + * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2 + * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 + * + * VF sends this message to enable or disable VLAN filtering. It also needs to + * specify an ethertype. The VF knows which VLAN ethertypes are allowed and + * whether or not it's allowed to enable/disable filtering via the + * VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to + * parse the virtchnl_vlan_caps.filtering fields to determine which, if any, + * filtering messages are allowed. + * + * For example, if the PF populates the virtchnl_vlan_caps.filtering in the + * following manner the VF will be allowed to enable/disable 0x8100 and 0x88a8 + * outer VLAN filtering together. Note, that the VIRTCHNL_VLAN_ETHERTYPE_AND + * means that all filtering ethertypes will to be enabled and disabled together + * regardless of the request from the VF. This means that the underlying + * hardware only supports VLAN filtering for all VLAN the specified ethertypes + * or none of them. + * + * virtchnl_vlan_caps.filtering.filtering_support.outer = + * VIRTCHNL_VLAN_TOGGLE | + * VIRTCHNL_VLAN_ETHERTYPE_8100 | + * VIRTHCNL_VLAN_ETHERTYPE_88A8 | + * VIRTCHNL_VLAN_ETHERTYPE_9100 | + * VIRTCHNL_VLAN_ETHERTYPE_AND; + * + * In order to enable outer VLAN filtering for 0x88a8 and 0x8100 VLANs (0x9100 + * VLANs aren't supported by the VF driver), the VF would populate the + * virtchnl_vlan_setting structure in the following manner and send the + * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2. The same message format would be used + * to disable outer VLAN filtering for 0x88a8 and 0x8100 VLANs, but the + * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 opcode is used. + * + * virtchnl_vlan_setting.outer_ethertype_setting = + * VIRTCHNL_VLAN_ETHERTYPE_8100 | + * VIRTCHNL_VLAN_ETHERTYPE_88A8; + * + */ +struct virtchnl_vlan_setting { + u32 outer_ethertype_setting; + u32 inner_ethertype_setting; + u16 vport_id; + u8 pad[6]; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_setting); + +/* VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE + * VF sends VSI id and flags. + * PF returns status code in retval. + * Note: we assume that broadcast accept mode is always enabled. + */ +struct virtchnl_promisc_info { + u16 vsi_id; + u16 flags; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(4, virtchnl_promisc_info); + +#define FLAG_VF_UNICAST_PROMISC 0x00000001 +#define FLAG_VF_MULTICAST_PROMISC 0x00000002 + +/* VIRTCHNL_OP_GET_STATS + * VF sends this message to request stats for the selected VSI. VF uses + * the virtchnl_queue_select struct to specify the VSI. The queue_id + * field is ignored by the PF. + * + * PF replies with struct virtchnl_eth_stats in an external buffer. + */ + +struct virtchnl_eth_stats { + u64 rx_bytes; /* received bytes */ + u64 rx_unicast; /* received unicast pkts */ + u64 rx_multicast; /* received multicast pkts */ + u64 rx_broadcast; /* received broadcast pkts */ + u64 rx_discards; + u64 rx_unknown_protocol; + u64 tx_bytes; /* transmitted bytes */ + u64 tx_unicast; /* transmitted unicast pkts */ + u64 tx_multicast; /* transmitted multicast pkts */ + u64 tx_broadcast; /* transmitted broadcast pkts */ + u64 tx_discards; + u64 tx_errors; +}; + +/* VIRTCHNL_OP_CONFIG_RSS_KEY + * VIRTCHNL_OP_CONFIG_RSS_LUT + * VF sends these messages to configure RSS. Only supported if both PF + * and VF drivers set the VIRTCHNL_VF_OFFLOAD_RSS_PF bit during + * configuration negotiation. If this is the case, then the RSS fields in + * the VF resource struct are valid. + * Both the key and LUT are initialized to 0 by the PF, meaning that + * RSS is effectively disabled until set up by the VF. + */ +struct virtchnl_rss_key { + u16 vsi_id; + u16 key_len; + u8 key[1]; /* RSS hash key, packed bytes */ +}; + +VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_key); + +struct virtchnl_rss_lut { + u16 vsi_id; + u16 lut_entries; + u8 lut[1]; /* RSS lookup table */ +}; + +VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_lut); + +/* enum virthcnl_hash_filter + * + * Bits defining the hash filters in the hena field of the virtchnl_rss_hena + * structure. Each bit indicates a specific hash filter for RSS. + * + * Note that not all bits are supported on all hardware. The VF should use + * VIRTCHNL_OP_GET_RSS_HENA_CAPS to determine which bits the PF is capable of + * before using VIRTCHNL_OP_SET_RSS_HENA to enable specific filters. + */ +enum virtchnl_hash_filter { + /* Bits 0 through 28 are reserved for future use */ + /* Bit 29, 30, and 32 are not supported on XL710 a X710 */ + VIRTCHNL_HASH_FILTER_UNICAST_IPV4_UDP = 29, + VIRTCHNL_HASH_FILTER_MULTICAST_IPV4_UDP = 30, + VIRTCHNL_HASH_FILTER_IPV4_UDP = 31, + VIRTCHNL_HASH_FILTER_IPV4_TCP_SYN_NO_ACK = 32, + VIRTCHNL_HASH_FILTER_IPV4_TCP = 33, + VIRTCHNL_HASH_FILTER_IPV4_SCTP = 34, + VIRTCHNL_HASH_FILTER_IPV4_OTHER = 35, + VIRTCHNL_HASH_FILTER_FRAG_IPV4 = 36, + /* Bits 37 and 38 are reserved for future use */ + /* Bit 39, 40, and 42 are not supported on XL710 a X710 */ + VIRTCHNL_HASH_FILTER_UNICAST_IPV6_UDP = 39, + VIRTCHNL_HASH_FILTER_MULTICAST_IPV6_UDP = 40, + VIRTCHNL_HASH_FILTER_IPV6_UDP = 41, + VIRTCHNL_HASH_FILTER_IPV6_TCP_SYN_NO_ACK = 42, + VIRTCHNL_HASH_FILTER_IPV6_TCP = 43, + VIRTCHNL_HASH_FILTER_IPV6_SCTP = 44, + VIRTCHNL_HASH_FILTER_IPV6_OTHER = 45, + VIRTCHNL_HASH_FILTER_FRAG_IPV6 = 46, + /* Bit 37 is reserved for future use */ + VIRTCHNL_HASH_FILTER_FCOE_OX = 48, + VIRTCHNL_HASH_FILTER_FCOE_RX = 49, + VIRTCHNL_HASH_FILTER_FCOE_OTHER = 50, + /* Bits 51 through 62 are reserved for future use */ + VIRTCHNL_HASH_FILTER_L2_PAYLOAD = 63, +}; + +#define VIRTCHNL_HASH_FILTER_INVALID (0) + +/* VIRTCHNL_OP_GET_RSS_HENA_CAPS + * VIRTCHNL_OP_SET_RSS_HENA + * VF sends these messages to get and set the hash filter enable bits for RSS. + * By default, the PF sets these to all possible traffic types that the + * hardware supports. The VF can query this value if it wants to change the + * traffic types that are hashed by the hardware. + */ +struct virtchnl_rss_hena { + /* see enum virtchnl_hash_filter */ + u64 hena; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_rss_hena); + +/* Type of RSS algorithm */ +enum virtchnl_rss_algorithm { + VIRTCHNL_RSS_ALG_TOEPLITZ_ASYMMETRIC = 0, + VIRTCHNL_RSS_ALG_R_ASYMMETRIC = 1, + VIRTCHNL_RSS_ALG_TOEPLITZ_SYMMETRIC = 2, + VIRTCHNL_RSS_ALG_XOR_SYMMETRIC = 3, +}; + +/* This is used by PF driver to enforce how many channels can be supported. + * When ADQ_V2 capability is negotiated, it will allow 16 channels otherwise + * PF driver will allow only max 4 channels + */ +#define VIRTCHNL_MAX_ADQ_CHANNELS 4 +#define VIRTCHNL_MAX_ADQ_V2_CHANNELS 16 + +/* VIRTCHNL_OP_ENABLE_CHANNELS + * VIRTCHNL_OP_DISABLE_CHANNELS + * VF sends these messages to enable or disable channels based on + * the user specified queue count and queue offset for each traffic class. + * This struct encompasses all the information that the PF needs from + * VF to create a channel. + */ +struct virtchnl_channel_info { + u16 count; /* number of queues in a channel */ + u16 offset; /* queues in a channel start from 'offset' */ + u32 pad; + u64 max_tx_rate; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_channel_info); + +struct virtchnl_tc_info { + u32 num_tc; + u32 pad; + struct virtchnl_channel_info list[1]; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_tc_info); + +/* VIRTCHNL_ADD_CLOUD_FILTER + * VIRTCHNL_DEL_CLOUD_FILTER + * VF sends these messages to add or delete a cloud filter based on the + * user specified match and action filters. These structures encompass + * all the information that the PF needs from the VF to add/delete a + * cloud filter. + */ + +struct virtchnl_l4_spec { + u8 src_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS]; + u8 dst_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS]; + /* vlan_prio is part of this 16 bit field even from OS perspective + * vlan_id:12 is actual vlan_id, then vlanid:bit14..12 is vlan_prio + * in future, when decided to offload vlan_prio, pass that information + * as part of the "vlan_id" field, Bit14..12 + */ + __be16 vlan_id; + __be16 pad; /* reserved for future use */ + __be32 src_ip[4]; + __be32 dst_ip[4]; + __be16 src_port; + __be16 dst_port; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(52, virtchnl_l4_spec); + +union virtchnl_flow_spec { + struct virtchnl_l4_spec tcp_spec; + u8 buffer[128]; /* reserved for future use */ +}; + +VIRTCHNL_CHECK_UNION_LEN(128, virtchnl_flow_spec); + +enum virtchnl_action { + /* action types */ + VIRTCHNL_ACTION_DROP = 0, + VIRTCHNL_ACTION_TC_REDIRECT, + VIRTCHNL_ACTION_PASSTHRU, + VIRTCHNL_ACTION_QUEUE, + VIRTCHNL_ACTION_Q_REGION, + VIRTCHNL_ACTION_MARK, + VIRTCHNL_ACTION_COUNT, +}; + +enum virtchnl_flow_type { + /* flow types */ + VIRTCHNL_TCP_V4_FLOW = 0, + VIRTCHNL_TCP_V6_FLOW, + VIRTCHNL_UDP_V4_FLOW, + VIRTCHNL_UDP_V6_FLOW, +}; + +struct virtchnl_filter { + union virtchnl_flow_spec data; + union virtchnl_flow_spec mask; + + /* see enum virtchnl_flow_type */ + s32 flow_type; + + /* see enum virtchnl_action */ + s32 action; + u32 action_meta; + u8 field_flags; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(272, virtchnl_filter); + +struct virtchnl_shaper_bw { + /* Unit is Kbps */ + u32 committed; + u32 peak; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_shaper_bw); + + + +/* VIRTCHNL_OP_EVENT + * PF sends this message to inform the VF driver of events that may affect it. + * No direct response is expected from the VF, though it may generate other + * messages in response to this one. + */ +enum virtchnl_event_codes { + VIRTCHNL_EVENT_UNKNOWN = 0, + VIRTCHNL_EVENT_LINK_CHANGE, + VIRTCHNL_EVENT_RESET_IMPENDING, + VIRTCHNL_EVENT_PF_DRIVER_CLOSE, +}; + +#define PF_EVENT_SEVERITY_INFO 0 +#define PF_EVENT_SEVERITY_ATTENTION 1 +#define PF_EVENT_SEVERITY_ACTION_REQUIRED 2 +#define PF_EVENT_SEVERITY_CERTAIN_DOOM 255 + +struct virtchnl_pf_event { + /* see enum virtchnl_event_codes */ + s32 event; + union { + /* If the PF driver does not support the new speed reporting + * capabilities then use link_event else use link_event_adv to + * get the speed and link information. The ability to understand + * new speeds is indicated by setting the capability flag + * VIRTCHNL_VF_CAP_ADV_LINK_SPEED in vf_cap_flags parameter + * in virtchnl_vf_resource struct and can be used to determine + * which link event struct to use below. + */ + struct { + enum virtchnl_link_speed link_speed; + bool link_status; + u8 pad[3]; + } link_event; + struct { + /* link_speed provided in Mbps */ + u32 link_speed; + u8 link_status; + u8 pad[3]; + } link_event_adv; + } event_data; + + s32 severity; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_pf_event); + + +/* VF reset states - these are written into the RSTAT register: + * VFGEN_RSTAT on the VF + * When the PF initiates a reset, it writes 0 + * When the reset is complete, it writes 1 + * When the PF detects that the VF has recovered, it writes 2 + * VF checks this register periodically to determine if a reset has occurred, + * then polls it to know when the reset is complete. + * If either the PF or VF reads the register while the hardware + * is in a reset state, it will return DEADBEEF, which, when masked + * will result in 3. + */ +enum virtchnl_vfr_states { + VIRTCHNL_VFR_INPROGRESS = 0, + VIRTCHNL_VFR_COMPLETED, + VIRTCHNL_VFR_VFACTIVE, +}; + +#define VIRTCHNL_MAX_NUM_PROTO_HDRS 32 +#define VIRTCHNL_MAX_SIZE_RAW_PACKET 1024 +#define PROTO_HDR_SHIFT 5 +#define PROTO_HDR_FIELD_START(proto_hdr_type) \ + (proto_hdr_type << PROTO_HDR_SHIFT) +#define PROTO_HDR_FIELD_MASK ((1UL << PROTO_HDR_SHIFT) - 1) + +/* VF use these macros to configure each protocol header. + * Specify which protocol headers and protocol header fields base on + * virtchnl_proto_hdr_type and virtchnl_proto_hdr_field. + * @param hdr: a struct of virtchnl_proto_hdr + * @param hdr_type: ETH/IPV4/TCP, etc + * @param field: SRC/DST/TEID/SPI, etc + */ +#define VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, field) \ + ((hdr)->field_selector |= BIT((field) & PROTO_HDR_FIELD_MASK)) +#define VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, field) \ + ((hdr)->field_selector &= ~BIT((field) & PROTO_HDR_FIELD_MASK)) +#define VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val) \ + ((hdr)->field_selector & BIT((val) & PROTO_HDR_FIELD_MASK)) +#define VIRTCHNL_GET_PROTO_HDR_FIELD(hdr) ((hdr)->field_selector) + +#define VIRTCHNL_ADD_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \ + (VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, \ + VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field)) +#define VIRTCHNL_DEL_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \ + (VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, \ + VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field)) + +#define VIRTCHNL_SET_PROTO_HDR_TYPE(hdr, hdr_type) \ + ((hdr)->type = VIRTCHNL_PROTO_HDR_ ## hdr_type) +#define VIRTCHNL_GET_PROTO_HDR_TYPE(hdr) \ + (((hdr)->type) >> PROTO_HDR_SHIFT) +#define VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) \ + ((hdr)->type == ((s32)((val) >> PROTO_HDR_SHIFT))) +#define VIRTCHNL_TEST_PROTO_HDR(hdr, val) \ + (VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) && \ + VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val)) + +/* Protocol header type within a packet segment. A segment consists of one or + * more protocol headers that make up a logical group of protocol headers. Each + * logical group of protocol headers encapsulates or is encapsulated using/by + * tunneling or encapsulation protocols for network virtualization. + */ +enum virtchnl_proto_hdr_type { + VIRTCHNL_PROTO_HDR_NONE, + VIRTCHNL_PROTO_HDR_ETH, + VIRTCHNL_PROTO_HDR_S_VLAN, + VIRTCHNL_PROTO_HDR_C_VLAN, + VIRTCHNL_PROTO_HDR_IPV4, + VIRTCHNL_PROTO_HDR_IPV6, + VIRTCHNL_PROTO_HDR_TCP, + VIRTCHNL_PROTO_HDR_UDP, + VIRTCHNL_PROTO_HDR_SCTP, + VIRTCHNL_PROTO_HDR_GTPU_IP, + VIRTCHNL_PROTO_HDR_GTPU_EH, + VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN, + VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP, + VIRTCHNL_PROTO_HDR_PPPOE, + VIRTCHNL_PROTO_HDR_L2TPV3, + VIRTCHNL_PROTO_HDR_ESP, + VIRTCHNL_PROTO_HDR_AH, + VIRTCHNL_PROTO_HDR_PFCP, + VIRTCHNL_PROTO_HDR_GTPC, + VIRTCHNL_PROTO_HDR_ECPRI, + VIRTCHNL_PROTO_HDR_L2TPV2, + VIRTCHNL_PROTO_HDR_PPP, + /* IPv4 and IPv6 Fragment header types are only associated to + * VIRTCHNL_PROTO_HDR_IPV4 and VIRTCHNL_PROTO_HDR_IPV6 respectively, + * cannot be used independently. + */ + VIRTCHNL_PROTO_HDR_IPV4_FRAG, + VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG, + VIRTCHNL_PROTO_HDR_GRE, +}; + +/* Protocol header field within a protocol header. */ +enum virtchnl_proto_hdr_field { + /* ETHER */ + VIRTCHNL_PROTO_HDR_ETH_SRC = + PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ETH), + VIRTCHNL_PROTO_HDR_ETH_DST, + VIRTCHNL_PROTO_HDR_ETH_ETHERTYPE, + /* S-VLAN */ + VIRTCHNL_PROTO_HDR_S_VLAN_ID = + PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_S_VLAN), + /* C-VLAN */ + VIRTCHNL_PROTO_HDR_C_VLAN_ID = + PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_C_VLAN), + /* IPV4 */ + VIRTCHNL_PROTO_HDR_IPV4_SRC = + PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4), + VIRTCHNL_PROTO_HDR_IPV4_DST, + VIRTCHNL_PROTO_HDR_IPV4_DSCP, + VIRTCHNL_PROTO_HDR_IPV4_TTL, + VIRTCHNL_PROTO_HDR_IPV4_PROT, + VIRTCHNL_PROTO_HDR_IPV4_CHKSUM, + /* IPV6 */ + VIRTCHNL_PROTO_HDR_IPV6_SRC = + PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6), + VIRTCHNL_PROTO_HDR_IPV6_DST, + VIRTCHNL_PROTO_HDR_IPV6_TC, + VIRTCHNL_PROTO_HDR_IPV6_HOP_LIMIT, + VIRTCHNL_PROTO_HDR_IPV6_PROT, + /* IPV6 Prefix */ + VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_SRC, + VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_DST, + VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_SRC, + VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_DST, + VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_SRC, + VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_DST, + VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_SRC, + VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_DST, + VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_SRC, + VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_DST, + VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_SRC, + VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_DST, + /* TCP */ + VIRTCHNL_PROTO_HDR_TCP_SRC_PORT = + PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_TCP), + VIRTCHNL_PROTO_HDR_TCP_DST_PORT, + VIRTCHNL_PROTO_HDR_TCP_CHKSUM, + /* UDP */ + VIRTCHNL_PROTO_HDR_UDP_SRC_PORT = + PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_UDP), + VIRTCHNL_PROTO_HDR_UDP_DST_PORT, + VIRTCHNL_PROTO_HDR_UDP_CHKSUM, + /* SCTP */ + VIRTCHNL_PROTO_HDR_SCTP_SRC_PORT = + PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_SCTP), + VIRTCHNL_PROTO_HDR_SCTP_DST_PORT, + VIRTCHNL_PROTO_HDR_SCTP_CHKSUM, + /* GTPU_IP */ + VIRTCHNL_PROTO_HDR_GTPU_IP_TEID = + PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_IP), + /* GTPU_EH */ + VIRTCHNL_PROTO_HDR_GTPU_EH_PDU = + PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH), + VIRTCHNL_PROTO_HDR_GTPU_EH_QFI, + /* PPPOE */ + VIRTCHNL_PROTO_HDR_PPPOE_SESS_ID = + PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PPPOE), + /* L2TPV3 */ + VIRTCHNL_PROTO_HDR_L2TPV3_SESS_ID = + PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV3), + /* ESP */ + VIRTCHNL_PROTO_HDR_ESP_SPI = + PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ESP), + /* AH */ + VIRTCHNL_PROTO_HDR_AH_SPI = + PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_AH), + /* PFCP */ + VIRTCHNL_PROTO_HDR_PFCP_S_FIELD = + PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PFCP), + VIRTCHNL_PROTO_HDR_PFCP_SEID, + /* GTPC */ + VIRTCHNL_PROTO_HDR_GTPC_TEID = + PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPC), + /* ECPRI */ + VIRTCHNL_PROTO_HDR_ECPRI_MSG_TYPE = + PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ECPRI), + VIRTCHNL_PROTO_HDR_ECPRI_PC_RTC_ID, + /* IPv4 Dummy Fragment */ + VIRTCHNL_PROTO_HDR_IPV4_FRAG_PKID = + PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4_FRAG), + /* IPv6 Extension Fragment */ + VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG_PKID = + PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG), + /* GTPU_DWN/UP */ + VIRTCHNL_PROTO_HDR_GTPU_DWN_QFI = + PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN), + VIRTCHNL_PROTO_HDR_GTPU_UP_QFI = + PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP), + /* L2TPv2 */ + VIRTCHNL_PROTO_HDR_L2TPV2_SESS_ID = + PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV2), + VIRTCHNL_PROTO_HDR_L2TPV2_LEN_SESS_ID, +}; + +struct virtchnl_proto_hdr { + /* see enum virtchnl_proto_hdr_type */ + s32 type; + u32 field_selector; /* a bit mask to select field for header type */ + u8 buffer[64]; + /** + * binary buffer in network order for specific header type. + * For example, if type = VIRTCHNL_PROTO_HDR_IPV4, a IPv4 + * header is expected to be copied into the buffer. + */ +}; + +VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_proto_hdr); + +struct virtchnl_proto_hdrs { + u8 tunnel_level; + /** + * specify where protocol header start from. + * must be 0 when sending a raw packet request. + * 0 - from the outer layer + * 1 - from the first inner layer + * 2 - from the second inner layer + * .... + */ + int count; + /** + * number of proto layers, must < VIRTCHNL_MAX_NUM_PROTO_HDRS + * must be 0 for a raw packet request. + */ + union { + struct virtchnl_proto_hdr + proto_hdr[VIRTCHNL_MAX_NUM_PROTO_HDRS]; + struct { + u16 pkt_len; + u8 spec[VIRTCHNL_MAX_SIZE_RAW_PACKET]; + u8 mask[VIRTCHNL_MAX_SIZE_RAW_PACKET]; + } raw; + }; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(2312, virtchnl_proto_hdrs); + +struct virtchnl_rss_cfg { + struct virtchnl_proto_hdrs proto_hdrs; /* protocol headers */ + + /* see enum virtchnl_rss_algorithm; rss algorithm type */ + s32 rss_algorithm; + u8 reserved[128]; /* reserve for future */ +}; + +VIRTCHNL_CHECK_STRUCT_LEN(2444, virtchnl_rss_cfg); + +/* action configuration for FDIR */ +struct virtchnl_filter_action { + /* see enum virtchnl_action type */ + s32 type; + union { + /* used for queue and qgroup action */ + struct { + u16 index; + u8 region; + } queue; + /* used for count action */ + struct { + /* share counter ID with other flow rules */ + u8 shared; + u32 id; /* counter ID */ + } count; + /* used for mark action */ + u32 mark_id; + u8 reserve[32]; + } act_conf; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_filter_action); + +#define VIRTCHNL_MAX_NUM_ACTIONS 8 + +struct virtchnl_filter_action_set { + /* action number must be less then VIRTCHNL_MAX_NUM_ACTIONS */ + int count; + struct virtchnl_filter_action actions[VIRTCHNL_MAX_NUM_ACTIONS]; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(292, virtchnl_filter_action_set); + +/* pattern and action for FDIR rule */ +struct virtchnl_fdir_rule { + struct virtchnl_proto_hdrs proto_hdrs; + struct virtchnl_filter_action_set action_set; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(2604, virtchnl_fdir_rule); + +/* Status returned to VF after VF requests FDIR commands + * VIRTCHNL_FDIR_SUCCESS + * VF FDIR related request is successfully done by PF + * The request can be OP_ADD/DEL/QUERY_FDIR_FILTER. + * + * VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE + * OP_ADD_FDIR_FILTER request is failed due to no Hardware resource. + * + * VIRTCHNL_FDIR_FAILURE_RULE_EXIST + * OP_ADD_FDIR_FILTER request is failed due to the rule is already existed. + * + * VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT + * OP_ADD_FDIR_FILTER request is failed due to conflict with existing rule. + * + * VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST + * OP_DEL_FDIR_FILTER request is failed due to this rule doesn't exist. + * + * VIRTCHNL_FDIR_FAILURE_RULE_INVALID + * OP_ADD_FDIR_FILTER request is failed due to parameters validation + * or HW doesn't support. + * + * VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT + * OP_ADD/DEL_FDIR_FILTER request is failed due to timing out + * for programming. + * + * VIRTCHNL_FDIR_FAILURE_QUERY_INVALID + * OP_QUERY_FDIR_FILTER request is failed due to parameters validation, + * for example, VF query counter of a rule who has no counter action. + */ +enum virtchnl_fdir_prgm_status { + VIRTCHNL_FDIR_SUCCESS = 0, + VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE, + VIRTCHNL_FDIR_FAILURE_RULE_EXIST, + VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT, + VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST, + VIRTCHNL_FDIR_FAILURE_RULE_INVALID, + VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT, + VIRTCHNL_FDIR_FAILURE_QUERY_INVALID, +}; + +/* VIRTCHNL_OP_ADD_FDIR_FILTER + * VF sends this request to PF by filling out vsi_id, + * validate_only and rule_cfg. PF will return flow_id + * if the request is successfully done and return add_status to VF. + */ +struct virtchnl_fdir_add { + u16 vsi_id; /* INPUT */ + /* + * 1 for validating a fdir rule, 0 for creating a fdir rule. + * Validate and create share one ops: VIRTCHNL_OP_ADD_FDIR_FILTER. + */ + u16 validate_only; /* INPUT */ + u32 flow_id; /* OUTPUT */ + struct virtchnl_fdir_rule rule_cfg; /* INPUT */ + + /* see enum virtchnl_fdir_prgm_status; OUTPUT */ + s32 status; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(2616, virtchnl_fdir_add); + +/* VIRTCHNL_OP_DEL_FDIR_FILTER + * VF sends this request to PF by filling out vsi_id + * and flow_id. PF will return del_status to VF. + */ +struct virtchnl_fdir_del { + u16 vsi_id; /* INPUT */ + u16 pad; + u32 flow_id; /* INPUT */ + + /* see enum virtchnl_fdir_prgm_status; OUTPUT */ + s32 status; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_fdir_del); + +/* VIRTCHNL_OP_GET_QOS_CAPS + * VF sends this message to get its QoS Caps, such as + * TC number, Arbiter and Bandwidth. + */ +struct virtchnl_qos_cap_elem { + u8 tc_num; + u8 tc_prio; +#define VIRTCHNL_ABITER_STRICT 0 +#define VIRTCHNL_ABITER_ETS 2 + u8 arbiter; +#define VIRTCHNL_STRICT_WEIGHT 1 + u8 weight; + enum virtchnl_bw_limit_type type; + union { + struct virtchnl_shaper_bw shaper; + u8 pad2[32]; + }; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_qos_cap_elem); + +struct virtchnl_qos_cap_list { + u16 vsi_id; + u16 num_elem; + struct virtchnl_qos_cap_elem cap[1]; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(44, virtchnl_qos_cap_list); + +/* VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP + * VF sends message virtchnl_queue_tc_mapping to set queue to tc + * mapping for all the Tx and Rx queues with a specified VSI, and + * would get response about bitmap of valid user priorities + * associated with queues. + */ +struct virtchnl_queue_tc_mapping { + u16 vsi_id; + u16 num_tc; + u16 num_queue_pairs; + u8 pad[2]; + union { + struct { + u16 start_queue_id; + u16 queue_count; + } req; + struct { +#define VIRTCHNL_USER_PRIO_TYPE_UP 0 +#define VIRTCHNL_USER_PRIO_TYPE_DSCP 1 + u16 prio_type; + u16 valid_prio_bitmap; + } resp; + } tc[1]; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_tc_mapping); + +/* VIRTCHNL_OP_CONFIG_QUEUE_BW */ +struct virtchnl_queue_bw { + u16 queue_id; + u8 tc; + u8 pad; + struct virtchnl_shaper_bw shaper; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_bw); + +struct virtchnl_queues_bw_cfg { + u16 vsi_id; + u16 num_queues; + struct virtchnl_queue_bw cfg[1]; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queues_bw_cfg); + +/* queue types */ +enum virtchnl_queue_type { + VIRTCHNL_QUEUE_TYPE_TX = 0, + VIRTCHNL_QUEUE_TYPE_RX = 1, +}; + +/* structure to specify a chunk of contiguous queues */ +struct virtchnl_queue_chunk { + /* see enum virtchnl_queue_type */ + s32 type; + u16 start_queue_id; + u16 num_queues; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_queue_chunk); + +/* structure to specify several chunks of contiguous queues */ +struct virtchnl_queue_chunks { + u16 num_chunks; + u16 rsvd; + struct virtchnl_queue_chunk chunks[1]; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_chunks); + +/* VIRTCHNL_OP_ENABLE_QUEUES_V2 + * VIRTCHNL_OP_DISABLE_QUEUES_V2 + * + * These opcodes can be used if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in + * VIRTCHNL_OP_GET_VF_RESOURCES + * + * VF sends virtchnl_ena_dis_queues struct to specify the queues to be + * enabled/disabled in chunks. Also applicable to single queue RX or + * TX. PF performs requested action and returns status. + */ +struct virtchnl_del_ena_dis_queues { + u16 vport_id; + u16 pad; + struct virtchnl_queue_chunks chunks; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_del_ena_dis_queues); + +/* Virtchannel interrupt throttling rate index */ +enum virtchnl_itr_idx { + VIRTCHNL_ITR_IDX_0 = 0, + VIRTCHNL_ITR_IDX_1 = 1, + VIRTCHNL_ITR_IDX_NO_ITR = 3, +}; + +/* Queue to vector mapping */ +struct virtchnl_queue_vector { + u16 queue_id; + u16 vector_id; + u8 pad[4]; + + /* see enum virtchnl_itr_idx */ + s32 itr_idx; + + /* see enum virtchnl_queue_type */ + s32 queue_type; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queue_vector); + +/* VIRTCHNL_OP_MAP_QUEUE_VECTOR + * + * This opcode can be used only if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated + * in VIRTCHNL_OP_GET_VF_RESOURCES + * + * VF sends this message to map queues to vectors and ITR index registers. + * External data buffer contains virtchnl_queue_vector_maps structure + * that contains num_qv_maps of virtchnl_queue_vector structures. + * PF maps the requested queue vector maps after validating the queue and vector + * ids and returns a status code. + */ +struct virtchnl_queue_vector_maps { + u16 vport_id; + u16 num_qv_maps; + u8 pad[4]; + struct virtchnl_queue_vector qv_maps[1]; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_queue_vector_maps); + +struct virtchnl_quanta_cfg { + u16 quanta_size; + struct virtchnl_queue_chunk queue_select; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_quanta_cfg); + +/* VIRTCHNL_VF_CAP_PTP + * VIRTCHNL_OP_1588_PTP_GET_CAPS + * VIRTCHNL_OP_1588_PTP_GET_TIME + * VIRTCHNL_OP_1588_PTP_SET_TIME + * VIRTCHNL_OP_1588_PTP_ADJ_TIME + * VIRTCHNL_OP_1588_PTP_ADJ_FREQ + * VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP + * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS + * VIRTCHNL_OP_1588_PTP_SET_PIN_CFG + * VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP + * + * Support for offloading control of the device PTP hardware clock (PHC) is enabled + * by VIRTCHNL_VF_CAP_PTP. This capability allows a VF to request that PF + * enable Tx and Rx timestamps, and request access to read and/or write the + * PHC on the device, as well as query if the VF has direct access to the PHC + * time registers. + * + * The VF must set VIRTCHNL_VF_CAP_PTP in its capabilities when requesting + * resources. If the capability is set in reply, the VF must then send + * a VIRTCHNL_OP_1588_PTP_GET_CAPS request during initialization. The VF indicates + * what extended capabilities it wants by setting the appropriate flags in the + * caps field. The PF reply will indicate what features are enabled for + * that VF. + */ +#define VIRTCHNL_1588_PTP_CAP_TX_TSTAMP BIT(0) +#define VIRTCHNL_1588_PTP_CAP_RX_TSTAMP BIT(1) +#define VIRTCHNL_1588_PTP_CAP_READ_PHC BIT(2) +#define VIRTCHNL_1588_PTP_CAP_WRITE_PHC BIT(3) +#define VIRTCHNL_1588_PTP_CAP_PHC_REGS BIT(4) +#define VIRTCHNL_1588_PTP_CAP_PIN_CFG BIT(5) + +/** + * virtchnl_phc_regs + * + * Structure defines how the VF should access PHC related registers. The VF + * must request VIRTCHNL_1588_PTP_CAP_PHC_REGS. If the VF has access to PHC + * registers, the PF will reply with the capability flag set, and with this + * structure detailing what PCIe region and what offsets to use. If direct + * access is not available, this entire structure is reserved and the fields + * will be zero. + * + * If necessary in a future extension, a separate capability mutually + * exclusive with VIRTCHNL_1588_PTP_CAP_PHC_REGS might be used to change the + * entire format of this structure within virtchnl_ptp_caps. + * + * @clock_hi: Register offset of the high 32 bits of clock time + * @clock_lo: Register offset of the low 32 bits of clock time + * @pcie_region: The PCIe region the registers are located in. + * @rsvd: Reserved bits for future extension + */ +struct virtchnl_phc_regs { + u32 clock_hi; + u32 clock_lo; + u8 pcie_region; + u8 rsvd[15]; +}; +VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_regs); + +/* timestamp format enumeration + * + * VIRTCHNL_1588_PTP_TSTAMP_40BIT + * + * This format indicates a timestamp that uses the 40bit format from the + * flexible Rx descriptors. It is also the default Tx timestamp format used + * today. + * + * Such a timestamp has the following 40bit format: + * + * *--------------------------------*-------------------------------*-----------* + * | 32 bits of time in nanoseconds | 7 bits of sub-nanosecond time | valid bit | + * *--------------------------------*-------------------------------*-----------* + * + * The timestamp is passed in a u64, with the upper 24bits of the field + * reserved as zero. + * + * With this format, in order to report a full 64bit timestamp to userspace + * applications, the VF is responsible for performing timestamp extension by + * carefully comparing the timestamp with the PHC time. This can correctly + * be achieved with a recent cached copy of the PHC time by doing delta + * comparison between the 32bits of nanoseconds in the timestamp with the + * lower 32 bits of the clock time. For this to work, the cached PHC time + * must be from within 2^31 nanoseconds (~2.1 seconds) of when the timestamp + * was captured. + * + * VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS + * + * This format indicates a timestamp that is 64 bits of nanoseconds. + */ +enum virtchnl_ptp_tstamp_format { + VIRTCHNL_1588_PTP_TSTAMP_40BIT = 0, + VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS = 1, +}; + +/** + * virtchnl_ptp_caps + * + * Structure that defines the PTP capabilities available to the VF. The VF + * sends VIRTCHNL_OP_1588_PTP_GET_CAPS, and must fill in the ptp_caps field + * indicating what capabilities it is requesting. The PF will respond with the + * same message with the virtchnl_ptp_caps structure indicating what is + * enabled for the VF. + * + * @phc_regs: If VIRTCHNL_1588_PTP_CAP_PHC_REGS is set, contains information + * on the PHC related registers available to the VF. + * @caps: On send, VF sets what capabilities it requests. On reply, PF + * indicates what has been enabled for this VF. The PF shall not set + * bits which were not requested by the VF. + * @max_adj: The maximum adjustment capable of being requested by + * VIRTCHNL_OP_1588_PTP_ADJ_FREQ, in parts per billion. Note that 1 ppb + * is approximately 65.5 scaled_ppm. The PF shall clamp any + * frequency adjustment in VIRTCHNL_op_1588_ADJ_FREQ to +/- max_adj. + * Use of ppb in this field allows fitting the value into 4 bytes + * instead of potentially requiring 8 if scaled_ppm units were used. + * @tx_tstamp_idx: The Tx timestamp index to set in the transmit descriptor + * when requesting a timestamp for an outgoing packet. + * Reserved if VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is not enabled. + * @n_ext_ts: Number of external timestamp functions available. Reserved + * if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled. + * @n_per_out: Number of periodic output functions available. Reserved if + * VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled. + * @n_pins: Number of physical programmable pins able to be controlled. + * Reserved if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled. + * @tx_tstamp_format: Format of the Tx timestamps. Valid formats are defined + * by the virtchnl_ptp_tstamp enumeration. Note that Rx + * timestamps are tied to the descriptor format, and do not + * have a separate format field. + * @rsvd: Reserved bits for future extension. + * + * PTP capabilities + * + * VIRTCHNL_1588_PTP_CAP_TX_TSTAMP indicates that the VF can request transmit + * timestamps for packets in its transmit descriptors. If this is unset, + * transmit timestamp requests are ignored. Note that only one outstanding Tx + * timestamp request will be honored at a time. The PF shall handle receipt of + * the timestamp from the hardware, and will forward this to the VF by sending + * a VIRTCHNL_OP_1588_TX_TIMESTAMP message. + * + * VIRTCHNL_1588_PTP_CAP_RX_TSTAMP indicates that the VF receive queues have + * receive timestamps enabled in the flexible descriptors. Note that this + * requires a VF to also negotiate to enable advanced flexible descriptors in + * the receive path instead of the default legacy descriptor format. + * + * For a detailed description of the current Tx and Rx timestamp format, see + * the section on virtchnl_phc_tx_tstamp. Future extensions may indicate + * timestamp format in the capability structure. + * + * VIRTCHNL_1588_PTP_CAP_READ_PHC indicates that the VF may read the PHC time + * via the VIRTCHNL_OP_1588_PTP_GET_TIME command, or by directly reading PHC + * registers if VIRTCHNL_1588_PTP_CAP_PHC_REGS is also set. + * + * VIRTCHNL_1588_PTP_CAP_WRITE_PHC indicates that the VF may request updates + * to the PHC time via VIRTCHNL_OP_1588_PTP_SET_TIME, + * VIRTCHNL_OP_1588_PTP_ADJ_TIME, and VIRTCHNL_OP_1588_PTP_ADJ_FREQ. + * + * VIRTCHNL_1588_PTP_CAP_PHC_REGS indicates that the VF has direct access to + * certain PHC related registers, primarily for lower latency access to the + * PHC time. If this is set, the VF shall read the virtchnl_phc_regs section + * of the capabilities to determine the location of the clock registers. If + * this capability is not set, the entire 24 bytes of virtchnl_phc_regs is + * reserved as zero. Future extensions define alternative formats for this + * data, in which case they will be mutually exclusive with this capability. + * + * VIRTCHNL_1588_PTP_CAP_PIN_CFG indicates that the VF has the capability to + * control software defined pins. These pins can be assigned either as an + * input to timestamp external events, or as an output to cause a periodic + * signal output. + * + * Note that in the future, additional capability flags may be added which + * indicate additional extended support. All fields marked as reserved by this + * header will be set to zero. VF implementations should verify this to ensure + * that future extensions do not break compatibility. + */ +struct virtchnl_ptp_caps { + struct virtchnl_phc_regs phc_regs; + u32 caps; + s32 max_adj; + u8 tx_tstamp_idx; + u8 n_ext_ts; + u8 n_per_out; + u8 n_pins; + /* see enum virtchnl_ptp_tstamp_format */ + u8 tx_tstamp_format; + u8 rsvd[11]; +}; +VIRTCHNL_CHECK_STRUCT_LEN(48, virtchnl_ptp_caps); + +/** + * virtchnl_phc_time + * @time: PHC time in nanoseconds + * @rsvd: Reserved for future extension + * + * Structure sent with VIRTCHNL_OP_1588_PTP_SET_TIME and received with + * VIRTCHNL_OP_1588_PTP_GET_TIME. Contains the 64bits of PHC clock time in + * nanoseconds. + * + * VIRTCHNL_OP_1588_PTP_SET_TIME may be sent by the VF if + * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set. This will request that the PHC time + * be set to the requested value. This operation is non-atomic and thus does + * not adjust for the delay between request and completion. It is recommended + * that the VF use VIRTCHNL_OP_1588_PTP_ADJ_TIME and + * VIRTCHNL_OP_1588_PTP_ADJ_FREQ when possible to steer the PHC clock. + * + * VIRTCHNL_OP_1588_PTP_GET_TIME may be sent to request the current time of + * the PHC. This op is available in case direct access via the PHC registers + * is not available. + */ +struct virtchnl_phc_time { + u64 time; + u8 rsvd[8]; +}; +VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_time); + +/** + * virtchnl_phc_adj_time + * @delta: offset requested to adjust clock by + * @rsvd: reserved for future extension + * + * Sent with VIRTCHNL_OP_1588_PTP_ADJ_TIME. Used to request an adjustment of + * the clock time by the provided delta, with negative values representing + * subtraction. VIRTCHNL_OP_1588_PTP_ADJ_TIME may not be sent unless + * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set. + * + * The atomicity of this operation is not guaranteed. The PF should perform an + * atomic update using appropriate mechanisms if possible. However, this is + * not guaranteed. + */ +struct virtchnl_phc_adj_time { + s64 delta; + u8 rsvd[8]; +}; +VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_time); + +/** + * virtchnl_phc_adj_freq + * @scaled_ppm: frequency adjustment represented in scaled parts per million + * @rsvd: Reserved for future extension + * + * Sent with the VIRTCHNL_OP_1588_PTP_ADJ_FREQ to request an adjustment to the + * clock frequency. The adjustment is in scaled_ppm, which is parts per + * million with a 16bit binary fractional portion. 1 part per billion is + * approximately 65.5 scaled_ppm. + * + * ppm = scaled_ppm / 2^16 + * + * ppb = scaled_ppm * 1000 / 2^16 or + * + * ppb = scaled_ppm * 125 / 2^13 + * + * The PF shall clamp any adjustment request to plus or minus the specified + * max_adj in the PTP capabilities. + * + * Requests for adjustment are always based off of nominal clock frequency and + * not compounding. To reset clock frequency, send a request with a scaled_ppm + * of 0. + */ +struct virtchnl_phc_adj_freq { + s64 scaled_ppm; + u8 rsvd[8]; +}; +VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_freq); + +/** + * virtchnl_phc_tx_stamp + * @tstamp: timestamp value + * @rsvd: Reserved for future extension + * + * Sent along with VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP from the PF when a Tx + * timestamp for the index associated with this VF in the tx_tstamp_idx field + * is captured by hardware. + * + * If VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is set, the VF may request a timestamp + * for a packet in its transmit context descriptor by setting the appropriate + * flag and setting the timestamp index provided by the PF. On transmission, + * the timestamp will be captured and sent to the PF. The PF will forward this + * timestamp to the VF via the VIRTCHNL_1588_PTP_CAP_TX_TSTAMP op. + * + * The timestamp format is defined by the tx_tstamp_format field of the + * virtchnl_ptp_caps structure. + */ +struct virtchnl_phc_tx_tstamp { + u64 tstamp; + u8 rsvd[8]; +}; +VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_tx_tstamp); + +enum virtchnl_phc_pin_func { + VIRTCHNL_PHC_PIN_FUNC_NONE = 0, /* Not assigned to any function */ + VIRTCHNL_PHC_PIN_FUNC_EXT_TS = 1, /* Assigned to external timestamp */ + VIRTCHNL_PHC_PIN_FUNC_PER_OUT = 2, /* Assigned to periodic output */ +}; + +/* Length of the pin configuration data. All pin configurations belong within + * the same union and *must* have this length in bytes. + */ +#define VIRTCHNL_PIN_CFG_LEN 64 + +/* virtchnl_phc_ext_ts_mode + * + * Mode of the external timestamp, indicating which edges of the input signal + * to timestamp. + */ +enum virtchnl_phc_ext_ts_mode { + VIRTCHNL_PHC_EXT_TS_NONE = 0, + VIRTCHNL_PHC_EXT_TS_RISING_EDGE = 1, + VIRTCHNL_PHC_EXT_TS_FALLING_EDGE = 2, + VIRTCHNL_PHC_EXT_TS_BOTH_EDGES = 3, +}; + +/** + * virtchnl_phc_ext_ts + * @mode: mode of external timestamp request + * @rsvd: reserved for future extension + * + * External timestamp configuration. Defines the configuration for this + * external timestamp function. + * + * If mode is VIRTCHNL_PHC_EXT_TS_NONE, the function is essentially disabled, + * timestamping nothing. + * + * If mode is VIRTCHNL_PHC_EXT_TS_RISING_EDGE, the function shall timestamp + * the rising edge of the input when it transitions from low to high signal. + * + * If mode is VIRTCHNL_PHC_EXT_TS_FALLING_EDGE, the function shall timestamp + * the falling edge of the input when it transitions from high to low signal. + * + * If mode is VIRTCHNL_PHC_EXT_TS_BOTH_EDGES, the function shall timestamp + * both the rising and falling edge of the signal whenever it changes. + * + * The PF shall return an error if the requested mode cannot be implemented on + * the function. + */ +struct virtchnl_phc_ext_ts { + u8 mode; /* see virtchnl_phc_ext_ts_mode */ + u8 rsvd[63]; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_ext_ts); + +/* virtchnl_phc_per_out_flags + * + * Flags defining periodic output functionality. + */ +enum virtchnl_phc_per_out_flags { + VIRTCHNL_PHC_PER_OUT_PHASE_START = BIT(0), +}; + +/** + * virtchnl_phc_per_out + * @start: absolute start time (if VIRTCHNL_PHC_PER_OUT_PHASE_START unset) + * @phase: phase offset to start (if VIRTCHNL_PHC_PER_OUT_PHASE_START set) + * @period: time to complete a full clock cycle (low - > high -> low) + * @on: length of time the signal should stay high + * @flags: flags defining the periodic output operation. + * rsvd: reserved for future extension + * + * Configuration for a periodic output signal. Used to define the signal that + * should be generated on a given function. + * + * The period field determines the full length of the clock cycle, including + * both duration hold high transition and duration to hold low transition in + * nanoseconds. + * + * The on field determines how long the signal should remain high. For + * a traditional square wave clock that is on for some duration and off for + * the same duration, use an on length of precisely half the period. The duty + * cycle of the clock is period/on. + * + * If VIRTCHNL_PHC_PER_OUT_PHASE_START is unset, then the request is to start + * a clock an absolute time. This means that the clock should start precisely + * at the specified time in the start field. If the start time is in the past, + * then the periodic output should start at the next valid multiple of the + * period plus the start time: + * + * new_start = (n * period) + start + * (choose n such that new start is in the future) + * + * Note that the PF should not reject a start time in the past because it is + * possible that such a start time was valid when the request was made, but + * became invalid due to delay in programming the pin. + * + * If VIRTCHNL_PHC_PER_OUT_PHASE_START is set, then the request is to start + * the next multiple of the period plus the phase offset. The phase must be + * less than the period. In this case, the clock should start as soon possible + * at the next available multiple of the period. To calculate a start time + * when programming this mode, use: + * + * start = (n * period) + phase + * (choose n such that start is in the future) + * + * A period of zero should be treated as a request to disable the clock + * output. + */ +struct virtchnl_phc_per_out { + union { + u64 start; + u64 phase; + }; + u64 period; + u64 on; + u32 flags; + u8 rsvd[36]; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_per_out); + +/* virtchnl_phc_pin_cfg_flags + * + * Definition of bits in the flags field of the virtchnl_phc_pin_cfg + * structure. + */ +enum virtchnl_phc_pin_cfg_flags { + /* Valid for VIRTCHNL_OP_1588_PTP_SET_PIN_CFG. If set, indicates this + * is a request to verify if the function can be assigned to the + * provided pin. In this case, the ext_ts and per_out fields are + * ignored, and the PF response must be an error if the pin cannot be + * assigned to that function index. + */ + VIRTCHNL_PHC_PIN_CFG_VERIFY = BIT(0), +}; + +/** + * virtchnl_phc_set_pin + * @pin_index: The pin to get or set + * @func: the function type the pin is assigned to + * @func_index: the index of the function the pin is assigned to + * @ext_ts: external timestamp configuration + * @per_out: periodic output configuration + * @rsvd1: Reserved for future extension + * @rsvd2: Reserved for future extension + * + * Sent along with the VIRTCHNL_OP_1588_PTP_SET_PIN_CFG op. + * + * The VF issues a VIRTCHNL_OP_1588_PTP_SET_PIN_CFG to assign the pin to one + * of the functions. It must set the pin_index field, the func field, and + * the func_index field. The pin_index must be less than n_pins, and the + * func_index must be less than the n_ext_ts or n_per_out depending on which + * function type is selected. If func is for an external timestamp, the + * ext_ts field must be filled in with the desired configuration. Similarly, + * if the function is for a periodic output, the per_out field must be + * configured. + * + * If the VIRTCHNL_PHC_PIN_CFG_VERIFY bit of the flag field is set, this is + * a request only to verify the configuration, not to set it. In this case, + * the PF should simply report an error if the requested pin cannot be + * assigned to the requested function. This allows VF to determine whether or + * not a given function can be assigned to a specific pin. Other flag bits are + * currently reserved and must be verified as zero on both sides. They may be + * extended in the future. + */ +struct virtchnl_phc_set_pin { + u32 flags; /* see virtchnl_phc_pin_cfg_flags */ + u8 pin_index; + u8 func; /* see virtchnl_phc_pin_func */ + u8 func_index; + u8 rsvd1; + union { + struct virtchnl_phc_ext_ts ext_ts; + struct virtchnl_phc_per_out per_out; + }; + u8 rsvd2[8]; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_set_pin); + +/** + * virtchnl_phc_pin + * @pin_index: The pin to get or set + * @func: the function type the pin is assigned to + * @func_index: the index of the function the pin is assigned to + * @rsvd: Reserved for future extension + * @name: human readable pin name, supplied by PF on GET_PIN_CFGS + * + * Sent by the PF as part of the VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS response. + * + * The VF issues a VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS request to the PF in + * order to obtain the current pin configuration for all of the pins that were + * assigned to this VF. + * + * This structure details the pin configuration state, including a pin name + * and which function is assigned to the pin currently. + */ +struct virtchnl_phc_pin { + u8 pin_index; + u8 func; /* see virtchnl_phc_pin_func */ + u8 func_index; + u8 rsvd[5]; + char name[64]; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_phc_pin); + +/** + * virtchnl_phc_pin_cfg + * @len: length of the variable pin config array + * @pins: variable length pin configuration array + * + * Variable structure sent by the PF in reply to + * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS. The VF does not send this structure with + * its request of the operation. + * + * It is possible that the PF may need to send more pin configuration data + * than can be sent in one virtchnl message. To handle this, the PF should + * issue multiple VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS responses. Each response + * will indicate the number of pins it covers. The VF should be ready to wait + * for multiple responses until it has received a total length equal to the + * number of n_pins negotiated during extended PTP capabilities exchange. + */ +struct virtchnl_phc_get_pins { + u8 len; + u8 rsvd[7]; + struct virtchnl_phc_pin pins[1]; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_get_pins); + +/** + * virtchnl_phc_ext_stamp + * @tstamp: timestamp value + * @tstamp_rsvd: Reserved for future extension of the timestamp value. + * @tstamp_format: format of the timstamp + * @func_index: external timestamp function this timestamp is for + * @rsvd2: Reserved for future extension + * + * Sent along with the VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP from the PF when an + * external timestamp function is triggered. + * + * This will be sent only if one of the external timestamp functions is + * configured by the VF, and is only valid if VIRTCHNL_1588_PTP_CAP_PIN_CFG is + * negotiated with the PF. + * + * The timestamp format is defined by the tstamp_format field using the + * virtchnl_ptp_tstamp_format enumeration. The tstamp_rsvd field is + * exclusively reserved for possible future variants of the timestamp format, + * and its access will be controlled by the tstamp_format field. + */ +struct virtchnl_phc_ext_tstamp { + u64 tstamp; + u8 tstamp_rsvd[8]; + u8 tstamp_format; + u8 func_index; + u8 rsvd2[6]; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_ext_tstamp); + +/* Since VF messages are limited by u16 size, precalculate the maximum possible + * values of nested elements in virtchnl structures that virtual channel can + * possibly handle in a single message. + */ +enum virtchnl_vector_limits { + VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX = + ((u16)(~0) - sizeof(struct virtchnl_vsi_queue_config_info)) / + sizeof(struct virtchnl_queue_pair_info), + + VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX = + ((u16)(~0) - sizeof(struct virtchnl_irq_map_info)) / + sizeof(struct virtchnl_vector_map), + + VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX = + ((u16)(~0) - sizeof(struct virtchnl_ether_addr_list)) / + sizeof(struct virtchnl_ether_addr), + + VIRTCHNL_OP_ADD_DEL_VLAN_MAX = + ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list)) / + sizeof(u16), + + + VIRTCHNL_OP_ENABLE_CHANNELS_MAX = + ((u16)(~0) - sizeof(struct virtchnl_tc_info)) / + sizeof(struct virtchnl_channel_info), + + VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX = + ((u16)(~0) - sizeof(struct virtchnl_del_ena_dis_queues)) / + sizeof(struct virtchnl_queue_chunk), + + VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX = + ((u16)(~0) - sizeof(struct virtchnl_queue_vector_maps)) / + sizeof(struct virtchnl_queue_vector), + + VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX = + ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list_v2)) / + sizeof(struct virtchnl_vlan_filter), +}; + +/** + * virtchnl_vc_validate_vf_msg + * @ver: Virtchnl version info + * @v_opcode: Opcode for the message + * @msg: pointer to the msg buffer + * @msglen: msg length + * + * validate msg format against struct for each opcode + */ +static inline int +virtchnl_vc_validate_vf_msg(struct virtchnl_version_info *ver, u32 v_opcode, + u8 *msg, u16 msglen) +{ + bool err_msg_format = false; + u32 valid_len = 0; + + /* Validate message length. */ + switch (v_opcode) { + case VIRTCHNL_OP_VERSION: + valid_len = sizeof(struct virtchnl_version_info); + break; + case VIRTCHNL_OP_RESET_VF: + break; + case VIRTCHNL_OP_GET_VF_RESOURCES: + if (VF_IS_V11(ver)) + valid_len = sizeof(u32); + break; + case VIRTCHNL_OP_CONFIG_TX_QUEUE: + valid_len = sizeof(struct virtchnl_txq_info); + break; + case VIRTCHNL_OP_CONFIG_RX_QUEUE: + valid_len = sizeof(struct virtchnl_rxq_info); + break; + case VIRTCHNL_OP_CONFIG_VSI_QUEUES: + valid_len = sizeof(struct virtchnl_vsi_queue_config_info); + if (msglen >= valid_len) { + struct virtchnl_vsi_queue_config_info *vqc = + (struct virtchnl_vsi_queue_config_info *)msg; + + if (vqc->num_queue_pairs == 0 || vqc->num_queue_pairs > + VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX) { + err_msg_format = true; + break; + } + + valid_len += (vqc->num_queue_pairs * + sizeof(struct + virtchnl_queue_pair_info)); + } + break; + case VIRTCHNL_OP_CONFIG_IRQ_MAP: + valid_len = sizeof(struct virtchnl_irq_map_info); + if (msglen >= valid_len) { + struct virtchnl_irq_map_info *vimi = + (struct virtchnl_irq_map_info *)msg; + + if (vimi->num_vectors == 0 || vimi->num_vectors > + VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX) { + err_msg_format = true; + break; + } + + valid_len += (vimi->num_vectors * + sizeof(struct virtchnl_vector_map)); + } + break; + case VIRTCHNL_OP_ENABLE_QUEUES: + case VIRTCHNL_OP_DISABLE_QUEUES: + valid_len = sizeof(struct virtchnl_queue_select); + break; + case VIRTCHNL_OP_GET_MAX_RSS_QREGION: + break; + case VIRTCHNL_OP_ADD_ETH_ADDR: + case VIRTCHNL_OP_DEL_ETH_ADDR: + valid_len = sizeof(struct virtchnl_ether_addr_list); + if (msglen >= valid_len) { + struct virtchnl_ether_addr_list *veal = + (struct virtchnl_ether_addr_list *)msg; + + if (veal->num_elements == 0 || veal->num_elements > + VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX) { + err_msg_format = true; + break; + } + + valid_len += veal->num_elements * + sizeof(struct virtchnl_ether_addr); + } + break; + case VIRTCHNL_OP_ADD_VLAN: + case VIRTCHNL_OP_DEL_VLAN: + valid_len = sizeof(struct virtchnl_vlan_filter_list); + if (msglen >= valid_len) { + struct virtchnl_vlan_filter_list *vfl = + (struct virtchnl_vlan_filter_list *)msg; + + if (vfl->num_elements == 0 || vfl->num_elements > + VIRTCHNL_OP_ADD_DEL_VLAN_MAX) { + err_msg_format = true; + break; + } + + valid_len += vfl->num_elements * sizeof(u16); + } + break; + case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE: + valid_len = sizeof(struct virtchnl_promisc_info); + break; + case VIRTCHNL_OP_GET_STATS: + valid_len = sizeof(struct virtchnl_queue_select); + break; + case VIRTCHNL_OP_CONFIG_RSS_KEY: + valid_len = sizeof(struct virtchnl_rss_key); + if (msglen >= valid_len) { + struct virtchnl_rss_key *vrk = + (struct virtchnl_rss_key *)msg; + + if (vrk->key_len == 0) { + /* zero length is allowed as input */ + break; + } + + valid_len += vrk->key_len - 1; + } + break; + case VIRTCHNL_OP_CONFIG_RSS_LUT: + valid_len = sizeof(struct virtchnl_rss_lut); + if (msglen >= valid_len) { + struct virtchnl_rss_lut *vrl = + (struct virtchnl_rss_lut *)msg; + + if (vrl->lut_entries == 0) { + /* zero entries is allowed as input */ + break; + } + + valid_len += vrl->lut_entries - 1; + } + break; + case VIRTCHNL_OP_GET_RSS_HENA_CAPS: + break; + case VIRTCHNL_OP_SET_RSS_HENA: + valid_len = sizeof(struct virtchnl_rss_hena); + break; + case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING: + case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING: + break; + case VIRTCHNL_OP_REQUEST_QUEUES: + valid_len = sizeof(struct virtchnl_vf_res_request); + break; + case VIRTCHNL_OP_ENABLE_CHANNELS: + valid_len = sizeof(struct virtchnl_tc_info); + if (msglen >= valid_len) { + struct virtchnl_tc_info *vti = + (struct virtchnl_tc_info *)msg; + + if (vti->num_tc == 0 || vti->num_tc > + VIRTCHNL_OP_ENABLE_CHANNELS_MAX) { + err_msg_format = true; + break; + } + + valid_len += (vti->num_tc - 1) * + sizeof(struct virtchnl_channel_info); + } + break; + case VIRTCHNL_OP_DISABLE_CHANNELS: + break; + case VIRTCHNL_OP_ADD_CLOUD_FILTER: + case VIRTCHNL_OP_DEL_CLOUD_FILTER: + valid_len = sizeof(struct virtchnl_filter); + break; + case VIRTCHNL_OP_ADD_RSS_CFG: + case VIRTCHNL_OP_DEL_RSS_CFG: + valid_len = sizeof(struct virtchnl_rss_cfg); + break; + case VIRTCHNL_OP_ADD_FDIR_FILTER: + valid_len = sizeof(struct virtchnl_fdir_add); + break; + case VIRTCHNL_OP_DEL_FDIR_FILTER: + valid_len = sizeof(struct virtchnl_fdir_del); + break; + case VIRTCHNL_OP_GET_QOS_CAPS: + break; + case VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP: + valid_len = sizeof(struct virtchnl_queue_tc_mapping); + if (msglen >= valid_len) { + struct virtchnl_queue_tc_mapping *q_tc = + (struct virtchnl_queue_tc_mapping *)msg; + if (q_tc->num_tc == 0) { + err_msg_format = true; + break; + } + valid_len += (q_tc->num_tc - 1) * + sizeof(q_tc->tc[0]); + } + break; + case VIRTCHNL_OP_CONFIG_QUEUE_BW: + valid_len = sizeof(struct virtchnl_queues_bw_cfg); + if (msglen >= valid_len) { + struct virtchnl_queues_bw_cfg *q_bw = + (struct virtchnl_queues_bw_cfg *)msg; + if (q_bw->num_queues == 0) { + err_msg_format = true; + break; + } + valid_len += (q_bw->num_queues - 1) * + sizeof(q_bw->cfg[0]); + } + break; + case VIRTCHNL_OP_CONFIG_QUANTA: + valid_len = sizeof(struct virtchnl_quanta_cfg); + if (msglen >= valid_len) { + struct virtchnl_quanta_cfg *q_quanta = + (struct virtchnl_quanta_cfg *)msg; + if (q_quanta->quanta_size == 0 || + q_quanta->queue_select.num_queues == 0) { + err_msg_format = true; + break; + } + } + break; + case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS: + break; + case VIRTCHNL_OP_ADD_VLAN_V2: + case VIRTCHNL_OP_DEL_VLAN_V2: + valid_len = sizeof(struct virtchnl_vlan_filter_list_v2); + if (msglen >= valid_len) { + struct virtchnl_vlan_filter_list_v2 *vfl = + (struct virtchnl_vlan_filter_list_v2 *)msg; + + if (vfl->num_elements == 0 || vfl->num_elements > + VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX) { + err_msg_format = true; + break; + } + + valid_len += (vfl->num_elements - 1) * + sizeof(struct virtchnl_vlan_filter); + } + break; + case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2: + case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2: + case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2: + case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2: + case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2: + case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2: + valid_len = sizeof(struct virtchnl_vlan_setting); + break; + case VIRTCHNL_OP_1588_PTP_GET_CAPS: + valid_len = sizeof(struct virtchnl_ptp_caps); + break; + case VIRTCHNL_OP_1588_PTP_GET_TIME: + case VIRTCHNL_OP_1588_PTP_SET_TIME: + valid_len = sizeof(struct virtchnl_phc_time); + break; + case VIRTCHNL_OP_1588_PTP_ADJ_TIME: + valid_len = sizeof(struct virtchnl_phc_adj_time); + break; + case VIRTCHNL_OP_1588_PTP_ADJ_FREQ: + valid_len = sizeof(struct virtchnl_phc_adj_freq); + break; + case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP: + valid_len = sizeof(struct virtchnl_phc_tx_tstamp); + break; + case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG: + valid_len = sizeof(struct virtchnl_phc_set_pin); + break; + case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS: + break; + case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP: + valid_len = sizeof(struct virtchnl_phc_ext_tstamp); + break; + case VIRTCHNL_OP_ENABLE_QUEUES_V2: + case VIRTCHNL_OP_DISABLE_QUEUES_V2: + valid_len = sizeof(struct virtchnl_del_ena_dis_queues); + if (msglen >= valid_len) { + struct virtchnl_del_ena_dis_queues *qs = + (struct virtchnl_del_ena_dis_queues *)msg; + if (qs->chunks.num_chunks == 0 || + qs->chunks.num_chunks > VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX) { + err_msg_format = true; + break; + } + valid_len += (qs->chunks.num_chunks - 1) * + sizeof(struct virtchnl_queue_chunk); + } + break; + case VIRTCHNL_OP_MAP_QUEUE_VECTOR: + valid_len = sizeof(struct virtchnl_queue_vector_maps); + if (msglen >= valid_len) { + struct virtchnl_queue_vector_maps *v_qp = + (struct virtchnl_queue_vector_maps *)msg; + if (v_qp->num_qv_maps == 0 || + v_qp->num_qv_maps > VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) { + err_msg_format = true; + break; + } + valid_len += (v_qp->num_qv_maps - 1) * + sizeof(struct virtchnl_queue_vector); + } + break; + /* These are always errors coming from the VF. */ + case VIRTCHNL_OP_EVENT: + case VIRTCHNL_OP_UNKNOWN: + default: + return VIRTCHNL_STATUS_ERR_PARAM; + } + /* few more checks */ + if (err_msg_format || valid_len != msglen) + return VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH; + + return 0; +} +#endif /* _VIRTCHNL_H_ */ diff --git a/drivers/common/idpf/base/virtchnl2.h b/drivers/common/idpf/base/virtchnl2.h new file mode 100644 index 0000000000..d496f2388e --- /dev/null +++ b/drivers/common/idpf/base/virtchnl2.h @@ -0,0 +1,1462 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2001-2022 Intel Corporation + */ + +#ifndef _VIRTCHNL2_H_ +#define _VIRTCHNL2_H_ + +/* All opcodes associated with virtchnl 2 are prefixed with virtchnl2 or + * VIRTCHNL2. Any future opcodes, offloads/capabilities, structures, + * and defines must be prefixed with virtchnl2 or VIRTCHNL2 to avoid confusion. + */ + +#include "virtchnl2_lan_desc.h" + +/* Error Codes + * Note that many older versions of various iAVF drivers convert the reported + * status code directly into an iavf_status enumeration. For this reason, it + * is important that the values of these enumerations line up. + */ +#define VIRTCHNL2_STATUS_SUCCESS 0 +#define VIRTCHNL2_STATUS_ERR_PARAM -5 +#define VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH -38 + +/* These macros are used to generate compilation errors if a structure/union + * is not exactly the correct length. It gives a divide by zero error if the + * structure/union is not of the correct size, otherwise it creates an enum + * that is never used. + */ +#define VIRTCHNL2_CHECK_STRUCT_LEN(n, X) enum virtchnl2_static_assert_enum_##X \ + { virtchnl2_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) } +#define VIRTCHNL2_CHECK_UNION_LEN(n, X) enum virtchnl2_static_asset_enum_##X \ + { virtchnl2_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) } + +/* New major set of opcodes introduced and so leaving room for + * old misc opcodes to be added in future. Also these opcodes may only + * be used if both the PF and VF have successfully negotiated the + * VIRTCHNL version as 2.0 during VIRTCHNL22_OP_VERSION exchange. + */ +#define VIRTCHNL2_OP_UNKNOWN 0 +#define VIRTCHNL2_OP_VERSION 1 +#define VIRTCHNL2_OP_GET_CAPS 500 +#define VIRTCHNL2_OP_CREATE_VPORT 501 +#define VIRTCHNL2_OP_DESTROY_VPORT 502 +#define VIRTCHNL2_OP_ENABLE_VPORT 503 +#define VIRTCHNL2_OP_DISABLE_VPORT 504 +#define VIRTCHNL2_OP_CONFIG_TX_QUEUES 505 +#define VIRTCHNL2_OP_CONFIG_RX_QUEUES 506 +#define VIRTCHNL2_OP_ENABLE_QUEUES 507 +#define VIRTCHNL2_OP_DISABLE_QUEUES 508 +#define VIRTCHNL2_OP_ADD_QUEUES 509 +#define VIRTCHNL2_OP_DEL_QUEUES 510 +#define VIRTCHNL2_OP_MAP_QUEUE_VECTOR 511 +#define VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR 512 +#define VIRTCHNL2_OP_GET_RSS_KEY 513 +#define VIRTCHNL2_OP_SET_RSS_KEY 514 +#define VIRTCHNL2_OP_GET_RSS_LUT 515 +#define VIRTCHNL2_OP_SET_RSS_LUT 516 +#define VIRTCHNL2_OP_GET_RSS_HASH 517 +#define VIRTCHNL2_OP_SET_RSS_HASH 518 +#define VIRTCHNL2_OP_SET_SRIOV_VFS 519 +#define VIRTCHNL2_OP_ALLOC_VECTORS 520 +#define VIRTCHNL2_OP_DEALLOC_VECTORS 521 +#define VIRTCHNL2_OP_EVENT 522 +#define VIRTCHNL2_OP_GET_STATS 523 +#define VIRTCHNL2_OP_RESET_VF 524 + /* opcode 525 is reserved */ +#define VIRTCHNL2_OP_GET_PTYPE_INFO 526 + /* opcode 527 and 528 are reserved for VIRTCHNL2_OP_GET_PTYPE_ID and + * VIRTCHNL2_OP_GET_PTYPE_INFO_RAW + */ + /* opcodes 529, 530, and 531 are reserved */ +#define VIRTCHNL2_OP_CREATE_ADI 532 +#define VIRTCHNL2_OP_DESTROY_ADI 533 +#define VIRTCHNL2_OP_LOOPBACK 534 +#define VIRTCHNL2_OP_ADD_MAC_ADDR 535 +#define VIRTCHNL2_OP_DEL_MAC_ADDR 536 +#define VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE 537 + +#define VIRTCHNL2_RDMA_INVALID_QUEUE_IDX 0xFFFF + +/* VIRTCHNL2_VPORT_TYPE + * Type of virtual port + */ +#define VIRTCHNL2_VPORT_TYPE_DEFAULT 0 +#define VIRTCHNL2_VPORT_TYPE_SRIOV 1 +#define VIRTCHNL2_VPORT_TYPE_SIOV 2 +#define VIRTCHNL2_VPORT_TYPE_SUBDEV 3 +#define VIRTCHNL2_VPORT_TYPE_MNG 4 + +/* VIRTCHNL2_QUEUE_MODEL + * Type of queue model + * + * In the single queue model, the same transmit descriptor queue is used by + * software to post descriptors to hardware and by hardware to post completed + * descriptors to software. + * Likewise, the same receive descriptor queue is used by hardware to post + * completions to software and by software to post buffers to hardware. + */ +#define VIRTCHNL2_QUEUE_MODEL_SINGLE 0 +/* In the split queue model, hardware uses transmit completion queues to post + * descriptor/buffer completions to software, while software uses transmit + * descriptor queues to post descriptors to hardware. + * Likewise, hardware posts descriptor completions to the receive descriptor + * queue, while software uses receive buffer queues to post buffers to hardware. + */ +#define VIRTCHNL2_QUEUE_MODEL_SPLIT 1 + +/* VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS + * Checksum offload capability flags + */ +#define VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 BIT(0) +#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP BIT(1) +#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP BIT(2) +#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP BIT(3) +#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP BIT(4) +#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP BIT(5) +#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP BIT(6) +#define VIRTCHNL2_CAP_TX_CSUM_GENERIC BIT(7) +#define VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 BIT(8) +#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP BIT(9) +#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP BIT(10) +#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP BIT(11) +#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP BIT(12) +#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP BIT(13) +#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP BIT(14) +#define VIRTCHNL2_CAP_RX_CSUM_GENERIC BIT(15) +#define VIRTCHNL2_CAP_TX_CSUM_L3_SINGLE_TUNNEL BIT(16) +#define VIRTCHNL2_CAP_TX_CSUM_L3_DOUBLE_TUNNEL BIT(17) +#define VIRTCHNL2_CAP_RX_CSUM_L3_SINGLE_TUNNEL BIT(18) +#define VIRTCHNL2_CAP_RX_CSUM_L3_DOUBLE_TUNNEL BIT(19) +#define VIRTCHNL2_CAP_TX_CSUM_L4_SINGLE_TUNNEL BIT(20) +#define VIRTCHNL2_CAP_TX_CSUM_L4_DOUBLE_TUNNEL BIT(21) +#define VIRTCHNL2_CAP_RX_CSUM_L4_SINGLE_TUNNEL BIT(22) +#define VIRTCHNL2_CAP_RX_CSUM_L4_DOUBLE_TUNNEL BIT(23) + +/* VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS + * Segmentation offload capability flags + */ +#define VIRTCHNL2_CAP_SEG_IPV4_TCP BIT(0) +#define VIRTCHNL2_CAP_SEG_IPV4_UDP BIT(1) +#define VIRTCHNL2_CAP_SEG_IPV4_SCTP BIT(2) +#define VIRTCHNL2_CAP_SEG_IPV6_TCP BIT(3) +#define VIRTCHNL2_CAP_SEG_IPV6_UDP BIT(4) +#define VIRTCHNL2_CAP_SEG_IPV6_SCTP BIT(5) +#define VIRTCHNL2_CAP_SEG_GENERIC BIT(6) +#define VIRTCHNL2_CAP_SEG_TX_SINGLE_TUNNEL BIT(7) +#define VIRTCHNL2_CAP_SEG_TX_DOUBLE_TUNNEL BIT(8) + +/* VIRTCHNL2_RSS_FLOW_TYPE_CAPS + * Receive Side Scaling Flow type capability flags + */ +#define VIRTCHNL2_CAP_RSS_IPV4_TCP BIT(0) +#define VIRTCHNL2_CAP_RSS_IPV4_UDP BIT(1) +#define VIRTCHNL2_CAP_RSS_IPV4_SCTP BIT(2) +#define VIRTCHNL2_CAP_RSS_IPV4_OTHER BIT(3) +#define VIRTCHNL2_CAP_RSS_IPV6_TCP BIT(4) +#define VIRTCHNL2_CAP_RSS_IPV6_UDP BIT(5) +#define VIRTCHNL2_CAP_RSS_IPV6_SCTP BIT(6) +#define VIRTCHNL2_CAP_RSS_IPV6_OTHER BIT(7) +#define VIRTCHNL2_CAP_RSS_IPV4_AH BIT(8) +#define VIRTCHNL2_CAP_RSS_IPV4_ESP BIT(9) +#define VIRTCHNL2_CAP_RSS_IPV4_AH_ESP BIT(10) +#define VIRTCHNL2_CAP_RSS_IPV6_AH BIT(11) +#define VIRTCHNL2_CAP_RSS_IPV6_ESP BIT(12) +#define VIRTCHNL2_CAP_RSS_IPV6_AH_ESP BIT(13) + +/* VIRTCHNL2_HEADER_SPLIT_CAPS + * Header split capability flags + */ +/* for prepended metadata */ +#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 BIT(0) +/* all VLANs go into header buffer */ +#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 BIT(1) +#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 BIT(2) +#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6 BIT(3) + +/* VIRTCHNL2_RSC_OFFLOAD_CAPS + * Receive Side Coalescing offload capability flags + */ +#define VIRTCHNL2_CAP_RSC_IPV4_TCP BIT(0) +#define VIRTCHNL2_CAP_RSC_IPV4_SCTP BIT(1) +#define VIRTCHNL2_CAP_RSC_IPV6_TCP BIT(2) +#define VIRTCHNL2_CAP_RSC_IPV6_SCTP BIT(3) + +/* VIRTCHNL2_OTHER_CAPS + * Other capability flags + * SPLITQ_QSCHED: Queue based scheduling using split queue model + * TX_VLAN: VLAN tag insertion + * RX_VLAN: VLAN tag stripping + */ +#define VIRTCHNL2_CAP_RDMA BIT(0) +#define VIRTCHNL2_CAP_SRIOV BIT(1) +#define VIRTCHNL2_CAP_MACFILTER BIT(2) +#define VIRTCHNL2_CAP_FLOW_DIRECTOR BIT(3) +#define VIRTCHNL2_CAP_SPLITQ_QSCHED BIT(4) +#define VIRTCHNL2_CAP_CRC BIT(5) +#define VIRTCHNL2_CAP_ADQ BIT(6) +#define VIRTCHNL2_CAP_WB_ON_ITR BIT(7) +#define VIRTCHNL2_CAP_PROMISC BIT(8) +#define VIRTCHNL2_CAP_LINK_SPEED BIT(9) +#define VIRTCHNL2_CAP_INLINE_IPSEC BIT(10) +#define VIRTCHNL2_CAP_LARGE_NUM_QUEUES BIT(11) +/* require additional info */ +#define VIRTCHNL2_CAP_VLAN BIT(12) +#define VIRTCHNL2_CAP_PTP BIT(13) +#define VIRTCHNL2_CAP_ADV_RSS BIT(15) +#define VIRTCHNL2_CAP_FDIR BIT(16) +#define VIRTCHNL2_CAP_RX_FLEX_DESC BIT(17) +#define VIRTCHNL2_CAP_PTYPE BIT(18) +#define VIRTCHNL2_CAP_LOOPBACK BIT(19) +#define VIRTCHNL2_CAP_OEM BIT(20) + +/* VIRTCHNL2_DEVICE_TYPE */ +/* underlying device type */ +#define VIRTCHNL2_MEV_DEVICE 0 + +/* VIRTCHNL2_TXQ_SCHED_MODE + * Transmit Queue Scheduling Modes - Queue mode is the legacy mode i.e. inorder + * completions where descriptors and buffers are completed at the same time. + * Flow scheduling mode allows for out of order packet processing where + * descriptors are cleaned in order, but buffers can be completed out of order. + */ +#define VIRTCHNL2_TXQ_SCHED_MODE_QUEUE 0 +#define VIRTCHNL2_TXQ_SCHED_MODE_FLOW 1 + +/* VIRTCHNL2_TXQ_FLAGS + * Transmit Queue feature flags + * + * Enable rule miss completion type; packet completion for a packet + * sent on exception path; only relevant in flow scheduling mode + */ +#define VIRTCHNL2_TXQ_ENABLE_MISS_COMPL BIT(0) + +/* VIRTCHNL2_PEER_TYPE + * Transmit mailbox peer type + */ +#define VIRTCHNL2_RDMA_CPF 0 +#define VIRTCHNL2_NVME_CPF 1 +#define VIRTCHNL2_ATE_CPF 2 +#define VIRTCHNL2_LCE_CPF 3 + +/* VIRTCHNL2_RXQ_FLAGS + * Receive Queue Feature flags + */ +#define VIRTCHNL2_RXQ_RSC BIT(0) +#define VIRTCHNL2_RXQ_HDR_SPLIT BIT(1) +/* When set, packet descriptors are flushed by hardware immediately after + * processing each packet. + */ +#define VIRTCHNL2_RXQ_IMMEDIATE_WRITE_BACK BIT(2) +#define VIRTCHNL2_RX_DESC_SIZE_16BYTE BIT(3) +#define VIRTCHNL2_RX_DESC_SIZE_32BYTE BIT(4) + +/* VIRTCHNL2_RSS_ALGORITHM + * Type of RSS algorithm + */ +#define VIRTCHNL2_RSS_ALG_TOEPLITZ_ASYMMETRIC 0 +#define VIRTCHNL2_RSS_ALG_R_ASYMMETRIC 1 +#define VIRTCHNL2_RSS_ALG_TOEPLITZ_SYMMETRIC 2 +#define VIRTCHNL2_RSS_ALG_XOR_SYMMETRIC 3 + +/* VIRTCHNL2_EVENT_CODES + * Type of event + */ +#define VIRTCHNL2_EVENT_UNKNOWN 0 +#define VIRTCHNL2_EVENT_LINK_CHANGE 1 +/* These messages are only sent to PF from CP */ +#define VIRTCHNL2_EVENT_START_RESET_ADI 2 +#define VIRTCHNL2_EVENT_FINISH_RESET_ADI 3 + +/* VIRTCHNL2_QUEUE_TYPE + * Transmit and Receive queue types are valid in legacy as well as split queue + * models. With Split Queue model, 2 additional types are introduced - + * TX_COMPLETION and RX_BUFFER. In split queue model, receive corresponds to + * the queue where hardware posts completions. + */ +#define VIRTCHNL2_QUEUE_TYPE_TX 0 +#define VIRTCHNL2_QUEUE_TYPE_RX 1 +#define VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION 2 +#define VIRTCHNL2_QUEUE_TYPE_RX_BUFFER 3 +#define VIRTCHNL2_QUEUE_TYPE_CONFIG_TX 4 +#define VIRTCHNL2_QUEUE_TYPE_CONFIG_RX 5 +#define VIRTCHNL2_QUEUE_TYPE_P2P_TX 6 +#define VIRTCHNL2_QUEUE_TYPE_P2P_RX 7 +#define VIRTCHNL2_QUEUE_TYPE_P2P_TX_COMPLETION 8 +#define VIRTCHNL2_QUEUE_TYPE_P2P_RX_BUFFER 9 +#define VIRTCHNL2_QUEUE_TYPE_MBX_TX 10 +#define VIRTCHNL2_QUEUE_TYPE_MBX_RX 11 + +/* VIRTCHNL2_ITR_IDX + * Virtchannel interrupt throttling rate index + */ +#define VIRTCHNL2_ITR_IDX_0 0 +#define VIRTCHNL2_ITR_IDX_1 1 +#define VIRTCHNL2_ITR_IDX_2 2 +#define VIRTCHNL2_ITR_IDX_NO_ITR 3 + +/* VIRTCHNL2_VECTOR_LIMITS + * Since PF/VF messages are limited by __le16 size, precalculate the maximum + * possible values of nested elements in virtchnl structures that virtual + * channel can possibly handle in a single message. + */ + +#define VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX (\ + ((__le16)(~0) - sizeof(struct virtchnl2_del_ena_dis_queues)) / \ + sizeof(struct virtchnl2_queue_chunk)) + +#define VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX (\ + ((__le16)(~0) - sizeof(struct virtchnl2_queue_vector_maps)) / \ + sizeof(struct virtchnl2_queue_vector)) + +/* VIRTCHNL2_MAC_TYPE + * VIRTCHNL2_MAC_ADDR_PRIMARY + * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_PRIMARY for the + * primary/device unicast MAC address filter for VIRTCHNL2_OP_ADD_MAC_ADDR and + * VIRTCHNL2_OP_DEL_MAC_ADDR. This allows for the underlying control plane + * function to accurately track the MAC address and for VM/function reset. + * + * VIRTCHNL2_MAC_ADDR_EXTRA + * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_EXTRA for any extra + * unicast and/or multicast filters that are being added/deleted via + * VIRTCHNL2_OP_ADD_MAC_ADDR/VIRTCHNL2_OP_DEL_MAC_ADDR respectively. + */ +#define VIRTCHNL2_MAC_ADDR_PRIMARY 1 +#define VIRTCHNL2_MAC_ADDR_EXTRA 2 + +/* VIRTCHNL2_PROMISC_FLAGS + * Flags used for promiscuous mode + */ +#define VIRTCHNL2_UNICAST_PROMISC BIT(0) +#define VIRTCHNL2_MULTICAST_PROMISC BIT(1) + +/* VIRTCHNL2_PROTO_HDR_TYPE + * Protocol header type within a packet segment. A segment consists of one or + * more protocol headers that make up a logical group of protocol headers. Each + * logical group of protocol headers encapsulates or is encapsulated using/by + * tunneling or encapsulation protocols for network virtualization. + */ +/* VIRTCHNL2_PROTO_HDR_ANY is a mandatory protocol id */ +#define VIRTCHNL2_PROTO_HDR_ANY 0 +#define VIRTCHNL2_PROTO_HDR_PRE_MAC 1 +/* VIRTCHNL2_PROTO_HDR_MAC is a mandatory protocol id */ +#define VIRTCHNL2_PROTO_HDR_MAC 2 +#define VIRTCHNL2_PROTO_HDR_POST_MAC 3 +#define VIRTCHNL2_PROTO_HDR_ETHERTYPE 4 +#define VIRTCHNL2_PROTO_HDR_VLAN 5 +#define VIRTCHNL2_PROTO_HDR_SVLAN 6 +#define VIRTCHNL2_PROTO_HDR_CVLAN 7 +#define VIRTCHNL2_PROTO_HDR_MPLS 8 +#define VIRTCHNL2_PROTO_HDR_UMPLS 9 +#define VIRTCHNL2_PROTO_HDR_MMPLS 10 +#define VIRTCHNL2_PROTO_HDR_PTP 11 +#define VIRTCHNL2_PROTO_HDR_CTRL 12 +#define VIRTCHNL2_PROTO_HDR_LLDP 13 +#define VIRTCHNL2_PROTO_HDR_ARP 14 +#define VIRTCHNL2_PROTO_HDR_ECP 15 +#define VIRTCHNL2_PROTO_HDR_EAPOL 16 +#define VIRTCHNL2_PROTO_HDR_PPPOD 17 +#define VIRTCHNL2_PROTO_HDR_PPPOE 18 +/* VIRTCHNL2_PROTO_HDR_IPV4 is a mandatory protocol id */ +#define VIRTCHNL2_PROTO_HDR_IPV4 19 +/* IPv4 and IPv6 Fragment header types are only associated to + * VIRTCHNL2_PROTO_HDR_IPV4 and VIRTCHNL2_PROTO_HDR_IPV6 respectively, + * cannot be used independently. + */ +/* VIRTCHNL2_PROTO_HDR_IPV4_FRAG is a mandatory protocol id */ +#define VIRTCHNL2_PROTO_HDR_IPV4_FRAG 20 +/* VIRTCHNL2_PROTO_HDR_IPV6 is a mandatory protocol id */ +#define VIRTCHNL2_PROTO_HDR_IPV6 21 +/* VIRTCHNL2_PROTO_HDR_IPV6_FRAG is a mandatory protocol id */ +#define VIRTCHNL2_PROTO_HDR_IPV6_FRAG 22 +#define VIRTCHNL2_PROTO_HDR_IPV6_EH 23 +/* VIRTCHNL2_PROTO_HDR_UDP is a mandatory protocol id */ +#define VIRTCHNL2_PROTO_HDR_UDP 24 +/* VIRTCHNL2_PROTO_HDR_TCP is a mandatory protocol id */ +#define VIRTCHNL2_PROTO_HDR_TCP 25 +/* VIRTCHNL2_PROTO_HDR_SCTP is a mandatory protocol id */ +#define VIRTCHNL2_PROTO_HDR_SCTP 26 +/* VIRTCHNL2_PROTO_HDR_ICMP is a mandatory protocol id */ +#define VIRTCHNL2_PROTO_HDR_ICMP 27 +/* VIRTCHNL2_PROTO_HDR_ICMPV6 is a mandatory protocol id */ +#define VIRTCHNL2_PROTO_HDR_ICMPV6 28 +#define VIRTCHNL2_PROTO_HDR_IGMP 29 +#define VIRTCHNL2_PROTO_HDR_AH 30 +#define VIRTCHNL2_PROTO_HDR_ESP 31 +#define VIRTCHNL2_PROTO_HDR_IKE 32 +#define VIRTCHNL2_PROTO_HDR_NATT_KEEP 33 +/* VIRTCHNL2_PROTO_HDR_PAY is a mandatory protocol id */ +#define VIRTCHNL2_PROTO_HDR_PAY 34 +#define VIRTCHNL2_PROTO_HDR_L2TPV2 35 +#define VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL 36 +#define VIRTCHNL2_PROTO_HDR_L2TPV3 37 +#define VIRTCHNL2_PROTO_HDR_GTP 38 +#define VIRTCHNL2_PROTO_HDR_GTP_EH 39 +#define VIRTCHNL2_PROTO_HDR_GTPCV2 40 +#define VIRTCHNL2_PROTO_HDR_GTPC_TEID 41 +#define VIRTCHNL2_PROTO_HDR_GTPU 42 +#define VIRTCHNL2_PROTO_HDR_GTPU_UL 43 +#define VIRTCHNL2_PROTO_HDR_GTPU_DL 44 +#define VIRTCHNL2_PROTO_HDR_ECPRI 45 +#define VIRTCHNL2_PROTO_HDR_VRRP 46 +#define VIRTCHNL2_PROTO_HDR_OSPF 47 +/* VIRTCHNL2_PROTO_HDR_TUN is a mandatory protocol id */ +#define VIRTCHNL2_PROTO_HDR_TUN 48 +#define VIRTCHNL2_PROTO_HDR_GRE 49 +#define VIRTCHNL2_PROTO_HDR_NVGRE 50 +#define VIRTCHNL2_PROTO_HDR_VXLAN 51 +#define VIRTCHNL2_PROTO_HDR_VXLAN_GPE 52 +#define VIRTCHNL2_PROTO_HDR_GENEVE 53 +#define VIRTCHNL2_PROTO_HDR_NSH 54 +#define VIRTCHNL2_PROTO_HDR_QUIC 55 +#define VIRTCHNL2_PROTO_HDR_PFCP 56 +#define VIRTCHNL2_PROTO_HDR_PFCP_NODE 57 +#define VIRTCHNL2_PROTO_HDR_PFCP_SESSION 58 +#define VIRTCHNL2_PROTO_HDR_RTP 59 +#define VIRTCHNL2_PROTO_HDR_ROCE 60 +#define VIRTCHNL2_PROTO_HDR_ROCEV1 61 +#define VIRTCHNL2_PROTO_HDR_ROCEV2 62 +/* protocol ids up to 32767 are reserved for AVF use */ +/* 32768 - 65534 are used for user defined protocol ids */ +/* VIRTCHNL2_PROTO_HDR_NO_PROTO is a mandatory protocol id */ +#define VIRTCHNL2_PROTO_HDR_NO_PROTO 65535 + +#define VIRTCHNL2_VERSION_MAJOR_2 2 +#define VIRTCHNL2_VERSION_MINOR_0 0 + + +/* VIRTCHNL2_OP_VERSION + * VF posts its version number to the CP. CP responds with its version number + * in the same format, along with a return code. + * If there is a major version mismatch, then the VF cannot operate. + * If there is a minor version mismatch, then the VF can operate but should + * add a warning to the system log. + * + * This version opcode MUST always be specified as == 1, regardless of other + * changes in the API. The CP must always respond to this message without + * error regardless of version mismatch. + */ +struct virtchnl2_version_info { + u32 major; + u32 minor; +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_version_info); + +/* VIRTCHNL2_OP_GET_CAPS + * Dataplane driver sends this message to CP to negotiate capabilities and + * provides a virtchnl2_get_capabilities structure with its desired + * capabilities, max_sriov_vfs and num_allocated_vectors. + * CP responds with a virtchnl2_get_capabilities structure updated + * with allowed capabilities and the other fields as below. + * If PF sets max_sriov_vfs as 0, CP will respond with max number of VFs + * that can be created by this PF. For any other value 'n', CP responds + * with max_sriov_vfs set to min(n, x) where x is the max number of VFs + * allowed by CP's policy. max_sriov_vfs is not applicable for VFs. + * If dataplane driver sets num_allocated_vectors as 0, CP will respond with 1 + * which is default vector associated with the default mailbox. For any other + * value 'n', CP responds with a value <= n based on the CP's policy of + * max number of vectors for a PF. + * CP will respond with the vector ID of mailbox allocated to the PF in + * mailbox_vector_id and the number of itr index registers in itr_idx_map. + * It also responds with default number of vports that the dataplane driver + * should comeup with in default_num_vports and maximum number of vports that + * can be supported in max_vports + */ +struct virtchnl2_get_capabilities { + /* see VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS definitions */ + __le32 csum_caps; + + /* see VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS definitions */ + __le32 seg_caps; + + /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */ + __le32 hsplit_caps; + + /* see VIRTCHNL2_RSC_OFFLOAD_CAPS definitions */ + __le32 rsc_caps; + + /* see VIRTCHNL2_RSS_FLOW_TYPE_CAPS definitions */ + __le64 rss_caps; + + + /* see VIRTCHNL2_OTHER_CAPS definitions */ + __le64 other_caps; + + /* DYN_CTL register offset and vector id for mailbox provided by CP */ + __le32 mailbox_dyn_ctl; + __le16 mailbox_vector_id; + /* Maximum number of allocated vectors for the device */ + __le16 num_allocated_vectors; + + /* Maximum number of queues that can be supported */ + __le16 max_rx_q; + __le16 max_tx_q; + __le16 max_rx_bufq; + __le16 max_tx_complq; + + /* The PF sends the maximum VFs it is requesting. The CP responds with + * the maximum VFs granted. + */ + __le16 max_sriov_vfs; + + /* maximum number of vports that can be supported */ + __le16 max_vports; + /* default number of vports driver should allocate on load */ + __le16 default_num_vports; + + /* Max header length hardware can parse/checksum, in bytes */ + __le16 max_tx_hdr_size; + + /* Max number of scatter gather buffers that can be sent per transmit + * packet without needing to be linearized + */ + u8 max_sg_bufs_per_tx_pkt; + + /* see VIRTCHNL2_ITR_IDX definition */ + u8 itr_idx_map; + + __le16 pad1; + + /* version of Control Plane that is running */ + __le16 oem_cp_ver_major; + __le16 oem_cp_ver_minor; + /* see VIRTCHNL2_DEVICE_TYPE definitions */ + __le32 device_type; + + u8 reserved[12]; +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(80, virtchnl2_get_capabilities); + +struct virtchnl2_queue_reg_chunk { + /* see VIRTCHNL2_QUEUE_TYPE definitions */ + __le32 type; + __le32 start_queue_id; + __le32 num_queues; + __le32 pad; + + /* Queue tail register offset and spacing provided by CP */ + __le64 qtail_reg_start; + __le32 qtail_reg_spacing; + + u8 reserved[4]; +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_queue_reg_chunk); + +/* structure to specify several chunks of contiguous queues */ +struct virtchnl2_queue_reg_chunks { + __le16 num_chunks; + u8 reserved[6]; + struct virtchnl2_queue_reg_chunk chunks[1]; +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_reg_chunks); + +#define VIRTCHNL2_ETH_LENGTH_OF_ADDRESS 6 + +/* VIRTCHNL2_OP_CREATE_VPORT + * PF sends this message to CP to create a vport by filling in required + * fields of virtchnl2_create_vport structure. + * CP responds with the updated virtchnl2_create_vport structure containing the + * necessary fields followed by chunks which in turn will have an array of + * num_chunks entries of virtchnl2_queue_chunk structures. + */ +struct virtchnl2_create_vport { + /* PF/VF populates the following fields on request */ + /* see VIRTCHNL2_VPORT_TYPE definitions */ + __le16 vport_type; + + /* see VIRTCHNL2_QUEUE_MODEL definitions */ + __le16 txq_model; + + /* see VIRTCHNL2_QUEUE_MODEL definitions */ + __le16 rxq_model; + __le16 num_tx_q; + /* valid only if txq_model is split queue */ + __le16 num_tx_complq; + __le16 num_rx_q; + /* valid only if rxq_model is split queue */ + __le16 num_rx_bufq; + /* relative receive queue index to be used as default */ + __le16 default_rx_q; + /* used to align PF and CP in case of default multiple vports, it is + * filled by the PF and CP returns the same value, to enable the driver + * to support multiple asynchronous parallel CREATE_VPORT requests and + * associate a response to a specific request + */ + __le16 vport_index; + + /* CP populates the following fields on response */ + __le16 max_mtu; + __le32 vport_id; + u8 default_mac_addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS]; + __le16 pad; + /* see VIRTCHNL2_RX_DESC_IDS definitions */ + __le64 rx_desc_ids; + /* see VIRTCHNL2_TX_DESC_IDS definitions */ + __le64 tx_desc_ids; + +#define MAX_Q_REGIONS 16 + __le32 max_qs_per_qregion[MAX_Q_REGIONS]; + __le32 qregion_total_qs; + __le16 qregion_type; + __le16 pad2; + + /* see VIRTCHNL2_RSS_ALGORITHM definitions */ + __le32 rss_algorithm; + __le16 rss_key_size; + __le16 rss_lut_size; + + /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */ + __le32 rx_split_pos; + + u8 reserved[20]; + struct virtchnl2_queue_reg_chunks chunks; +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(192, virtchnl2_create_vport); + +/* VIRTCHNL2_OP_DESTROY_VPORT + * VIRTCHNL2_OP_ENABLE_VPORT + * VIRTCHNL2_OP_DISABLE_VPORT + * PF sends this message to CP to destroy, enable or disable a vport by filling + * in the vport_id in virtchnl2_vport structure. + * CP responds with the status of the requested operation. + */ +struct virtchnl2_vport { + __le32 vport_id; + u8 reserved[4]; +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_vport); + +/* Transmit queue config info */ +struct virtchnl2_txq_info { + __le64 dma_ring_addr; + + /* see VIRTCHNL2_QUEUE_TYPE definitions */ + __le32 type; + + __le32 queue_id; + /* valid only if queue model is split and type is transmit queue. Used + * in many to one mapping of transmit queues to completion queue + */ + __le16 relative_queue_id; + + /* see VIRTCHNL2_QUEUE_MODEL definitions */ + __le16 model; + + /* see VIRTCHNL2_TXQ_SCHED_MODE definitions */ + __le16 sched_mode; + + /* see VIRTCHNL2_TXQ_FLAGS definitions */ + __le16 qflags; + __le16 ring_len; + + /* valid only if queue model is split and type is transmit queue */ + __le16 tx_compl_queue_id; + /* valid only if queue type is VIRTCHNL2_QUEUE_TYPE_MAILBOX_TX */ + /* see VIRTCHNL2_PEER_TYPE definitions */ + __le16 peer_type; + /* valid only if queue type is CONFIG_TX and used to deliver messages + * for the respective CONFIG_TX queue + */ + __le16 peer_rx_queue_id; + + /* value ranges from 0 to 15 */ + __le16 qregion_id; + u8 pad[2]; + + /* Egress pasid is used for SIOV use case */ + __le32 egress_pasid; + __le32 egress_hdr_pasid; + __le32 egress_buf_pasid; + + u8 reserved[8]; +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_txq_info); + +/* VIRTCHNL2_OP_CONFIG_TX_QUEUES + * PF sends this message to set up parameters for one or more transmit queues. + * This message contains an array of num_qinfo instances of virtchnl2_txq_info + * structures. CP configures requested queues and returns a status code. If + * num_qinfo specified is greater than the number of queues associated with the + * vport, an error is returned and no queues are configured. + */ +struct virtchnl2_config_tx_queues { + __le32 vport_id; + __le16 num_qinfo; + + u8 reserved[10]; + struct virtchnl2_txq_info qinfo[1]; +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(72, virtchnl2_config_tx_queues); + +/* Receive queue config info */ +struct virtchnl2_rxq_info { + /* see VIRTCHNL2_RX_DESC_IDS definitions */ + __le64 desc_ids; + __le64 dma_ring_addr; + + /* see VIRTCHNL2_QUEUE_TYPE definitions */ + __le32 type; + __le32 queue_id; + + /* see QUEUE_MODEL definitions */ + __le16 model; + + __le16 hdr_buffer_size; + __le32 data_buffer_size; + __le32 max_pkt_size; + + __le16 ring_len; + u8 buffer_notif_stride; + u8 pad[1]; + + /* Applicable only for receive buffer queues */ + __le64 dma_head_wb_addr; + + /* Applicable only for receive completion queues */ + /* see VIRTCHNL2_RXQ_FLAGS definitions */ + __le16 qflags; + + __le16 rx_buffer_low_watermark; + + /* valid only in split queue model */ + __le16 rx_bufq1_id; + /* valid only in split queue model */ + __le16 rx_bufq2_id; + /* it indicates if there is a second buffer, rx_bufq2_id is valid only + * if this field is set + */ + u8 bufq2_ena; + u8 pad2; + + /* value ranges from 0 to 15 */ + __le16 qregion_id; + + /* Ingress pasid is used for SIOV use case */ + __le32 ingress_pasid; + __le32 ingress_hdr_pasid; + __le32 ingress_buf_pasid; + + u8 reserved[16]; +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(88, virtchnl2_rxq_info); + +/* VIRTCHNL2_OP_CONFIG_RX_QUEUES + * PF sends this message to set up parameters for one or more receive queues. + * This message contains an array of num_qinfo instances of virtchnl2_rxq_info + * structures. CP configures requested queues and returns a status code. + * If the number of queues specified is greater than the number of queues + * associated with the vport, an error is returned and no queues are configured. + */ +struct virtchnl2_config_rx_queues { + __le32 vport_id; + __le16 num_qinfo; + + u8 reserved[18]; + struct virtchnl2_rxq_info qinfo[1]; +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(112, virtchnl2_config_rx_queues); + +/* VIRTCHNL2_OP_ADD_QUEUES + * PF sends this message to request additional transmit/receive queues beyond + * the ones that were assigned via CREATE_VPORT request. virtchnl2_add_queues + * structure is used to specify the number of each type of queues. + * CP responds with the same structure with the actual number of queues assigned + * followed by num_chunks of virtchnl2_queue_chunk structures. + */ +struct virtchnl2_add_queues { + __le32 vport_id; + __le16 num_tx_q; + __le16 num_tx_complq; + __le16 num_rx_q; + __le16 num_rx_bufq; + u8 reserved[4]; + struct virtchnl2_queue_reg_chunks chunks; +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_add_queues); + +/* Structure to specify a chunk of contiguous interrupt vectors */ +struct virtchnl2_vector_chunk { + __le16 start_vector_id; + __le16 start_evv_id; + __le16 num_vectors; + __le16 pad1; + + /* Register offsets and spacing provided by CP. + * dynamic control registers are used for enabling/disabling/re-enabling + * interrupts and updating interrupt rates in the hotpath. Any changes + * to interrupt rates in the dynamic control registers will be reflected + * in the interrupt throttling rate registers. + * itrn registers are used to update interrupt rates for specific + * interrupt indices without modifying the state of the interrupt. + */ + __le32 dynctl_reg_start; + __le32 dynctl_reg_spacing; + + __le32 itrn_reg_start; + __le32 itrn_reg_spacing; + u8 reserved[8]; +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_vector_chunk); + +/* Structure to specify several chunks of contiguous interrupt vectors */ +struct virtchnl2_vector_chunks { + __le16 num_vchunks; + u8 reserved[14]; + struct virtchnl2_vector_chunk vchunks[1]; +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(48, virtchnl2_vector_chunks); + +/* VIRTCHNL2_OP_ALLOC_VECTORS + * PF sends this message to request additional interrupt vectors beyond the + * ones that were assigned via GET_CAPS request. virtchnl2_alloc_vectors + * structure is used to specify the number of vectors requested. CP responds + * with the same structure with the actual number of vectors assigned followed + * by virtchnl2_vector_chunks structure identifying the vector ids. + */ +struct virtchnl2_alloc_vectors { + __le16 num_vectors; + u8 reserved[14]; + struct virtchnl2_vector_chunks vchunks; +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(64, virtchnl2_alloc_vectors); + +/* VIRTCHNL2_OP_DEALLOC_VECTORS + * PF sends this message to release the vectors. + * PF sends virtchnl2_vector_chunks struct to specify the vectors it is giving + * away. CP performs requested action and returns status. + */ + +/* VIRTCHNL2_OP_GET_RSS_LUT + * VIRTCHNL2_OP_SET_RSS_LUT + * PF sends this message to get or set RSS lookup table. Only supported if + * both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration + * negotiation. Uses the virtchnl2_rss_lut structure + */ +struct virtchnl2_rss_lut { + __le32 vport_id; + __le16 lut_entries_start; + __le16 lut_entries; + u8 reserved[4]; + __le32 lut[1]; /* RSS lookup table */ +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_lut); + +/* VIRTCHNL2_OP_GET_RSS_KEY + * PF sends this message to get RSS key. Only supported if both PF and CP + * drivers set the VIRTCHNL2_CAP_RSS bit during configuration negotiation. Uses + * the virtchnl2_rss_key structure + */ + +/* VIRTCHNL2_OP_GET_RSS_HASH + * VIRTCHNL2_OP_SET_RSS_HASH + * PF sends these messages to get and set the hash filter enable bits for RSS. + * By default, the CP sets these to all possible traffic types that the + * hardware supports. The PF can query this value if it wants to change the + * traffic types that are hashed by the hardware. + * Only supported if both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit + * during configuration negotiation. + */ +struct virtchnl2_rss_hash { + /* Packet Type Groups bitmap */ + __le64 ptype_groups; + __le32 vport_id; + u8 reserved[4]; +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_hash); + +/* VIRTCHNL2_OP_SET_SRIOV_VFS + * This message is used to set number of SRIOV VFs to be created. The actual + * allocation of resources for the VFs in terms of vport, queues and interrupts + * is done by CP. When this call completes, the APF driver calls + * pci_enable_sriov to let the OS instantiate the SRIOV PCIE devices. + * The number of VFs set to 0 will destroy all the VFs of this function. + */ + +struct virtchnl2_sriov_vfs_info { + __le16 num_vfs; + __le16 pad; +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_sriov_vfs_info); + +/* VIRTCHNL2_OP_CREATE_ADI + * PF sends this message to CP to create ADI by filling in required + * fields of virtchnl2_create_adi structure. + * CP responds with the updated virtchnl2_create_adi structure containing the + * necessary fields followed by chunks which in turn will have an array of + * num_chunks entries of virtchnl2_queue_chunk structures. + */ +struct virtchnl2_create_adi { + /* PF sends PASID to CP */ + __le32 pasid; + /* + * mbx_id is set to 1 by PF when requesting CP to provide HW mailbox + * id else it is set to 0 by PF + */ + __le16 mbx_id; + /* PF sends mailbox vector id to CP */ + __le16 mbx_vec_id; + /* CP populates ADI id */ + __le16 adi_id; + u8 reserved[64]; + u8 pad[6]; + /* CP populates queue chunks */ + struct virtchnl2_queue_reg_chunks chunks; + /* PF sends vector chunks to CP */ + struct virtchnl2_vector_chunks vchunks; +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(168, virtchnl2_create_adi); + +/* VIRTCHNL2_OP_DESTROY_ADI + * PF sends this message to CP to destroy ADI by filling + * in the adi_id in virtchnl2_destropy_adi structure. + * CP responds with the status of the requested operation. + */ +struct virtchnl2_destroy_adi { + __le16 adi_id; + u8 reserved[2]; +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_destroy_adi); + +/* Based on the descriptor type the PF supports, CP fills ptype_id_10 or + * ptype_id_8 for flex and base descriptor respectively. If ptype_id_10 value + * is set to 0xFFFF, PF should consider this ptype as dummy one and it is the + * last ptype. + */ +struct virtchnl2_ptype { + __le16 ptype_id_10; + u8 ptype_id_8; + /* number of protocol ids the packet supports, maximum of 32 + * protocol ids are supported + */ + u8 proto_id_count; + __le16 pad; + /* proto_id_count decides the allocation of protocol id array */ + /* see VIRTCHNL2_PROTO_HDR_TYPE */ + __le16 proto_id[1]; +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_ptype); + +/* VIRTCHNL2_OP_GET_PTYPE_INFO + * PF sends this message to CP to get all supported packet types. It does by + * filling in start_ptype_id and num_ptypes. Depending on descriptor type the + * PF supports, it sets num_ptypes to 1024 (10-bit ptype) for flex descriptor + * and 256 (8-bit ptype) for base descriptor support. CP responds back to PF by + * populating start_ptype_id, num_ptypes and array of ptypes. If all ptypes + * doesn't fit into one mailbox buffer, CP splits ptype info into multiple + * messages, where each message will have the start ptype id, number of ptypes + * sent in that message and the ptype array itself. When CP is done updating + * all ptype information it extracted from the package (number of ptypes + * extracted might be less than what PF expects), it will append a dummy ptype + * (which has 'ptype_id_10' of 'struct virtchnl2_ptype' as 0xFFFF) to the ptype + * array. PF is expected to receive multiple VIRTCHNL2_OP_GET_PTYPE_INFO + * messages. + */ +struct virtchnl2_get_ptype_info { + __le16 start_ptype_id; + __le16 num_ptypes; + __le32 pad; + struct virtchnl2_ptype ptype[1]; +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_get_ptype_info); + +/* VIRTCHNL2_OP_GET_STATS + * PF/VF sends this message to CP to get the update stats by specifying the + * vport_id. CP responds with stats in struct virtchnl2_vport_stats. + */ +struct virtchnl2_vport_stats { + __le32 vport_id; + u8 pad[4]; + + __le64 rx_bytes; /* received bytes */ + __le64 rx_unicast; /* received unicast pkts */ + __le64 rx_multicast; /* received multicast pkts */ + __le64 rx_broadcast; /* received broadcast pkts */ + __le64 rx_discards; + __le64 rx_errors; + __le64 rx_unknown_protocol; + __le64 tx_bytes; /* transmitted bytes */ + __le64 tx_unicast; /* transmitted unicast pkts */ + __le64 tx_multicast; /* transmitted multicast pkts */ + __le64 tx_broadcast; /* transmitted broadcast pkts */ + __le64 tx_discards; + __le64 tx_errors; + __le64 rx_invalid_frame_length; + __le64 rx_overflow_drop; +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(128, virtchnl2_vport_stats); + +/* VIRTCHNL2_OP_EVENT + * CP sends this message to inform the PF/VF driver of events that may affect + * it. No direct response is expected from the driver, though it may generate + * other messages in response to this one. + */ +struct virtchnl2_event { + /* see VIRTCHNL2_EVENT_CODES definitions */ + __le32 event; + /* link_speed provided in Mbps */ + __le32 link_speed; + __le32 vport_id; + u8 link_status; + u8 pad[1]; + /* CP sends reset notification to PF with corresponding ADI ID */ + __le16 adi_id; +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_event); + +/* VIRTCHNL2_OP_GET_RSS_KEY + * VIRTCHNL2_OP_SET_RSS_KEY + * PF/VF sends this message to get or set RSS key. Only supported if both + * PF/VF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration + * negotiation. Uses the virtchnl2_rss_key structure + */ +struct virtchnl2_rss_key { + __le32 vport_id; + __le16 key_len; + u8 pad; + u8 key[1]; /* RSS hash key, packed bytes */ +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_rss_key); + +/* structure to specify a chunk of contiguous queues */ +struct virtchnl2_queue_chunk { + /* see VIRTCHNL2_QUEUE_TYPE definitions */ + __le32 type; + __le32 start_queue_id; + __le32 num_queues; + u8 reserved[4]; +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_queue_chunk); + +/* structure to specify several chunks of contiguous queues */ +struct virtchnl2_queue_chunks { + __le16 num_chunks; + u8 reserved[6]; + struct virtchnl2_queue_chunk chunks[1]; +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_chunks); + +/* VIRTCHNL2_OP_ENABLE_QUEUES + * VIRTCHNL2_OP_DISABLE_QUEUES + * VIRTCHNL2_OP_DEL_QUEUES + * + * PF sends these messages to enable, disable or delete queues specified in + * chunks. PF sends virtchnl2_del_ena_dis_queues struct to specify the queues + * to be enabled/disabled/deleted. Also applicable to single queue receive or + * transmit. CP performs requested action and returns status. + */ +struct virtchnl2_del_ena_dis_queues { + __le32 vport_id; + u8 reserved[4]; + struct virtchnl2_queue_chunks chunks; +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_del_ena_dis_queues); + +/* Queue to vector mapping */ +struct virtchnl2_queue_vector { + __le32 queue_id; + __le16 vector_id; + u8 pad[2]; + + /* see VIRTCHNL2_ITR_IDX definitions */ + __le32 itr_idx; + + /* see VIRTCHNL2_QUEUE_TYPE definitions */ + __le32 queue_type; + u8 reserved[8]; +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_vector); + +/* VIRTCHNL2_OP_MAP_QUEUE_VECTOR + * VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR + * + * PF sends this message to map or unmap queues to vectors and interrupt + * throttling rate index registers. External data buffer contains + * virtchnl2_queue_vector_maps structure that contains num_qv_maps of + * virtchnl2_queue_vector structures. CP maps the requested queue vector maps + * after validating the queue and vector ids and returns a status code. + */ +struct virtchnl2_queue_vector_maps { + __le32 vport_id; + __le16 num_qv_maps; + u8 pad[10]; + struct virtchnl2_queue_vector qv_maps[1]; +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_vector_maps); + +/* VIRTCHNL2_OP_LOOPBACK + * + * PF/VF sends this message to transition to/from the loopback state. Setting + * the 'enable' to 1 enables the loopback state and setting 'enable' to 0 + * disables it. CP configures the state to loopback and returns status. + */ +struct virtchnl2_loopback { + __le32 vport_id; + u8 enable; + u8 pad[3]; +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_loopback); + +/* structure to specify each MAC address */ +struct virtchnl2_mac_addr { + u8 addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS]; + /* see VIRTCHNL2_MAC_TYPE definitions */ + u8 type; + u8 pad; +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_mac_addr); + +/* VIRTCHNL2_OP_ADD_MAC_ADDR + * VIRTCHNL2_OP_DEL_MAC_ADDR + * + * PF/VF driver uses this structure to send list of MAC addresses to be + * added/deleted to the CP where as CP performs the action and returns the + * status. + */ +struct virtchnl2_mac_addr_list { + __le32 vport_id; + __le16 num_mac_addr; + u8 pad[2]; + struct virtchnl2_mac_addr mac_addr_list[1]; +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_mac_addr_list); + +/* VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE + * + * PF/VF sends vport id and flags to the CP where as CP performs the action + * and returns the status. + */ +struct virtchnl2_promisc_info { + __le32 vport_id; + /* see VIRTCHNL2_PROMISC_FLAGS definitions */ + __le16 flags; + u8 pad[2]; +}; + +VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_promisc_info); + + +static inline const char *virtchnl2_op_str(__le32 v_opcode) +{ + switch (v_opcode) { + case VIRTCHNL2_OP_VERSION: + return "VIRTCHNL2_OP_VERSION"; + case VIRTCHNL2_OP_GET_CAPS: + return "VIRTCHNL2_OP_GET_CAPS"; + case VIRTCHNL2_OP_CREATE_VPORT: + return "VIRTCHNL2_OP_CREATE_VPORT"; + case VIRTCHNL2_OP_DESTROY_VPORT: + return "VIRTCHNL2_OP_DESTROY_VPORT"; + case VIRTCHNL2_OP_ENABLE_VPORT: + return "VIRTCHNL2_OP_ENABLE_VPORT"; + case VIRTCHNL2_OP_DISABLE_VPORT: + return "VIRTCHNL2_OP_DISABLE_VPORT"; + case VIRTCHNL2_OP_CONFIG_TX_QUEUES: + return "VIRTCHNL2_OP_CONFIG_TX_QUEUES"; + case VIRTCHNL2_OP_CONFIG_RX_QUEUES: + return "VIRTCHNL2_OP_CONFIG_RX_QUEUES"; + case VIRTCHNL2_OP_ENABLE_QUEUES: + return "VIRTCHNL2_OP_ENABLE_QUEUES"; + case VIRTCHNL2_OP_DISABLE_QUEUES: + return "VIRTCHNL2_OP_DISABLE_QUEUES"; + case VIRTCHNL2_OP_ADD_QUEUES: + return "VIRTCHNL2_OP_ADD_QUEUES"; + case VIRTCHNL2_OP_DEL_QUEUES: + return "VIRTCHNL2_OP_DEL_QUEUES"; + case VIRTCHNL2_OP_MAP_QUEUE_VECTOR: + return "VIRTCHNL2_OP_MAP_QUEUE_VECTOR"; + case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR: + return "VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR"; + case VIRTCHNL2_OP_GET_RSS_KEY: + return "VIRTCHNL2_OP_GET_RSS_KEY"; + case VIRTCHNL2_OP_SET_RSS_KEY: + return "VIRTCHNL2_OP_SET_RSS_KEY"; + case VIRTCHNL2_OP_GET_RSS_LUT: + return "VIRTCHNL2_OP_GET_RSS_LUT"; + case VIRTCHNL2_OP_SET_RSS_LUT: + return "VIRTCHNL2_OP_SET_RSS_LUT"; + case VIRTCHNL2_OP_GET_RSS_HASH: + return "VIRTCHNL2_OP_GET_RSS_HASH"; + case VIRTCHNL2_OP_SET_RSS_HASH: + return "VIRTCHNL2_OP_SET_RSS_HASH"; + case VIRTCHNL2_OP_SET_SRIOV_VFS: + return "VIRTCHNL2_OP_SET_SRIOV_VFS"; + case VIRTCHNL2_OP_ALLOC_VECTORS: + return "VIRTCHNL2_OP_ALLOC_VECTORS"; + case VIRTCHNL2_OP_DEALLOC_VECTORS: + return "VIRTCHNL2_OP_DEALLOC_VECTORS"; + case VIRTCHNL2_OP_GET_PTYPE_INFO: + return "VIRTCHNL2_OP_GET_PTYPE_INFO"; + case VIRTCHNL2_OP_GET_STATS: + return "VIRTCHNL2_OP_GET_STATS"; + case VIRTCHNL2_OP_EVENT: + return "VIRTCHNL2_OP_EVENT"; + case VIRTCHNL2_OP_RESET_VF: + return "VIRTCHNL2_OP_RESET_VF"; + case VIRTCHNL2_OP_CREATE_ADI: + return "VIRTCHNL2_OP_CREATE_ADI"; + case VIRTCHNL2_OP_DESTROY_ADI: + return "VIRTCHNL2_OP_DESTROY_ADI"; + default: + return "Unsupported (update virtchnl2.h)"; + } +} + +/** + * virtchnl2_vc_validate_vf_msg + * @ver: Virtchnl2 version info + * @v_opcode: Opcode for the message + * @msg: pointer to the msg buffer + * @msglen: msg length + * + * validate msg format against struct for each opcode + */ +static inline int +virtchnl2_vc_validate_vf_msg(__rte_unused struct virtchnl2_version_info *ver, u32 v_opcode, + u8 *msg, __le16 msglen) +{ + bool err_msg_format = false; + __le32 valid_len = 0; + + /* Validate message length. */ + switch (v_opcode) { + case VIRTCHNL2_OP_VERSION: + valid_len = sizeof(struct virtchnl2_version_info); + break; + case VIRTCHNL2_OP_GET_CAPS: + valid_len = sizeof(struct virtchnl2_get_capabilities); + break; + case VIRTCHNL2_OP_CREATE_VPORT: + valid_len = sizeof(struct virtchnl2_create_vport); + if (msglen >= valid_len) { + struct virtchnl2_create_vport *cvport = + (struct virtchnl2_create_vport *)msg; + + if (cvport->chunks.num_chunks == 0) { + /* zero chunks is allowed as input */ + break; + } + + valid_len += (cvport->chunks.num_chunks - 1) * + sizeof(struct virtchnl2_queue_reg_chunk); + } + break; + case VIRTCHNL2_OP_CREATE_ADI: + valid_len = sizeof(struct virtchnl2_create_adi); + if (msglen >= valid_len) { + struct virtchnl2_create_adi *cadi = + (struct virtchnl2_create_adi *)msg; + + if (cadi->chunks.num_chunks == 0) { + /* zero chunks is allowed as input */ + break; + } + + if (cadi->vchunks.num_vchunks == 0) { + err_msg_format = true; + break; + } + valid_len += (cadi->chunks.num_chunks - 1) * + sizeof(struct virtchnl2_queue_reg_chunk); + valid_len += (cadi->vchunks.num_vchunks - 1) * + sizeof(struct virtchnl2_vector_chunk); + } + break; + case VIRTCHNL2_OP_DESTROY_ADI: + valid_len = sizeof(struct virtchnl2_destroy_adi); + break; + case VIRTCHNL2_OP_DESTROY_VPORT: + case VIRTCHNL2_OP_ENABLE_VPORT: + case VIRTCHNL2_OP_DISABLE_VPORT: + valid_len = sizeof(struct virtchnl2_vport); + break; + case VIRTCHNL2_OP_CONFIG_TX_QUEUES: + valid_len = sizeof(struct virtchnl2_config_tx_queues); + if (msglen >= valid_len) { + struct virtchnl2_config_tx_queues *ctq = + (struct virtchnl2_config_tx_queues *)msg; + if (ctq->num_qinfo == 0) { + err_msg_format = true; + break; + } + valid_len += (ctq->num_qinfo - 1) * + sizeof(struct virtchnl2_txq_info); + } + break; + case VIRTCHNL2_OP_CONFIG_RX_QUEUES: + valid_len = sizeof(struct virtchnl2_config_rx_queues); + if (msglen >= valid_len) { + struct virtchnl2_config_rx_queues *crq = + (struct virtchnl2_config_rx_queues *)msg; + if (crq->num_qinfo == 0) { + err_msg_format = true; + break; + } + valid_len += (crq->num_qinfo - 1) * + sizeof(struct virtchnl2_rxq_info); + } + break; + case VIRTCHNL2_OP_ADD_QUEUES: + valid_len = sizeof(struct virtchnl2_add_queues); + if (msglen >= valid_len) { + struct virtchnl2_add_queues *add_q = + (struct virtchnl2_add_queues *)msg; + + if (add_q->chunks.num_chunks == 0) { + /* zero chunks is allowed as input */ + break; + } + + valid_len += (add_q->chunks.num_chunks - 1) * + sizeof(struct virtchnl2_queue_reg_chunk); + } + break; + case VIRTCHNL2_OP_ENABLE_QUEUES: + case VIRTCHNL2_OP_DISABLE_QUEUES: + case VIRTCHNL2_OP_DEL_QUEUES: + valid_len = sizeof(struct virtchnl2_del_ena_dis_queues); + if (msglen >= valid_len) { + struct virtchnl2_del_ena_dis_queues *qs = + (struct virtchnl2_del_ena_dis_queues *)msg; + if (qs->chunks.num_chunks == 0 || + qs->chunks.num_chunks > VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX) { + err_msg_format = true; + break; + } + valid_len += (qs->chunks.num_chunks - 1) * + sizeof(struct virtchnl2_queue_chunk); + } + break; + case VIRTCHNL2_OP_MAP_QUEUE_VECTOR: + case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR: + valid_len = sizeof(struct virtchnl2_queue_vector_maps); + if (msglen >= valid_len) { + struct virtchnl2_queue_vector_maps *v_qp = + (struct virtchnl2_queue_vector_maps *)msg; + if (v_qp->num_qv_maps == 0 || + v_qp->num_qv_maps > VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) { + err_msg_format = true; + break; + } + valid_len += (v_qp->num_qv_maps - 1) * + sizeof(struct virtchnl2_queue_vector); + } + break; + case VIRTCHNL2_OP_ALLOC_VECTORS: + valid_len = sizeof(struct virtchnl2_alloc_vectors); + if (msglen >= valid_len) { + struct virtchnl2_alloc_vectors *v_av = + (struct virtchnl2_alloc_vectors *)msg; + + if (v_av->vchunks.num_vchunks == 0) { + /* zero chunks is allowed as input */ + break; + } + + valid_len += (v_av->vchunks.num_vchunks - 1) * + sizeof(struct virtchnl2_vector_chunk); + } + break; + case VIRTCHNL2_OP_DEALLOC_VECTORS: + valid_len = sizeof(struct virtchnl2_vector_chunks); + if (msglen >= valid_len) { + struct virtchnl2_vector_chunks *v_chunks = + (struct virtchnl2_vector_chunks *)msg; + if (v_chunks->num_vchunks == 0) { + err_msg_format = true; + break; + } + valid_len += (v_chunks->num_vchunks - 1) * + sizeof(struct virtchnl2_vector_chunk); + } + break; + case VIRTCHNL2_OP_GET_RSS_KEY: + case VIRTCHNL2_OP_SET_RSS_KEY: + valid_len = sizeof(struct virtchnl2_rss_key); + if (msglen >= valid_len) { + struct virtchnl2_rss_key *vrk = + (struct virtchnl2_rss_key *)msg; + + if (vrk->key_len == 0) { + /* zero length is allowed as input */ + break; + } + + valid_len += vrk->key_len - 1; + } + break; + case VIRTCHNL2_OP_GET_RSS_LUT: + case VIRTCHNL2_OP_SET_RSS_LUT: + valid_len = sizeof(struct virtchnl2_rss_lut); + if (msglen >= valid_len) { + struct virtchnl2_rss_lut *vrl = + (struct virtchnl2_rss_lut *)msg; + + if (vrl->lut_entries == 0) { + /* zero entries is allowed as input */ + break; + } + + valid_len += (vrl->lut_entries - 1) * sizeof(vrl->lut); + } + break; + case VIRTCHNL2_OP_GET_RSS_HASH: + case VIRTCHNL2_OP_SET_RSS_HASH: + valid_len = sizeof(struct virtchnl2_rss_hash); + break; + case VIRTCHNL2_OP_SET_SRIOV_VFS: + valid_len = sizeof(struct virtchnl2_sriov_vfs_info); + break; + case VIRTCHNL2_OP_GET_PTYPE_INFO: + valid_len = sizeof(struct virtchnl2_get_ptype_info); + break; + case VIRTCHNL2_OP_GET_STATS: + valid_len = sizeof(struct virtchnl2_vport_stats); + break; + case VIRTCHNL2_OP_RESET_VF: + break; + /* These are always errors coming from the VF. */ + case VIRTCHNL2_OP_EVENT: + case VIRTCHNL2_OP_UNKNOWN: + default: + return VIRTCHNL2_STATUS_ERR_PARAM; + } + /* few more checks */ + if (err_msg_format || valid_len != msglen) + return VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH; + + return 0; +} + +#endif /* _VIRTCHNL_2_H_ */ diff --git a/drivers/common/idpf/base/virtchnl2_lan_desc.h b/drivers/common/idpf/base/virtchnl2_lan_desc.h new file mode 100644 index 0000000000..b8cb22e474 --- /dev/null +++ b/drivers/common/idpf/base/virtchnl2_lan_desc.h @@ -0,0 +1,606 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2001-2022 Intel Corporation + */ +/* + * Copyright (C) 2019 Intel Corporation + * + * For licensing information, see the file 'LICENSE' in the root folder + */ +#ifndef _VIRTCHNL2_LAN_DESC_H_ +#define _VIRTCHNL2_LAN_DESC_H_ + +/* VIRTCHNL2_TX_DESC_IDS + * Transmit descriptor ID flags + */ +#define VIRTCHNL2_TXDID_DATA BIT(0) +#define VIRTCHNL2_TXDID_CTX BIT(1) +#define VIRTCHNL2_TXDID_REINJECT_CTX BIT(2) +#define VIRTCHNL2_TXDID_FLEX_DATA BIT(3) +#define VIRTCHNL2_TXDID_FLEX_CTX BIT(4) +#define VIRTCHNL2_TXDID_FLEX_TSO_CTX BIT(5) +#define VIRTCHNL2_TXDID_FLEX_TSYN_L2TAG1 BIT(6) +#define VIRTCHNL2_TXDID_FLEX_L2TAG1_L2TAG2 BIT(7) +#define VIRTCHNL2_TXDID_FLEX_TSO_L2TAG2_PARSTAG_CTX BIT(8) +#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_TSO_CTX BIT(9) +#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_CTX BIT(10) +#define VIRTCHNL2_TXDID_FLEX_L2TAG2_CTX BIT(11) +#define VIRTCHNL2_TXDID_FLEX_FLOW_SCHED BIT(12) +#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_TSO_CTX BIT(13) +#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_CTX BIT(14) +#define VIRTCHNL2_TXDID_DESC_DONE BIT(15) + +/* VIRTCHNL2_RX_DESC_IDS + * Receive descriptor IDs (range from 0 to 63) + */ +#define VIRTCHNL2_RXDID_0_16B_BASE 0 +#define VIRTCHNL2_RXDID_1_32B_BASE 1 +/* FLEX_SQ_NIC and FLEX_SPLITQ share desc ids because they can be + * differentiated based on queue model; e.g. single queue model can + * only use FLEX_SQ_NIC and split queue model can only use FLEX_SPLITQ + * for DID 2. + */ +#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ 2 +#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC 2 +#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW 3 +#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB 4 +#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL 5 +#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2 6 +#define VIRTCHNL2_RXDID_7_HW_RSVD 7 +/* 9 through 15 are reserved */ +#define VIRTCHNL2_RXDID_16_COMMS_GENERIC 16 +#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN 17 +#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4 18 +#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6 19 +#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW 20 +#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP 21 +/* 22 through 63 are reserved */ + +/* VIRTCHNL2_RX_DESC_ID_BITMASKS + * Receive descriptor ID bitmasks + */ +#define VIRTCHNL2_RXDID_0_16B_BASE_M BIT(VIRTCHNL2_RXDID_0_16B_BASE) +#define VIRTCHNL2_RXDID_1_32B_BASE_M BIT(VIRTCHNL2_RXDID_1_32B_BASE) +#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M BIT(VIRTCHNL2_RXDID_2_FLEX_SPLITQ) +#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M BIT(VIRTCHNL2_RXDID_2_FLEX_SQ_NIC) +#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW_M BIT(VIRTCHNL2_RXDID_3_FLEX_SQ_SW) +#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB_M BIT(VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB) +#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL_M BIT(VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL) +#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2_M BIT(VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2) +#define VIRTCHNL2_RXDID_7_HW_RSVD_M BIT(VIRTCHNL2_RXDID_7_HW_RSVD) +/* 9 through 15 are reserved */ +#define VIRTCHNL2_RXDID_16_COMMS_GENERIC_M BIT(VIRTCHNL2_RXDID_16_COMMS_GENERIC) +#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN_M BIT(VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN) +#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4_M BIT(VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4) +#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6_M BIT(VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6) +#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW_M BIT(VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW) +#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP_M BIT(VIRTCHNL2_RXDID_21_COMMS_AUX_TCP) +/* 22 through 63 are reserved */ + +/* Rx */ +/* For splitq virtchnl2_rx_flex_desc_adv desc members */ +#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S 0 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_M \ + MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S) +#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S 0 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M \ + MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S) +#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S 10 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_M \ + MAKEMASK(0x3UL, VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S) +#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S 12 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_M \ + MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S) +#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S 0 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M \ + MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S) +#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S 14 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M \ + BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S) +#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S 15 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M \ + BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S) +#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S 0 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_M \ + MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S) +#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S 10 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_M \ + BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S) +#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S 11 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_M \ + BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S) +#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_S 12 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M \ + MAKEMASK(0x7UL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M) +#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S 15 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_M \ + BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S) + +/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW1_BITS + * for splitq virtchnl2_rx_flex_desc_adv + * Note: These are predefined bit offsets + */ +#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_DD_S 0 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_EOF_S 1 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_HBO_S 2 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S 3 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S 4 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S 5 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S 6 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S 7 + +/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW0_BITS + * for splitq virtchnl2_rx_flex_desc_adv + * Note: These are predefined bit offsets + */ +#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LPBK_S 0 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_IPV6EXADD_S 1 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RXE_S 2 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_CRCP_S 3 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S 4 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L2TAG1P_S 5 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD0_VALID_S 6 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD1_VALID_S 7 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LAST 8 /* this entry must be last!!! */ + +/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_1_BITS + * for splitq virtchnl2_rx_flex_desc_adv + * Note: These are predefined bit offsets + */ +#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_RSVD_S 0 /* 2 bits */ +#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_ATRAEFAIL_S 2 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_L2TAG2P_S 3 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD2_VALID_S 4 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD3_VALID_S 5 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD4_VALID_S 6 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD5_VALID_S 7 +#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_LAST 8 /* this entry must be last!!! */ + +/* for singleq (flex) virtchnl2_rx_flex_desc fields */ +/* for virtchnl2_rx_flex_desc.ptype_flex_flags0 member */ +#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_S 0 +#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_M \ + MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_PTYPE_S) /* 10 bits */ + +/* for virtchnl2_rx_flex_desc.pkt_length member */ +#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S 0 +#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M \ + MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S) /* 14 bits */ + +/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_0_BITS + * for singleq (flex) virtchnl2_rx_flex_desc + * Note: These are predefined bit offsets + */ +#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S 0 +#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_EOF_S 1 +#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_HBO_S 2 +#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S 3 +#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S 4 +#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S 5 +#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S 6 +#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S 7 +#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LPBK_S 8 +#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_IPV6EXADD_S 9 +#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RXE_S 10 +#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_CRCP_S 11 +#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RSS_VALID_S 12 +#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L2TAG1P_S 13 +#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD0_VALID_S 14 +#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD1_VALID_S 15 +#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LAST 16 /* this entry must be last!!! */ + +/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_1_BITS + * for singleq (flex) virtchnl2_rx_flex_desc + * Note: These are predefined bit offsets + */ +#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CPM_S 0 /* 4 bits */ +#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_NAT_S 4 +#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CRYPTO_S 5 +/* [10:6] reserved */ +#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_L2TAG2P_S 11 +#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD2_VALID_S 12 +#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD3_VALID_S 13 +#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S 14 +#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S 15 +#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_LAST 16 /* this entry must be last!!! */ + +/* for virtchnl2_rx_flex_desc.ts_low member */ +#define VIRTCHNL2_RX_FLEX_TSTAMP_VALID BIT(0) + +/* For singleq (non flex) virtchnl2_singleq_base_rx_desc legacy desc members */ +#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S 63 +#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_M \ + BIT_ULL(VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S) +#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S 52 +#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_M \ + MAKEMASK(0x7FFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S) +#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S 38 +#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_M \ + MAKEMASK(0x3FFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S) +#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S 30 +#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_M \ + MAKEMASK(0xFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S) +#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S 19 +#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_M \ + MAKEMASK(0xFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S) +#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S 0 +#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_M \ + MAKEMASK(0x7FFFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S) + +/* VIRTCHNL2_RX_BASE_DESC_STATUS_BITS + * for singleq (base) virtchnl2_rx_base_desc + * Note: These are predefined bit offsets + */ +#define VIRTCHNL2_RX_BASE_DESC_STATUS_DD_S 0 +#define VIRTCHNL2_RX_BASE_DESC_STATUS_EOF_S 1 +#define VIRTCHNL2_RX_BASE_DESC_STATUS_L2TAG1P_S 2 +#define VIRTCHNL2_RX_BASE_DESC_STATUS_L3L4P_S 3 +#define VIRTCHNL2_RX_BASE_DESC_STATUS_CRCP_S 4 +#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD_S 5 /* 3 bits */ +#define VIRTCHNL2_RX_BASE_DESC_STATUS_EXT_UDP_0_S 8 +#define VIRTCHNL2_RX_BASE_DESC_STATUS_UMBCAST_S 9 /* 2 bits */ +#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLM_S 11 +#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLTSTAT_S 12 /* 2 bits */ +#define VIRTCHNL2_RX_BASE_DESC_STATUS_LPBK_S 14 +#define VIRTCHNL2_RX_BASE_DESC_STATUS_IPV6EXADD_S 15 +#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD1_S 16 /* 2 bits */ +#define VIRTCHNL2_RX_BASE_DESC_STATUS_INT_UDP_0_S 18 +#define VIRTCHNL2_RX_BASE_DESC_STATUS_LAST 19 /* this entry must be last!!! */ + +/* VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_BITS + * for singleq (base) virtchnl2_rx_base_desc + * Note: These are predefined bit offsets + */ +#define VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_L2TAG2P_S 0 + +/* VIRTCHNL2_RX_BASE_DESC_ERROR_BITS + * for singleq (base) virtchnl2_rx_base_desc + * Note: These are predefined bit offsets + */ +#define VIRTCHNL2_RX_BASE_DESC_ERROR_RXE_S 0 +#define VIRTCHNL2_RX_BASE_DESC_ERROR_ATRAEFAIL_S 1 +#define VIRTCHNL2_RX_BASE_DESC_ERROR_HBO_S 2 +#define VIRTCHNL2_RX_BASE_DESC_ERROR_L3L4E_S 3 /* 3 bits */ +#define VIRTCHNL2_RX_BASE_DESC_ERROR_IPE_S 3 +#define VIRTCHNL2_RX_BASE_DESC_ERROR_L4E_S 4 +#define VIRTCHNL2_RX_BASE_DESC_ERROR_EIPE_S 5 +#define VIRTCHNL2_RX_BASE_DESC_ERROR_OVERSIZE_S 6 +#define VIRTCHNL2_RX_BASE_DESC_ERROR_PPRS_S 7 + +/* VIRTCHNL2_RX_BASE_DESC_FLTSTAT_VALUES + * for singleq (base) virtchnl2_rx_base_desc + * Note: These are predefined bit offsets + */ +#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_NO_DATA 0 +#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_FD_ID 1 +#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSV 2 +#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSS_HASH 3 + +/* Receive Descriptors */ +/* splitq buf + * | 16| 0| + * ---------------------------------------------------------------- + * | RSV | Buffer ID | + * ---------------------------------------------------------------- + * | Rx packet buffer address | + * ---------------------------------------------------------------- + * | Rx header buffer address | + * ---------------------------------------------------------------- + * | RSV | + * ---------------------------------------------------------------- + * | 0| + */ +struct virtchnl2_splitq_rx_buf_desc { + struct { + __le16 buf_id; /* Buffer Identifier */ + __le16 rsvd0; + __le32 rsvd1; + } qword0; + __le64 pkt_addr; /* Packet buffer address */ + __le64 hdr_addr; /* Header buffer address */ + __le64 rsvd2; +}; /* read used with buffer queues*/ + +/* singleq buf + * | 0| + * ---------------------------------------------------------------- + * | Rx packet buffer address | + * ---------------------------------------------------------------- + * | Rx header buffer address | + * ---------------------------------------------------------------- + * | RSV | + * ---------------------------------------------------------------- + * | RSV | + * ---------------------------------------------------------------- + * | 0| + */ +struct virtchnl2_singleq_rx_buf_desc { + __le64 pkt_addr; /* Packet buffer address */ + __le64 hdr_addr; /* Header buffer address */ + __le64 rsvd1; + __le64 rsvd2; +}; /* read used with buffer queues*/ + +union virtchnl2_rx_buf_desc { + struct virtchnl2_singleq_rx_buf_desc read; + struct virtchnl2_splitq_rx_buf_desc split_rd; +}; + +/* (0x00) singleq wb(compl) */ +struct virtchnl2_singleq_base_rx_desc { + struct { + struct { + __le16 mirroring_status; + __le16 l2tag1; + } lo_dword; + union { + __le32 rss; /* RSS Hash */ + __le32 fd_id; /* Flow Director filter id */ + } hi_dword; + } qword0; + struct { + /* status/error/PTYPE/length */ + __le64 status_error_ptype_len; + } qword1; + struct { + __le16 ext_status; /* extended status */ + __le16 rsvd; + __le16 l2tag2_1; + __le16 l2tag2_2; + } qword2; + struct { + __le32 reserved; + __le32 fd_id; + } qword3; +}; /* writeback */ + +/* (0x01) singleq flex compl */ +struct virtchnl2_rx_flex_desc { + /* Qword 0 */ + u8 rxdid; /* descriptor builder profile id */ + u8 mir_id_umb_cast; /* mirror=[5:0], umb=[7:6] */ + __le16 ptype_flex_flags0; /* ptype=[9:0], ff0=[15:10] */ + __le16 pkt_len; /* [15:14] are reserved */ + __le16 hdr_len_sph_flex_flags1; /* header=[10:0] */ + /* sph=[11:11] */ + /* ff1/ext=[15:12] */ + + /* Qword 1 */ + __le16 status_error0; + __le16 l2tag1; + __le16 flex_meta0; + __le16 flex_meta1; + + /* Qword 2 */ + __le16 status_error1; + u8 flex_flags2; + u8 time_stamp_low; + __le16 l2tag2_1st; + __le16 l2tag2_2nd; + + /* Qword 3 */ + __le16 flex_meta2; + __le16 flex_meta3; + union { + struct { + __le16 flex_meta4; + __le16 flex_meta5; + } flex; + __le32 ts_high; + } flex_ts; +}; + +/* (0x02) */ +struct virtchnl2_rx_flex_desc_nic { + /* Qword 0 */ + u8 rxdid; + u8 mir_id_umb_cast; + __le16 ptype_flex_flags0; + __le16 pkt_len; + __le16 hdr_len_sph_flex_flags1; + + /* Qword 1 */ + __le16 status_error0; + __le16 l2tag1; + __le32 rss_hash; + + /* Qword 2 */ + __le16 status_error1; + u8 flexi_flags2; + u8 ts_low; + __le16 l2tag2_1st; + __le16 l2tag2_2nd; + + /* Qword 3 */ + __le32 flow_id; + union { + struct { + __le16 rsvd; + __le16 flow_id_ipv6; + } flex; + __le32 ts_high; + } flex_ts; +}; + +/* Rx Flex Descriptor Switch Profile + * RxDID Profile Id 3 + * Flex-field 0: Source Vsi + */ +struct virtchnl2_rx_flex_desc_sw { + /* Qword 0 */ + u8 rxdid; + u8 mir_id_umb_cast; + __le16 ptype_flex_flags0; + __le16 pkt_len; + __le16 hdr_len_sph_flex_flags1; + + /* Qword 1 */ + __le16 status_error0; + __le16 l2tag1; + __le16 src_vsi; /* [10:15] are reserved */ + __le16 flex_md1_rsvd; + + /* Qword 2 */ + __le16 status_error1; + u8 flex_flags2; + u8 ts_low; + __le16 l2tag2_1st; + __le16 l2tag2_2nd; + + /* Qword 3 */ + __le32 rsvd; /* flex words 2-3 are reserved */ + __le32 ts_high; +}; + + +/* Rx Flex Descriptor NIC Profile + * RxDID Profile Id 6 + * Flex-field 0: RSS hash lower 16-bits + * Flex-field 1: RSS hash upper 16-bits + * Flex-field 2: Flow Id lower 16-bits + * Flex-field 3: Source Vsi + * Flex-field 4: reserved, Vlan id taken from L2Tag + */ +struct virtchnl2_rx_flex_desc_nic_2 { + /* Qword 0 */ + u8 rxdid; + u8 mir_id_umb_cast; + __le16 ptype_flex_flags0; + __le16 pkt_len; + __le16 hdr_len_sph_flex_flags1; + + /* Qword 1 */ + __le16 status_error0; + __le16 l2tag1; + __le32 rss_hash; + + /* Qword 2 */ + __le16 status_error1; + u8 flexi_flags2; + u8 ts_low; + __le16 l2tag2_1st; + __le16 l2tag2_2nd; + + /* Qword 3 */ + __le16 flow_id; + __le16 src_vsi; + union { + struct { + __le16 rsvd; + __le16 flow_id_ipv6; + } flex; + __le32 ts_high; + } flex_ts; +}; + +/* Rx Flex Descriptor Advanced (Split Queue Model) + * RxDID Profile Id 7 + */ +struct virtchnl2_rx_flex_desc_adv { + /* Qword 0 */ + u8 rxdid_ucast; /* profile_id=[3:0] */ + /* rsvd=[5:4] */ + /* ucast=[7:6] */ + u8 status_err0_qw0; + __le16 ptype_err_fflags0; /* ptype=[9:0] */ + /* ip_hdr_err=[10:10] */ + /* udp_len_err=[11:11] */ + /* ff0=[15:12] */ + __le16 pktlen_gen_bufq_id; /* plen=[13:0] */ + /* gen=[14:14] only in splitq */ + /* bufq_id=[15:15] only in splitq */ + __le16 hdrlen_flags; /* header=[9:0] */ + /* rsc=[10:10] only in splitq */ + /* sph=[11:11] only in splitq */ + /* ext_udp_0=[12:12] */ + /* int_udp_0=[13:13] */ + /* trunc_mirr=[14:14] */ + /* miss_prepend=[15:15] */ + /* Qword 1 */ + u8 status_err0_qw1; + u8 status_err1; + u8 fflags1; + u8 ts_low; + __le16 fmd0; + __le16 fmd1; + /* Qword 2 */ + __le16 fmd2; + u8 fflags2; + u8 hash3; + __le16 fmd3; + __le16 fmd4; + /* Qword 3 */ + __le16 fmd5; + __le16 fmd6; + __le16 fmd7_0; + __le16 fmd7_1; +}; /* writeback */ + +/* Rx Flex Descriptor Advanced (Split Queue Model) NIC Profile + * RxDID Profile Id 8 + * Flex-field 0: BufferID + * Flex-field 1: Raw checksum/L2TAG1/RSC Seg Len (determined by HW) + * Flex-field 2: Hash[15:0] + * Flex-flags 2: Hash[23:16] + * Flex-field 3: L2TAG2 + * Flex-field 5: L2TAG1 + * Flex-field 7: Timestamp (upper 32 bits) + */ +struct virtchnl2_rx_flex_desc_adv_nic_3 { + /* Qword 0 */ + u8 rxdid_ucast; /* profile_id=[3:0] */ + /* rsvd=[5:4] */ + /* ucast=[7:6] */ + u8 status_err0_qw0; + __le16 ptype_err_fflags0; /* ptype=[9:0] */ + /* ip_hdr_err=[10:10] */ + /* udp_len_err=[11:11] */ + /* ff0=[15:12] */ + __le16 pktlen_gen_bufq_id; /* plen=[13:0] */ + /* gen=[14:14] only in splitq */ + /* bufq_id=[15:15] only in splitq */ + __le16 hdrlen_flags; /* header=[9:0] */ + /* rsc=[10:10] only in splitq */ + /* sph=[11:11] only in splitq */ + /* ext_udp_0=[12:12] */ + /* int_udp_0=[13:13] */ + /* trunc_mirr=[14:14] */ + /* miss_prepend=[15:15] */ + /* Qword 1 */ + u8 status_err0_qw1; + u8 status_err1; + u8 fflags1; + u8 ts_low; + __le16 buf_id; /* only in splitq */ + union { + __le16 raw_cs; + __le16 l2tag1; + __le16 rscseglen; + } misc; + /* Qword 2 */ + __le16 hash1; + union { + u8 fflags2; + u8 mirrorid; + u8 hash2; + } ff2_mirrid_hash2; + u8 hash3; + __le16 l2tag2; + __le16 fmd4; + /* Qword 3 */ + __le16 l2tag1; + __le16 fmd6; + __le32 ts_high; +}; /* writeback */ + +union virtchnl2_rx_desc { + struct virtchnl2_singleq_rx_buf_desc read; + struct virtchnl2_singleq_base_rx_desc base_wb; + struct virtchnl2_rx_flex_desc flex_wb; + struct virtchnl2_rx_flex_desc_nic flex_nic_wb; + struct virtchnl2_rx_flex_desc_sw flex_sw_wb; + struct virtchnl2_rx_flex_desc_nic_2 flex_nic_2_wb; + struct virtchnl2_rx_flex_desc_adv flex_adv_wb; + struct virtchnl2_rx_flex_desc_adv_nic_3 flex_adv_nic_3_wb; +}; + +#endif /* _VIRTCHNL_LAN_DESC_H_ */ diff --git a/drivers/common/idpf/base/virtchnl_inline_ipsec.h b/drivers/common/idpf/base/virtchnl_inline_ipsec.h new file mode 100644 index 0000000000..e19043ac47 --- /dev/null +++ b/drivers/common/idpf/base/virtchnl_inline_ipsec.h @@ -0,0 +1,567 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2001-2022 Intel Corporation + */ + +#ifndef _VIRTCHNL_INLINE_IPSEC_H_ +#define _VIRTCHNL_INLINE_IPSEC_H_ + +#define VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM 3 +#define VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM 16 +#define VIRTCHNL_IPSEC_MAX_TX_DESC_NUM 128 +#define VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER 2 +#define VIRTCHNL_IPSEC_MAX_KEY_LEN 128 +#define VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM 8 +#define VIRTCHNL_IPSEC_SA_DESTROY 0 +#define VIRTCHNL_IPSEC_BROADCAST_VFID 0xFFFFFFFF +#define VIRTCHNL_IPSEC_INVALID_REQ_ID 0xFFFF +#define VIRTCHNL_IPSEC_INVALID_SA_CFG_RESP 0xFFFFFFFF +#define VIRTCHNL_IPSEC_INVALID_SP_CFG_RESP 0xFFFFFFFF + +/* crypto type */ +#define VIRTCHNL_AUTH 1 +#define VIRTCHNL_CIPHER 2 +#define VIRTCHNL_AEAD 3 + +/* caps enabled */ +#define VIRTCHNL_IPSEC_ESN_ENA BIT(0) +#define VIRTCHNL_IPSEC_UDP_ENCAP_ENA BIT(1) +#define VIRTCHNL_IPSEC_SA_INDEX_SW_ENA BIT(2) +#define VIRTCHNL_IPSEC_AUDIT_ENA BIT(3) +#define VIRTCHNL_IPSEC_BYTE_LIMIT_ENA BIT(4) +#define VIRTCHNL_IPSEC_DROP_ON_AUTH_FAIL_ENA BIT(5) +#define VIRTCHNL_IPSEC_ARW_CHECK_ENA BIT(6) +#define VIRTCHNL_IPSEC_24BIT_SPI_ENA BIT(7) + +/* algorithm type */ +/* Hash Algorithm */ +#define VIRTCHNL_HASH_NO_ALG 0 /* NULL algorithm */ +#define VIRTCHNL_AES_CBC_MAC 1 /* AES-CBC-MAC algorithm */ +#define VIRTCHNL_AES_CMAC 2 /* AES CMAC algorithm */ +#define VIRTCHNL_AES_GMAC 3 /* AES GMAC algorithm */ +#define VIRTCHNL_AES_XCBC_MAC 4 /* AES XCBC algorithm */ +#define VIRTCHNL_MD5_HMAC 5 /* HMAC using MD5 algorithm */ +#define VIRTCHNL_SHA1_HMAC 6 /* HMAC using 128 bit SHA algorithm */ +#define VIRTCHNL_SHA224_HMAC 7 /* HMAC using 224 bit SHA algorithm */ +#define VIRTCHNL_SHA256_HMAC 8 /* HMAC using 256 bit SHA algorithm */ +#define VIRTCHNL_SHA384_HMAC 9 /* HMAC using 384 bit SHA algorithm */ +#define VIRTCHNL_SHA512_HMAC 10 /* HMAC using 512 bit SHA algorithm */ +#define VIRTCHNL_SHA3_224_HMAC 11 /* HMAC using 224 bit SHA3 algorithm */ +#define VIRTCHNL_SHA3_256_HMAC 12 /* HMAC using 256 bit SHA3 algorithm */ +#define VIRTCHNL_SHA3_384_HMAC 13 /* HMAC using 384 bit SHA3 algorithm */ +#define VIRTCHNL_SHA3_512_HMAC 14 /* HMAC using 512 bit SHA3 algorithm */ +/* Cipher Algorithm */ +#define VIRTCHNL_CIPHER_NO_ALG 15 /* NULL algorithm */ +#define VIRTCHNL_3DES_CBC 16 /* Triple DES algorithm in CBC mode */ +#define VIRTCHNL_AES_CBC 17 /* AES algorithm in CBC mode */ +#define VIRTCHNL_AES_CTR 18 /* AES algorithm in Counter mode */ +/* AEAD Algorithm */ +#define VIRTCHNL_AES_CCM 19 /* AES algorithm in CCM mode */ +#define VIRTCHNL_AES_GCM 20 /* AES algorithm in GCM mode */ +#define VIRTCHNL_CHACHA20_POLY1305 21 /* algorithm of ChaCha20-Poly1305 */ + +/* protocol type */ +#define VIRTCHNL_PROTO_ESP 1 +#define VIRTCHNL_PROTO_AH 2 +#define VIRTCHNL_PROTO_RSVD1 3 + +/* sa mode */ +#define VIRTCHNL_SA_MODE_TRANSPORT 1 +#define VIRTCHNL_SA_MODE_TUNNEL 2 +#define VIRTCHNL_SA_MODE_TRAN_TUN 3 +#define VIRTCHNL_SA_MODE_UNKNOWN 4 + +/* sa direction */ +#define VIRTCHNL_DIR_INGRESS 1 +#define VIRTCHNL_DIR_EGRESS 2 +#define VIRTCHNL_DIR_INGRESS_EGRESS 3 + +/* sa termination */ +#define VIRTCHNL_TERM_SOFTWARE 1 +#define VIRTCHNL_TERM_HARDWARE 2 + +/* sa ip type */ +#define VIRTCHNL_IPV4 1 +#define VIRTCHNL_IPV6 2 + +/* for virtchnl_ipsec_resp */ +enum inline_ipsec_resp { + INLINE_IPSEC_SUCCESS = 0, + INLINE_IPSEC_FAIL = -1, + INLINE_IPSEC_ERR_FIFO_FULL = -2, + INLINE_IPSEC_ERR_NOT_READY = -3, + INLINE_IPSEC_ERR_VF_DOWN = -4, + INLINE_IPSEC_ERR_INVALID_PARAMS = -5, + INLINE_IPSEC_ERR_NO_MEM = -6, +}; + +/* Detailed opcodes for DPDK and IPsec use */ +enum inline_ipsec_ops { + INLINE_IPSEC_OP_GET_CAP = 0, + INLINE_IPSEC_OP_GET_STATUS = 1, + INLINE_IPSEC_OP_SA_CREATE = 2, + INLINE_IPSEC_OP_SA_UPDATE = 3, + INLINE_IPSEC_OP_SA_DESTROY = 4, + INLINE_IPSEC_OP_SP_CREATE = 5, + INLINE_IPSEC_OP_SP_DESTROY = 6, + INLINE_IPSEC_OP_SA_READ = 7, + INLINE_IPSEC_OP_EVENT = 8, + INLINE_IPSEC_OP_RESP = 9, +}; + +#pragma pack(1) +/* Not all valid, if certain field is invalid, set 1 for all bits */ +struct virtchnl_algo_cap { + u32 algo_type; + + u16 block_size; + + u16 min_key_size; + u16 max_key_size; + u16 inc_key_size; + + u16 min_iv_size; + u16 max_iv_size; + u16 inc_iv_size; + + u16 min_digest_size; + u16 max_digest_size; + u16 inc_digest_size; + + u16 min_aad_size; + u16 max_aad_size; + u16 inc_aad_size; +}; +#pragma pack() + +/* vf record the capability of crypto from the virtchnl */ +struct virtchnl_sym_crypto_cap { + u8 crypto_type; + u8 algo_cap_num; + struct virtchnl_algo_cap algo_cap_list[VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM]; +}; + +/* VIRTCHNL_OP_GET_IPSEC_CAP + * VF pass virtchnl_ipsec_cap to PF + * and PF return capability of ipsec from virtchnl. + */ +#pragma pack(1) +struct virtchnl_ipsec_cap { + /* max number of SA per VF */ + u16 max_sa_num; + + /* IPsec SA Protocol - value ref VIRTCHNL_PROTO_XXX */ + u8 virtchnl_protocol_type; + + /* IPsec SA Mode - value ref VIRTCHNL_SA_MODE_XXX */ + u8 virtchnl_sa_mode; + + /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */ + u8 virtchnl_direction; + + /* termination mode - value ref VIRTCHNL_TERM_XXX */ + u8 termination_mode; + + /* number of supported crypto capability */ + u8 crypto_cap_num; + + /* descriptor ID */ + u16 desc_id; + + /* capabilities enabled - value ref VIRTCHNL_IPSEC_XXX_ENA */ + u32 caps_enabled; + + /* crypto capabilities */ + struct virtchnl_sym_crypto_cap cap[VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM]; +}; + +/* configuration of crypto function */ +struct virtchnl_ipsec_crypto_cfg_item { + u8 crypto_type; + + u32 algo_type; + + /* Length of valid IV data. */ + u16 iv_len; + + /* Length of digest */ + u16 digest_len; + + /* SA salt */ + u32 salt; + + /* The length of the symmetric key */ + u16 key_len; + + /* key data buffer */ + u8 key_data[VIRTCHNL_IPSEC_MAX_KEY_LEN]; +}; +#pragma pack() + +struct virtchnl_ipsec_sym_crypto_cfg { + struct virtchnl_ipsec_crypto_cfg_item + items[VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER]; +}; + +#pragma pack(1) +/* VIRTCHNL_OP_IPSEC_SA_CREATE + * VF send this SA configuration to PF using virtchnl; + * PF create SA as configuration and PF driver will return + * an unique index (sa_idx) for the created SA. + */ +struct virtchnl_ipsec_sa_cfg { + /* IPsec SA Protocol - AH/ESP */ + u8 virtchnl_protocol_type; + + /* termination mode - value ref VIRTCHNL_TERM_XXX */ + u8 virtchnl_termination; + + /* type of outer IP - IPv4/IPv6 */ + u8 virtchnl_ip_type; + + /* type of esn - !0:enable/0:disable */ + u8 esn_enabled; + + /* udp encap - !0:enable/0:disable */ + u8 udp_encap_enabled; + + /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */ + u8 virtchnl_direction; + + /* reserved */ + u8 reserved1; + + /* SA security parameter index */ + u32 spi; + + /* outer src ip address */ + u8 src_addr[16]; + + /* outer dst ip address */ + u8 dst_addr[16]; + + /* SPD reference. Used to link an SA with its policy. + * PF drivers may ignore this field. + */ + u16 spd_ref; + + /* high 32 bits of esn */ + u32 esn_hi; + + /* low 32 bits of esn */ + u32 esn_low; + + /* When enabled, sa_index must be valid */ + u8 sa_index_en; + + /* SA index when sa_index_en is true */ + u32 sa_index; + + /* auditing mode - enable/disable */ + u8 audit_en; + + /* lifetime byte limit - enable/disable + * When enabled, byte_limit_hard and byte_limit_soft + * must be valid. + */ + u8 byte_limit_en; + + /* hard byte limit count */ + u64 byte_limit_hard; + + /* soft byte limit count */ + u64 byte_limit_soft; + + /* drop on authentication failure - enable/disable */ + u8 drop_on_auth_fail_en; + + /* anti-reply window check - enable/disable + * When enabled, arw_size must be valid. + */ + u8 arw_check_en; + + /* size of arw window, offset by 1. Setting to 0 + * represents ARW window size of 1. Setting to 127 + * represents ARW window size of 128 + */ + u8 arw_size; + + /* no ip offload mode - enable/disable + * When enabled, ip type and address must not be valid. + */ + u8 no_ip_offload_en; + + /* SA Domain. Used to logical separate an SADB into groups. + * PF drivers supporting a single group ignore this field. + */ + u16 sa_domain; + + /* crypto configuration */ + struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg; +}; +#pragma pack() + +/* VIRTCHNL_OP_IPSEC_SA_UPDATE + * VF send configuration of index of SA to PF + * PF will update SA according to configuration + */ +struct virtchnl_ipsec_sa_update { + u32 sa_index; /* SA to update */ + u32 esn_hi; /* high 32 bits of esn */ + u32 esn_low; /* low 32 bits of esn */ +}; + +#pragma pack(1) +/* VIRTCHNL_OP_IPSEC_SA_DESTROY + * VF send configuration of index of SA to PF + * PF will destroy SA according to configuration + * flag bitmap indicate all SA or just selected SA will + * be destroyed + */ +struct virtchnl_ipsec_sa_destroy { + /* All zero bitmap indicates all SA will be destroyed. + * Non-zero bitmap indicates the selected SA in + * array sa_index will be destroyed. + */ + u8 flag; + + /* selected SA index */ + u32 sa_index[VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM]; +}; + +/* VIRTCHNL_OP_IPSEC_SA_READ + * VF send this SA configuration to PF using virtchnl; + * PF read SA and will return configuration for the created SA. + */ +struct virtchnl_ipsec_sa_read { + /* SA valid - invalid/valid */ + u8 valid; + + /* SA active - inactive/active */ + u8 active; + + /* SA SN rollover - not_rollover/rollover */ + u8 sn_rollover; + + /* IPsec SA Protocol - AH/ESP */ + u8 virtchnl_protocol_type; + + /* termination mode - value ref VIRTCHNL_TERM_XXX */ + u8 virtchnl_termination; + + /* auditing mode - enable/disable */ + u8 audit_en; + + /* lifetime byte limit - enable/disable + * When set to limit, byte_limit_hard and byte_limit_soft + * must be valid. + */ + u8 byte_limit_en; + + /* hard byte limit count */ + u64 byte_limit_hard; + + /* soft byte limit count */ + u64 byte_limit_soft; + + /* drop on authentication failure - enable/disable */ + u8 drop_on_auth_fail_en; + + /* anti-replay window check - enable/disable + * When set to check, arw_size, arw_top, and arw must be valid + */ + u8 arw_check_en; + + /* size of arw window, offset by 1. Setting to 0 + * represents ARW window size of 1. Setting to 127 + * represents ARW window size of 128 + */ + u8 arw_size; + + /* reserved */ + u8 reserved1; + + /* top of anti-replay-window */ + u64 arw_top; + + /* anti-replay-window */ + u8 arw[16]; + + /* packets processed */ + u64 packets_processed; + + /* bytes processed */ + u64 bytes_processed; + + /* packets dropped */ + u32 packets_dropped; + + /* authentication failures */ + u32 auth_fails; + + /* ARW check failures */ + u32 arw_fails; + + /* type of esn - enable/disable */ + u8 esn; + + /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */ + u8 virtchnl_direction; + + /* SA security parameter index */ + u32 spi; + + /* SA salt */ + u32 salt; + + /* high 32 bits of esn */ + u32 esn_hi; + + /* low 32 bits of esn */ + u32 esn_low; + + /* SA Domain. Used to logical separate an SADB into groups. + * PF drivers supporting a single group ignore this field. + */ + u16 sa_domain; + + /* SPD reference. Used to link an SA with its policy. + * PF drivers may ignore this field. + */ + u16 spd_ref; + + /* crypto configuration. Salt and keys are set to 0 */ + struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg; +}; +#pragma pack() + +/* Add allowlist entry in IES */ +struct virtchnl_ipsec_sp_cfg { + u32 spi; + u32 dip[4]; + + /* Drop frame if true or redirect to QAT if false. */ + u8 drop; + + /* Congestion domain. For future use. */ + u8 cgd; + + /* 0 for IPv4 table, 1 for IPv6 table. */ + u8 table_id; + + /* Set TC (congestion domain) if true. For future use. */ + u8 set_tc; + + /* 0 for NAT-T unsupported, 1 for NAT-T supported */ + u8 is_udp; + + /* reserved */ + u8 reserved; + + /* NAT-T UDP port number. Only valid in case NAT-T supported */ + u16 udp_port; +}; + +#pragma pack(1) +/* Delete allowlist entry in IES */ +struct virtchnl_ipsec_sp_destroy { + /* 0 for IPv4 table, 1 for IPv6 table. */ + u8 table_id; + u32 rule_id; +}; +#pragma pack() + +/* Response from IES to allowlist operations */ +struct virtchnl_ipsec_sp_cfg_resp { + u32 rule_id; +}; + +struct virtchnl_ipsec_sa_cfg_resp { + u32 sa_handle; +}; + +#define INLINE_IPSEC_EVENT_RESET 0x1 +#define INLINE_IPSEC_EVENT_CRYPTO_ON 0x2 +#define INLINE_IPSEC_EVENT_CRYPTO_OFF 0x4 + +struct virtchnl_ipsec_event { + u32 ipsec_event_data; +}; + +#define INLINE_IPSEC_STATUS_AVAILABLE 0x1 +#define INLINE_IPSEC_STATUS_UNAVAILABLE 0x2 + +struct virtchnl_ipsec_status { + u32 status; +}; + +struct virtchnl_ipsec_resp { + u32 resp; +}; + +/* Internal message descriptor for VF <-> IPsec communication */ +struct inline_ipsec_msg { + u16 ipsec_opcode; + u16 req_id; + + union { + /* IPsec request */ + struct virtchnl_ipsec_sa_cfg sa_cfg[0]; + struct virtchnl_ipsec_sp_cfg sp_cfg[0]; + struct virtchnl_ipsec_sa_update sa_update[0]; + struct virtchnl_ipsec_sa_destroy sa_destroy[0]; + struct virtchnl_ipsec_sp_destroy sp_destroy[0]; + + /* IPsec response */ + struct virtchnl_ipsec_sa_cfg_resp sa_cfg_resp[0]; + struct virtchnl_ipsec_sp_cfg_resp sp_cfg_resp[0]; + struct virtchnl_ipsec_cap ipsec_cap[0]; + struct virtchnl_ipsec_status ipsec_status[0]; + /* response to del_sa, del_sp, update_sa */ + struct virtchnl_ipsec_resp ipsec_resp[0]; + + /* IPsec event (no req_id is required) */ + struct virtchnl_ipsec_event event[0]; + + /* Reserved */ + struct virtchnl_ipsec_sa_read sa_read[0]; + } ipsec_data; +}; + +static inline u16 virtchnl_inline_ipsec_val_msg_len(u16 opcode) +{ + u16 valid_len = sizeof(struct inline_ipsec_msg); + + switch (opcode) { + case INLINE_IPSEC_OP_GET_CAP: + case INLINE_IPSEC_OP_GET_STATUS: + break; + case INLINE_IPSEC_OP_SA_CREATE: + valid_len += sizeof(struct virtchnl_ipsec_sa_cfg); + break; + case INLINE_IPSEC_OP_SP_CREATE: + valid_len += sizeof(struct virtchnl_ipsec_sp_cfg); + break; + case INLINE_IPSEC_OP_SA_UPDATE: + valid_len += sizeof(struct virtchnl_ipsec_sa_update); + break; + case INLINE_IPSEC_OP_SA_DESTROY: + valid_len += sizeof(struct virtchnl_ipsec_sa_destroy); + break; + case INLINE_IPSEC_OP_SP_DESTROY: + valid_len += sizeof(struct virtchnl_ipsec_sp_destroy); + break; + /* Only for msg length calculation of response to VF in case of + * inline ipsec failure. + */ + case INLINE_IPSEC_OP_RESP: + valid_len += sizeof(struct virtchnl_ipsec_resp); + break; + default: + valid_len = 0; + break; + } + + return valid_len; +} + +#endif /* _VIRTCHNL_INLINE_IPSEC_H_ */ diff --git a/drivers/common/idpf/meson.build b/drivers/common/idpf/meson.build new file mode 100644 index 0000000000..26881166e9 --- /dev/null +++ b/drivers/common/idpf/meson.build @@ -0,0 +1,4 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2022 Intel Corporation + +subdir('base') \ No newline at end of file diff --git a/drivers/common/idpf/version.map b/drivers/common/idpf/version.map new file mode 100644 index 0000000000..bfb246c752 --- /dev/null +++ b/drivers/common/idpf/version.map @@ -0,0 +1,12 @@ +INTERNAL { + global: + + idpf_ctlq_deinit; + idpf_ctlq_init; + idpf_ctlq_clean_sq; + idpf_ctlq_recv; + idpf_ctlq_send; + idpf_ctlq_post_rx_buffs; + + local: *; +}; diff --git a/drivers/common/meson.build b/drivers/common/meson.build index ea261dd70a..b63d899d50 100644 --- a/drivers/common/meson.build +++ b/drivers/common/meson.build @@ -6,6 +6,7 @@ drivers = [ 'cpt', 'dpaax', 'iavf', + 'idpf', 'mvep', 'octeontx', ] From patchwork Mon Oct 31 08:33:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Xing, Beilei" X-Patchwork-Id: 119341 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 33CF2A00C2; Mon, 31 Oct 2022 10:04:22 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1FF6740695; Mon, 31 Oct 2022 10:04:22 +0100 (CET) Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id B9ABB400D5 for ; Mon, 31 Oct 2022 10:04:16 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667207056; x=1698743056; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=6bZ1hRDKHV/mXdzRPakxeKCwRaUlF4gS5olfKOsCpww=; b=PIf2+SW+TMbXwNHy28e3L/dswY/8TXmyEsFDnlms/wxWjVn+fk7QC11x +PhxT70IwNqXNJUqZuIOWofNkzDBxijvRvDIs7RghpJQ3/9kvwJWmhNgj SWll+7D9i2XIN0+U9w76G4znmM8r0Z895NvUH0WBqXGk5LgV5wWa3HUrS 69JljcHWOBz55mbCAaq1c7qBLp58tE/ysBvPtGGVW/dZes9JQetF74OCI ct+ZnvVjVj3qn6CKqaWXG3XrfPSvEOGxoQkyZQN3oXjSuTjltZj0m+bQ+ pFmCF44ctiSZORDPB78ozmm5t4EH2geHVZ1yPrcHypEWClFlekw0I1KM8 w==; X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="370926269" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="370926269" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Oct 2022 02:04:02 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="878664774" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="878664774" Received: from dpdk-beileix-3.sh.intel.com ([10.67.110.253]) by fmsmga006.fm.intel.com with ESMTP; 31 Oct 2022 02:03:59 -0700 From: beilei.xing@intel.com To: jingjing.wu@intel.com, beilei.xing@intel.com Cc: dev@dpdk.org, Junfeng Guo , Xiaoyun Li , Xiao Wang , Wenjun Wu Subject: [PATCH v18 02/18] net/idpf: add support for device initialization Date: Mon, 31 Oct 2022 08:33:30 +0000 Message-Id: <20221031083346.16558-3-beilei.xing@intel.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20221031083346.16558-1-beilei.xing@intel.com> References: <20221031051556.98549-1-beilei.xing@intel.com> <20221031083346.16558-1-beilei.xing@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Junfeng Guo Support device init and add the following dev ops: - dev_configure - dev_close - dev_infos_get Signed-off-by: Beilei Xing Signed-off-by: Xiaoyun Li Signed-off-by: Xiao Wang Signed-off-by: Wenjun Wu Signed-off-by: Junfeng Guo --- MAINTAINERS | 9 + doc/guides/nics/features/idpf.ini | 9 + doc/guides/nics/idpf.rst | 66 ++ doc/guides/nics/index.rst | 1 + doc/guides/rel_notes/release_22_11.rst | 6 + drivers/net/idpf/idpf_ethdev.c | 891 +++++++++++++++++++++++++ drivers/net/idpf/idpf_ethdev.h | 189 ++++++ drivers/net/idpf/idpf_logs.h | 56 ++ drivers/net/idpf/idpf_vchnl.c | 416 ++++++++++++ drivers/net/idpf/meson.build | 15 + drivers/net/idpf/version.map | 3 + drivers/net/meson.build | 1 + 12 files changed, 1662 insertions(+) create mode 100644 doc/guides/nics/features/idpf.ini create mode 100644 doc/guides/nics/idpf.rst create mode 100644 drivers/net/idpf/idpf_ethdev.c create mode 100644 drivers/net/idpf/idpf_ethdev.h create mode 100644 drivers/net/idpf/idpf_logs.h create mode 100644 drivers/net/idpf/idpf_vchnl.c create mode 100644 drivers/net/idpf/meson.build create mode 100644 drivers/net/idpf/version.map diff --git a/MAINTAINERS b/MAINTAINERS index bdf233c9f8..cc66db25e8 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -770,6 +770,15 @@ F: drivers/net/ice/ F: doc/guides/nics/ice.rst F: doc/guides/nics/features/ice.ini +Intel idpf +M: Jingjing Wu +M: Beilei Xing +T: git://dpdk.org/next/dpdk-next-net-intel +F: drivers/net/idpf/ +F: drivers/common/idpf/ +F: doc/guides/nics/idpf.rst +F: doc/guides/nics/features/idpf.ini + Intel igc M: Junfeng Guo M: Simei Su diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini new file mode 100644 index 0000000000..46aab2eb61 --- /dev/null +++ b/doc/guides/nics/features/idpf.ini @@ -0,0 +1,9 @@ +; +; Supported features of the 'idpf' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Linux = Y +x86-32 = Y +x86-64 = Y diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst new file mode 100644 index 0000000000..c1001d5d0c --- /dev/null +++ b/doc/guides/nics/idpf.rst @@ -0,0 +1,66 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2022 Intel Corporation. + +IDPF Poll Mode Driver +====================== + +The [*EXPERIMENTAL*] idpf PMD (**librte_net_idpf**) provides poll mode driver support for +Intel® Infrastructure Processing Unit (Intel® IPU) E2000. + + +Linux Prerequisites +------------------- + +- Follow the DPDK :ref:`Getting Started Guide for Linux ` to setup the basic DPDK environment. + +- To get better performance on Intel platforms, please follow the "How to get best performance with NICs on Intel platforms" + section of the :ref:`Getting Started Guide for Linux `. + + +Pre-Installation Configuration +------------------------------ + +Runtime Config Options +~~~~~~~~~~~~~~~~~~~~~~ + +- ``vport`` (default ``0``) + + The IDPF PMD supports creation of multiple vports for one PCI device, each vport + corresponds to a single ethdev. Using the ``devargs`` parameter ``vport`` the user + can specify the vports with specific ID to be created, for example:: + + -a ca:00.0,vport=[0,2,3] + + Then idpf PMD will create 3 vports (ethdevs) for device ca:00.0. + NOTE: If the parameter is not provided, the vport 0 will be created by default. + +- ``rx_single`` (default ``0``) + + There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue + mode and split queue mode for Rx queue. User can choose Rx queue mode by the ``devargs`` + parameter ``rx_single``. + + -a ca:00.0,rx_single=1 + + Then idpf PMD will configure Rx queue with single queue mode. Otherwise, split queue + mode is chosen by default. + +- ``tx_single`` (default ``0``) + + There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue + mode and split queue mode for Tx queue. User can choose Tx queue mode by the ``devargs`` + parameter ``tx_single``. + + -a ca:00.0,tx_single=1 + + Then idpf PMD will configure Tx queue with single queue mode. Otherwise, split queue + mode is chosen by default. + + +Driver compilation and testing +------------------------------ + +Refer to the document :ref:`compiling and testing a PMD for a NIC ` +for details. + + diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst index 4d40ea29a3..12841ce407 100644 --- a/doc/guides/nics/index.rst +++ b/doc/guides/nics/index.rst @@ -34,6 +34,7 @@ Network Interface Controller Drivers hns3 i40e ice + idpf igb igc ionic diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst index 28812e092f..77674f2b06 100644 --- a/doc/guides/rel_notes/release_22_11.rst +++ b/doc/guides/rel_notes/release_22_11.rst @@ -158,6 +158,12 @@ New Features * Added protocol based buffer split support in scalar path. +* **Added IDPF PMD [*EXPERIMENTAL*].** + + Added the new ``idpf`` net driver for Intel® Infrastructure Processing Unit + (Intel® IPU) E2000. + See the :doc:`../nics/idpf` NIC guide for more details on this new driver. + * **Updated Marvell cnxk driver.** * Added support for flow action REPRESENTED_PORT. diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c new file mode 100644 index 0000000000..035f563275 --- /dev/null +++ b/drivers/net/idpf/idpf_ethdev.c @@ -0,0 +1,891 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2022 Intel Corporation + */ + +#include +#include +#include +#include +#include +#include +#include + +#include "idpf_ethdev.h" + +#define IDPF_TX_SINGLE_Q "tx_single" +#define IDPF_RX_SINGLE_Q "rx_single" +#define IDPF_VPORT "vport" + +rte_spinlock_t idpf_adapter_lock; +/* A list for all adapters, one adapter matches one PCI device */ +struct idpf_adapter_list idpf_adapter_list; +bool idpf_adapter_list_init; + +static const char * const idpf_valid_args[] = { + IDPF_TX_SINGLE_Q, + IDPF_RX_SINGLE_Q, + IDPF_VPORT, + NULL +}; + +static int +idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) +{ + struct idpf_vport *vport = dev->data->dev_private; + struct idpf_adapter *adapter = vport->adapter; + + dev_info->max_rx_queues = adapter->caps->max_rx_q; + dev_info->max_tx_queues = adapter->caps->max_tx_q; + dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE; + dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE; + + dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD; + dev_info->min_mtu = RTE_ETHER_MIN_MTU; + + return 0; +} + +static int +idpf_init_vport_req_info(struct rte_eth_dev *dev) +{ + struct idpf_vport *vport = dev->data->dev_private; + struct idpf_adapter *adapter = vport->adapter; + struct virtchnl2_create_vport *vport_info; + uint16_t idx = adapter->cur_vport_idx; + + if (idx == IDPF_INVALID_VPORT_IDX) { + PMD_INIT_LOG(ERR, "Invalid vport index."); + return -EINVAL; + } + + if (adapter->vport_req_info[idx] == NULL) { + adapter->vport_req_info[idx] = rte_zmalloc(NULL, + sizeof(struct virtchnl2_create_vport), 0); + if (adapter->vport_req_info[idx] == NULL) { + PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info"); + return -ENOMEM; + } + } + + vport_info = + (struct virtchnl2_create_vport *)adapter->vport_req_info[idx]; + + vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT); + if (adapter->txq_model == 0) { + vport_info->txq_model = + rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT); + vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM; + vport_info->num_tx_complq = + IDPF_DEFAULT_TXQ_NUM * IDPF_TX_COMPLQ_PER_GRP; + } else { + vport_info->txq_model = + rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE); + vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM; + vport_info->num_tx_complq = 0; + } + if (adapter->rxq_model == 0) { + vport_info->rxq_model = + rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT); + vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM; + vport_info->num_rx_bufq = + IDPF_DEFAULT_RXQ_NUM * IDPF_RX_BUFQ_PER_GRP; + } else { + vport_info->rxq_model = + rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE); + vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM; + vport_info->num_rx_bufq = 0; + } + + return 0; +} + +static int +idpf_parse_devarg_id(char *name) +{ + uint16_t val; + char *p; + + p = strstr(name, "vport_"); + + if (p == NULL) + return -EINVAL; + + p += sizeof("vport_") - 1; + + val = strtoul(p, NULL, 10); + + return val; +} + +static int +idpf_init_vport(struct rte_eth_dev *dev) +{ + struct idpf_vport *vport = dev->data->dev_private; + struct idpf_adapter *adapter = vport->adapter; + uint16_t idx = adapter->cur_vport_idx; + struct virtchnl2_create_vport *vport_info = + (struct virtchnl2_create_vport *)adapter->vport_recv_info[idx]; + int i, type, ret; + + vport->vport_id = vport_info->vport_id; + vport->txq_model = vport_info->txq_model; + vport->rxq_model = vport_info->rxq_model; + vport->num_tx_q = vport_info->num_tx_q; + vport->num_tx_complq = vport_info->num_tx_complq; + vport->num_rx_q = vport_info->num_rx_q; + vport->num_rx_bufq = vport_info->num_rx_bufq; + vport->max_mtu = vport_info->max_mtu; + rte_memcpy(vport->default_mac_addr, + vport_info->default_mac_addr, ETH_ALEN); + vport->sw_idx = idx; + + for (i = 0; i < vport_info->chunks.num_chunks; i++) { + type = vport_info->chunks.chunks[i].type; + switch (type) { + case VIRTCHNL2_QUEUE_TYPE_TX: + vport->chunks_info.tx_start_qid = + vport_info->chunks.chunks[i].start_queue_id; + vport->chunks_info.tx_qtail_start = + vport_info->chunks.chunks[i].qtail_reg_start; + vport->chunks_info.tx_qtail_spacing = + vport_info->chunks.chunks[i].qtail_reg_spacing; + break; + case VIRTCHNL2_QUEUE_TYPE_RX: + vport->chunks_info.rx_start_qid = + vport_info->chunks.chunks[i].start_queue_id; + vport->chunks_info.rx_qtail_start = + vport_info->chunks.chunks[i].qtail_reg_start; + vport->chunks_info.rx_qtail_spacing = + vport_info->chunks.chunks[i].qtail_reg_spacing; + break; + case VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION: + vport->chunks_info.tx_compl_start_qid = + vport_info->chunks.chunks[i].start_queue_id; + vport->chunks_info.tx_compl_qtail_start = + vport_info->chunks.chunks[i].qtail_reg_start; + vport->chunks_info.tx_compl_qtail_spacing = + vport_info->chunks.chunks[i].qtail_reg_spacing; + break; + case VIRTCHNL2_QUEUE_TYPE_RX_BUFFER: + vport->chunks_info.rx_buf_start_qid = + vport_info->chunks.chunks[i].start_queue_id; + vport->chunks_info.rx_buf_qtail_start = + vport_info->chunks.chunks[i].qtail_reg_start; + vport->chunks_info.rx_buf_qtail_spacing = + vport_info->chunks.chunks[i].qtail_reg_spacing; + break; + default: + PMD_INIT_LOG(ERR, "Unsupported queue type"); + break; + } + } + + ret = idpf_parse_devarg_id(dev->data->name); + if (ret < 0) { + PMD_INIT_LOG(ERR, "Failed to parse devarg id."); + return -EINVAL; + } + vport->devarg_id = ret; + + vport->dev_data = dev->data; + + adapter->vports[idx] = vport; + + return 0; +} + +static int +idpf_dev_configure(struct rte_eth_dev *dev) +{ + struct rte_eth_conf *conf = &dev->data->dev_conf; + + if (conf->link_speeds & RTE_ETH_LINK_SPEED_FIXED) { + PMD_INIT_LOG(ERR, "Setting link speed is not supported"); + return -ENOTSUP; + } + + if (dev->data->nb_rx_queues == 1 && conf->rxmode.mq_mode != RTE_ETH_MQ_RX_NONE) { + PMD_INIT_LOG(ERR, "Multi-queue packet distribution mode %d is not supported", + conf->rxmode.mq_mode); + return -ENOTSUP; + } + + if (conf->txmode.mq_mode != RTE_ETH_MQ_TX_NONE) { + PMD_INIT_LOG(ERR, "Multi-queue TX mode %d is not supported", + conf->txmode.mq_mode); + return -ENOTSUP; + } + + if (conf->lpbk_mode != 0) { + PMD_INIT_LOG(ERR, "Loopback operation mode %d is not supported", + conf->lpbk_mode); + return -ENOTSUP; + } + + if (conf->dcb_capability_en != 0) { + PMD_INIT_LOG(ERR, "Priority Flow Control(PFC) if not supported"); + return -ENOTSUP; + } + + if (conf->intr_conf.lsc != 0) { + PMD_INIT_LOG(ERR, "LSC interrupt is not supported"); + return -ENOTSUP; + } + + if (conf->intr_conf.rxq != 0) { + PMD_INIT_LOG(ERR, "RXQ interrupt is not supported"); + return -ENOTSUP; + } + + if (conf->intr_conf.rmv != 0) { + PMD_INIT_LOG(ERR, "RMV interrupt is not supported"); + return -ENOTSUP; + } + + return 0; +} + +static int +idpf_dev_close(struct rte_eth_dev *dev) +{ + struct idpf_vport *vport = dev->data->dev_private; + struct idpf_adapter *adapter = vport->adapter; + + idpf_vc_destroy_vport(vport); + + + adapter->cur_vports &= ~RTE_BIT32(vport->devarg_id); + + rte_free(vport); + dev->data->dev_private = NULL; + + return 0; +} + +static int +insert_value(struct idpf_adapter *adapter, uint16_t id) +{ + uint16_t i; + + for (i = 0; i < adapter->req_vport_nb; i++) { + if (adapter->req_vports[i] == id) + return 0; + } + + if (adapter->req_vport_nb >= RTE_DIM(adapter->req_vports)) { + PMD_INIT_LOG(ERR, "Total vport number can't be > %d", + IDPF_MAX_VPORT_NUM); + return -EINVAL; + } + + adapter->req_vports[adapter->req_vport_nb] = id; + adapter->req_vport_nb++; + + return 0; +} + +static const char * +parse_range(const char *value, struct idpf_adapter *adapter) +{ + uint16_t lo, hi, i; + int n = 0; + int result; + const char *pos = value; + + result = sscanf(value, "%hu%n-%hu%n", &lo, &n, &hi, &n); + if (result == 1) { + if (lo >= IDPF_MAX_VPORT_NUM) + return NULL; + if (insert_value(adapter, lo) != 0) + return NULL; + } else if (result == 2) { + if (lo > hi || hi >= IDPF_MAX_VPORT_NUM) + return NULL; + for (i = lo; i <= hi; i++) { + if (insert_value(adapter, i) != 0) + return NULL; + } + } else { + return NULL; + } + + return pos + n; +} + +static int +parse_vport(const char *key, const char *value, void *args) +{ + struct idpf_adapter *adapter = args; + const char *pos = value; + int i; + + adapter->req_vport_nb = 0; + + if (*pos == '[') + pos++; + + while (1) { + pos = parse_range(pos, adapter); + if (pos == NULL) { + PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ", + value, key); + return -EINVAL; + } + if (*pos != ',') + break; + pos++; + } + + if (*value == '[' && *pos != ']') { + PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ", + value, key); + return -EINVAL; + } + + if (adapter->cur_vport_nb + adapter->req_vport_nb > + IDPF_MAX_VPORT_NUM) { + PMD_INIT_LOG(ERR, "Total vport number can't be > %d", + IDPF_MAX_VPORT_NUM); + return -EINVAL; + } + + for (i = 0; i < adapter->req_vport_nb; i++) { + if ((adapter->cur_vports & RTE_BIT32(adapter->req_vports[i])) == 0) { + adapter->cur_vports |= RTE_BIT32(adapter->req_vports[i]); + adapter->cur_vport_nb++; + } else { + PMD_INIT_LOG(ERR, "Vport %d has been created", + adapter->req_vports[i]); + return -EINVAL; + } + } + + return 0; +} + +static int +parse_bool(const char *key, const char *value, void *args) +{ + int *i = args; + char *end; + int num; + + errno = 0; + + num = strtoul(value, &end, 10); + + if (errno == ERANGE || (num != 0 && num != 1)) { + PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", value must be 0 or 1", + value, key); + return -EINVAL; + } + + *i = num; + return 0; +} + +static int +idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter) +{ + struct rte_devargs *devargs = pci_dev->device.devargs; + struct rte_kvargs *kvlist; + int ret; + + if (devargs == NULL) + return 0; + + kvlist = rte_kvargs_parse(devargs->args, idpf_valid_args); + if (kvlist == NULL) { + PMD_INIT_LOG(ERR, "invalid kvargs key"); + return -EINVAL; + } + + ret = rte_kvargs_process(kvlist, IDPF_VPORT, &parse_vport, + adapter); + if (ret != 0) + goto bail; + + ret = rte_kvargs_process(kvlist, IDPF_TX_SINGLE_Q, &parse_bool, + &adapter->txq_model); + if (ret != 0) + goto bail; + + ret = rte_kvargs_process(kvlist, IDPF_RX_SINGLE_Q, &parse_bool, + &adapter->rxq_model); + if (ret != 0) + goto bail; + +bail: + rte_kvargs_free(kvlist); + return ret; +} + +static void +idpf_reset_pf(struct idpf_hw *hw) +{ + uint32_t reg; + + reg = IDPF_READ_REG(hw, PFGEN_CTRL); + IDPF_WRITE_REG(hw, PFGEN_CTRL, (reg | PFGEN_CTRL_PFSWR)); +} + +#define IDPF_RESET_WAIT_CNT 100 +static int +idpf_check_pf_reset_done(struct idpf_hw *hw) +{ + uint32_t reg; + int i; + + for (i = 0; i < IDPF_RESET_WAIT_CNT; i++) { + reg = IDPF_READ_REG(hw, PFGEN_RSTAT); + if (reg != 0xFFFFFFFF && (reg & PFGEN_RSTAT_PFR_STATE_M)) + return 0; + rte_delay_ms(1000); + } + + PMD_INIT_LOG(ERR, "IDPF reset timeout"); + return -EBUSY; +} + +#define CTLQ_NUM 2 +static int +idpf_init_mbx(struct idpf_hw *hw) +{ + struct idpf_ctlq_create_info ctlq_info[CTLQ_NUM] = { + { + .type = IDPF_CTLQ_TYPE_MAILBOX_TX, + .id = IDPF_CTLQ_ID, + .len = IDPF_CTLQ_LEN, + .buf_size = IDPF_DFLT_MBX_BUF_SIZE, + .reg = { + .head = PF_FW_ATQH, + .tail = PF_FW_ATQT, + .len = PF_FW_ATQLEN, + .bah = PF_FW_ATQBAH, + .bal = PF_FW_ATQBAL, + .len_mask = PF_FW_ATQLEN_ATQLEN_M, + .len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M, + .head_mask = PF_FW_ATQH_ATQH_M, + } + }, + { + .type = IDPF_CTLQ_TYPE_MAILBOX_RX, + .id = IDPF_CTLQ_ID, + .len = IDPF_CTLQ_LEN, + .buf_size = IDPF_DFLT_MBX_BUF_SIZE, + .reg = { + .head = PF_FW_ARQH, + .tail = PF_FW_ARQT, + .len = PF_FW_ARQLEN, + .bah = PF_FW_ARQBAH, + .bal = PF_FW_ARQBAL, + .len_mask = PF_FW_ARQLEN_ARQLEN_M, + .len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M, + .head_mask = PF_FW_ARQH_ARQH_M, + } + } + }; + struct idpf_ctlq_info *ctlq; + int ret; + + ret = idpf_ctlq_init(hw, CTLQ_NUM, ctlq_info); + if (ret != 0) + return ret; + + LIST_FOR_EACH_ENTRY_SAFE(ctlq, NULL, &hw->cq_list_head, + struct idpf_ctlq_info, cq_list) { + if (ctlq->q_id == IDPF_CTLQ_ID && + ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX) + hw->asq = ctlq; + if (ctlq->q_id == IDPF_CTLQ_ID && + ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX) + hw->arq = ctlq; + } + + if (hw->asq == NULL || hw->arq == NULL) { + idpf_ctlq_deinit(hw); + ret = -ENOENT; + } + + return ret; +} + +static int +idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter) +{ + struct idpf_hw *hw = &adapter->hw; + int ret = 0; + + hw->hw_addr = (void *)pci_dev->mem_resource[0].addr; + hw->hw_addr_len = pci_dev->mem_resource[0].len; + hw->back = adapter; + hw->vendor_id = pci_dev->id.vendor_id; + hw->device_id = pci_dev->id.device_id; + hw->subsystem_vendor_id = pci_dev->id.subsystem_vendor_id; + + strncpy(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE); + + idpf_reset_pf(hw); + ret = idpf_check_pf_reset_done(hw); + if (ret != 0) { + PMD_INIT_LOG(ERR, "IDPF is still resetting"); + goto err; + } + + ret = idpf_init_mbx(hw); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Failed to init mailbox"); + goto err; + } + + adapter->mbx_resp = rte_zmalloc("idpf_adapter_mbx_resp", + IDPF_DFLT_MBX_BUF_SIZE, 0); + if (adapter->mbx_resp == NULL) { + PMD_INIT_LOG(ERR, "Failed to allocate idpf_adapter_mbx_resp memory"); + ret = -ENOMEM; + goto err_mbx; + } + + ret = idpf_vc_check_api_version(adapter); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Failed to check api version"); + goto err_api; + } + + adapter->caps = rte_zmalloc("idpf_caps", + sizeof(struct virtchnl2_get_capabilities), 0); + if (adapter->caps == NULL) { + PMD_INIT_LOG(ERR, "Failed to allocate idpf_caps memory"); + ret = -ENOMEM; + goto err_api; + } + + ret = idpf_vc_get_caps(adapter); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Failed to get capabilities"); + goto err_caps; + } + + adapter->max_vport_nb = adapter->caps->max_vports; + + adapter->vport_req_info = rte_zmalloc("vport_req_info", + adapter->max_vport_nb * + sizeof(*adapter->vport_req_info), + 0); + if (adapter->vport_req_info == NULL) { + PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info memory"); + ret = -ENOMEM; + goto err_caps; + } + + adapter->vport_recv_info = rte_zmalloc("vport_recv_info", + adapter->max_vport_nb * + sizeof(*adapter->vport_recv_info), + 0); + if (adapter->vport_recv_info == NULL) { + PMD_INIT_LOG(ERR, "Failed to allocate vport_recv_info memory"); + ret = -ENOMEM; + goto err_vport_recv_info; + } + + adapter->vports = rte_zmalloc("vports", + adapter->max_vport_nb * + sizeof(*adapter->vports), + 0); + if (adapter->vports == NULL) { + PMD_INIT_LOG(ERR, "Failed to allocate vports memory"); + ret = -ENOMEM; + goto err_vports; + } + + adapter->max_rxq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE - + sizeof(struct virtchnl2_config_rx_queues)) / + sizeof(struct virtchnl2_rxq_info); + adapter->max_txq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE - + sizeof(struct virtchnl2_config_tx_queues)) / + sizeof(struct virtchnl2_txq_info); + + adapter->cur_vports = 0; + adapter->cur_vport_nb = 0; + + return ret; + +err_vports: + rte_free(adapter->vport_recv_info); + adapter->vport_recv_info = NULL; +err_vport_recv_info: + rte_free(adapter->vport_req_info); + adapter->vport_req_info = NULL; +err_caps: + rte_free(adapter->caps); + adapter->caps = NULL; +err_api: + rte_free(adapter->mbx_resp); + adapter->mbx_resp = NULL; +err_mbx: + idpf_ctlq_deinit(hw); +err: + return ret; +} + +static const struct eth_dev_ops idpf_eth_dev_ops = { + .dev_configure = idpf_dev_configure, + .dev_close = idpf_dev_close, + .dev_infos_get = idpf_dev_info_get, +}; + +static uint16_t +idpf_get_vport_idx(struct idpf_vport **vports, uint16_t max_vport_nb) +{ + uint16_t vport_idx; + uint16_t i; + + for (i = 0; i < max_vport_nb; i++) { + if (vports[i] == NULL) + break; + } + + if (i == max_vport_nb) + vport_idx = IDPF_INVALID_VPORT_IDX; + else + vport_idx = i; + + return vport_idx; +} + +static int +idpf_dev_init(struct rte_eth_dev *dev, void *init_params) +{ + struct idpf_vport *vport = dev->data->dev_private; + struct idpf_adapter *adapter = init_params; + int ret = 0; + + dev->dev_ops = &idpf_eth_dev_ops; + vport->adapter = adapter; + + ret = idpf_init_vport_req_info(dev); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Failed to init vport req_info."); + goto err; + } + + ret = idpf_vc_create_vport(adapter); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Failed to create vport."); + goto err_create_vport; + } + + ret = idpf_init_vport(dev); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Failed to init vports."); + goto err_init_vport; + } + + adapter->cur_vport_idx = idpf_get_vport_idx(adapter->vports, + adapter->max_vport_nb); + + dev->data->mac_addrs = rte_zmalloc(NULL, RTE_ETHER_ADDR_LEN, 0); + if (dev->data->mac_addrs == NULL) { + PMD_INIT_LOG(ERR, "Cannot allocate mac_addr memory."); + ret = -ENOMEM; + goto err_init_vport; + } + + rte_ether_addr_copy((struct rte_ether_addr *)vport->default_mac_addr, + &dev->data->mac_addrs[0]); + + return 0; + +err_init_vport: + idpf_vc_destroy_vport(vport); +err_create_vport: + rte_free(vport->adapter->vport_req_info[vport->adapter->cur_vport_idx]); +err: + return ret; +} + +static const struct rte_pci_id pci_id_idpf_map[] = { + { RTE_PCI_DEVICE(IDPF_INTEL_VENDOR_ID, IDPF_DEV_ID_PF) }, + { .vendor_id = 0, /* sentinel */ }, +}; + +struct idpf_adapter * +idpf_find_adapter(struct rte_pci_device *pci_dev) +{ + struct idpf_adapter *adapter; + int found = 0; + + if (pci_dev == NULL) + return NULL; + + rte_spinlock_lock(&idpf_adapter_lock); + TAILQ_FOREACH(adapter, &idpf_adapter_list, next) { + if (strncmp(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE) == 0) { + found = 1; + break; + } + } + rte_spinlock_unlock(&idpf_adapter_lock); + + if (found == 0) + return NULL; + + return adapter; +} + +static void +idpf_adapter_rel(struct idpf_adapter *adapter) +{ + struct idpf_hw *hw = &adapter->hw; + int i; + + idpf_ctlq_deinit(hw); + + rte_free(adapter->caps); + adapter->caps = NULL; + + rte_free(adapter->mbx_resp); + adapter->mbx_resp = NULL; + + if (adapter->vport_req_info != NULL) { + for (i = 0; i < adapter->max_vport_nb; i++) { + rte_free(adapter->vport_req_info[i]); + adapter->vport_req_info[i] = NULL; + } + rte_free(adapter->vport_req_info); + adapter->vport_req_info = NULL; + } + + if (adapter->vport_recv_info != NULL) { + for (i = 0; i < adapter->max_vport_nb; i++) { + rte_free(adapter->vport_recv_info[i]); + adapter->vport_recv_info[i] = NULL; + } + rte_free(adapter->vport_recv_info); + adapter->vport_recv_info = NULL; + } + + rte_free(adapter->vports); + adapter->vports = NULL; +} + +static int +idpf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, + struct rte_pci_device *pci_dev) +{ + struct idpf_adapter *adapter; + char name[RTE_ETH_NAME_MAX_LEN]; + int i, retval; + bool first_probe = false; + + if (!idpf_adapter_list_init) { + rte_spinlock_init(&idpf_adapter_lock); + TAILQ_INIT(&idpf_adapter_list); + idpf_adapter_list_init = true; + } + + adapter = idpf_find_adapter(pci_dev); + if (adapter == NULL) { + first_probe = true; + adapter = rte_zmalloc("idpf_adapter", + sizeof(struct idpf_adapter), 0); + if (adapter == NULL) { + PMD_INIT_LOG(ERR, "Failed to allocate adapter."); + return -ENOMEM; + } + + retval = idpf_adapter_init(pci_dev, adapter); + if (retval != 0) { + PMD_INIT_LOG(ERR, "Failed to init adapter."); + return retval; + } + + rte_spinlock_lock(&idpf_adapter_lock); + TAILQ_INSERT_TAIL(&idpf_adapter_list, adapter, next); + rte_spinlock_unlock(&idpf_adapter_lock); + } + + retval = idpf_parse_devargs(pci_dev, adapter); + if (retval != 0) { + PMD_INIT_LOG(ERR, "Failed to parse private devargs"); + goto err; + } + + if (adapter->req_vport_nb == 0) { + /* If no vport devarg, create vport 0 by default. */ + snprintf(name, sizeof(name), "idpf_%s_vport_0", + pci_dev->device.name); + retval = rte_eth_dev_create(&pci_dev->device, name, + sizeof(struct idpf_vport), + NULL, NULL, idpf_dev_init, + adapter); + if (retval != 0) + PMD_DRV_LOG(ERR, "Failed to create default vport 0"); + adapter->cur_vports |= RTE_BIT32(0); + adapter->cur_vport_nb++; + } else { + for (i = 0; i < adapter->req_vport_nb; i++) { + snprintf(name, sizeof(name), "idpf_%s_vport_%d", + pci_dev->device.name, + adapter->req_vports[i]); + retval = rte_eth_dev_create(&pci_dev->device, name, + sizeof(struct idpf_vport), + NULL, NULL, idpf_dev_init, + adapter); + if (retval != 0) + PMD_DRV_LOG(ERR, "Failed to create vport %d", + adapter->req_vports[i]); + } + } + + return 0; + +err: + if (first_probe) { + rte_spinlock_lock(&idpf_adapter_lock); + TAILQ_REMOVE(&idpf_adapter_list, adapter, next); + rte_spinlock_unlock(&idpf_adapter_lock); + idpf_adapter_rel(adapter); + rte_free(adapter); + } + return retval; +} + +static int +idpf_pci_remove(struct rte_pci_device *pci_dev) +{ + struct idpf_adapter *adapter = idpf_find_adapter(pci_dev); + uint16_t port_id; + + /* Ethdev created can be found RTE_ETH_FOREACH_DEV_OF through rte_device */ + RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device) { + rte_eth_dev_close(port_id); + } + + rte_spinlock_lock(&idpf_adapter_lock); + TAILQ_REMOVE(&idpf_adapter_list, adapter, next); + rte_spinlock_unlock(&idpf_adapter_lock); + idpf_adapter_rel(adapter); + rte_free(adapter); + + return 0; +} + +static struct rte_pci_driver rte_idpf_pmd = { + .id_table = pci_id_idpf_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING, + .probe = idpf_pci_probe, + .remove = idpf_pci_remove, +}; + +/** + * Driver initialization routine. + * Invoked once at EAL init time. + * Register itself as the [Poll Mode] Driver of PCI devices. + */ +RTE_PMD_REGISTER_PCI(net_idpf, rte_idpf_pmd); +RTE_PMD_REGISTER_PCI_TABLE(net_idpf, pci_id_idpf_map); +RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic | vfio-pci"); + +RTE_LOG_REGISTER_SUFFIX(idpf_logtype_init, init, NOTICE); +RTE_LOG_REGISTER_SUFFIX(idpf_logtype_driver, driver, NOTICE); diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h new file mode 100644 index 0000000000..84ae6641e2 --- /dev/null +++ b/drivers/net/idpf/idpf_ethdev.h @@ -0,0 +1,189 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2022 Intel Corporation + */ + +#ifndef _IDPF_ETHDEV_H_ +#define _IDPF_ETHDEV_H_ + +#include +#include +#include +#include +#include +#include +#include + +#include "idpf_logs.h" + +#include +#include + +#define IDPF_MAX_VPORT_NUM 8 + +#define IDPF_DEFAULT_RXQ_NUM 16 +#define IDPF_DEFAULT_TXQ_NUM 16 + +#define IDPF_INVALID_VPORT_IDX 0xffff +#define IDPF_TX_COMPLQ_PER_GRP 1 +#define IDPF_RX_BUFQ_PER_GRP 2 + +#define IDPF_CTLQ_ID -1 +#define IDPF_CTLQ_LEN 64 +#define IDPF_DFLT_MBX_BUF_SIZE 4096 + +#define IDPF_MIN_BUF_SIZE 1024 +#define IDPF_MAX_FRAME_SIZE 9728 + +#define IDPF_NUM_MACADDR_MAX 64 + +#define IDPF_VLAN_TAG_SIZE 4 +#define IDPF_ETH_OVERHEAD \ + (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2) + +#define IDPF_ADAPTER_NAME_LEN (PCI_PRI_STR_SIZE + 1) + +/* Message type read in virtual channel from PF */ +enum idpf_vc_result { + IDPF_MSG_ERR = -1, /* Meet error when accessing admin queue */ + IDPF_MSG_NON, /* Read nothing from admin queue */ + IDPF_MSG_SYS, /* Read system msg from admin queue */ + IDPF_MSG_CMD, /* Read async command result */ +}; + +struct idpf_chunks_info { + uint32_t tx_start_qid; + uint32_t rx_start_qid; + /* Valid only if split queue model */ + uint32_t tx_compl_start_qid; + uint32_t rx_buf_start_qid; + + uint64_t tx_qtail_start; + uint32_t tx_qtail_spacing; + uint64_t rx_qtail_start; + uint32_t rx_qtail_spacing; + uint64_t tx_compl_qtail_start; + uint32_t tx_compl_qtail_spacing; + uint64_t rx_buf_qtail_start; + uint32_t rx_buf_qtail_spacing; +}; + +struct idpf_vport { + struct idpf_adapter *adapter; /* Backreference to associated adapter */ + uint16_t vport_id; + uint32_t txq_model; + uint32_t rxq_model; + uint16_t num_tx_q; + /* valid only if txq_model is split Q */ + uint16_t num_tx_complq; + uint16_t num_rx_q; + /* valid only if rxq_model is split Q */ + uint16_t num_rx_bufq; + + uint16_t max_mtu; + uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS]; + + uint16_t sw_idx; /* SW idx */ + + struct rte_eth_dev_data *dev_data; /* Pointer to the device data */ + uint16_t max_pkt_len; /* Maximum packet length */ + + /* Chunk info */ + struct idpf_chunks_info chunks_info; + + uint16_t devarg_id; +}; + +struct idpf_adapter { + TAILQ_ENTRY(idpf_adapter) next; + struct idpf_hw hw; + char name[IDPF_ADAPTER_NAME_LEN]; + + struct virtchnl2_version_info virtchnl_version; + struct virtchnl2_get_capabilities *caps; + + volatile enum virtchnl_ops pend_cmd; /* pending command not finished */ + uint32_t cmd_retval; /* return value of the cmd response from ipf */ + uint8_t *mbx_resp; /* buffer to store the mailbox response from ipf */ + + uint32_t txq_model; /* 0 - split queue model, non-0 - single queue model */ + uint32_t rxq_model; /* 0 - split queue model, non-0 - single queue model */ + + /* Vport info */ + uint8_t **vport_req_info; + uint8_t **vport_recv_info; + struct idpf_vport **vports; + uint16_t max_vport_nb; + uint16_t req_vports[IDPF_MAX_VPORT_NUM]; + uint16_t req_vport_nb; + uint16_t cur_vports; + uint16_t cur_vport_nb; + uint16_t cur_vport_idx; + + /* Max config queue number per VC message */ + uint32_t max_rxq_per_msg; + uint32_t max_txq_per_msg; +}; + +TAILQ_HEAD(idpf_adapter_list, idpf_adapter); + +#define IDPF_DEV_TO_PCI(eth_dev) \ + RTE_DEV_TO_PCI((eth_dev)->device) + +/* structure used for sending and checking response of virtchnl ops */ +struct idpf_cmd_info { + uint32_t ops; + uint8_t *in_args; /* buffer for sending */ + uint32_t in_args_size; /* buffer size for sending */ + uint8_t *out_buffer; /* buffer for response */ + uint32_t out_size; /* buffer size for response */ +}; + +/* notify current command done. Only call in case execute + * _atomic_set_cmd successfully. + */ +static inline void +notify_cmd(struct idpf_adapter *adapter, int msg_ret) +{ + adapter->cmd_retval = msg_ret; + /* Return value may be checked in anither thread, need to ensure the coherence. */ + rte_wmb(); + adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN; +} + +/* clear current command. Only call in case execute + * _atomic_set_cmd successfully. + */ +static inline void +clear_cmd(struct idpf_adapter *adapter) +{ + /* Return value may be checked in anither thread, need to ensure the coherence. */ + rte_wmb(); + adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN; + adapter->cmd_retval = VIRTCHNL_STATUS_SUCCESS; +} + +/* Check there is pending cmd in execution. If none, set new command. */ +static inline bool +atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops) +{ + enum virtchnl_ops op_unk = VIRTCHNL_OP_UNKNOWN; + bool ret = __atomic_compare_exchange(&adapter->pend_cmd, &op_unk, &ops, + 0, __ATOMIC_ACQUIRE, __ATOMIC_ACQUIRE); + + if (!ret) + PMD_DRV_LOG(ERR, "There is incomplete cmd %d", adapter->pend_cmd); + + return !ret; +} + +struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev); +void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev); +int idpf_vc_check_api_version(struct idpf_adapter *adapter); +int idpf_vc_get_caps(struct idpf_adapter *adapter); +int idpf_vc_create_vport(struct idpf_adapter *adapter); +int idpf_vc_destroy_vport(struct idpf_vport *vport); +int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable); +int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops, + uint16_t buf_len, uint8_t *buf); + +#endif /* _IDPF_ETHDEV_H_ */ diff --git a/drivers/net/idpf/idpf_logs.h b/drivers/net/idpf/idpf_logs.h new file mode 100644 index 0000000000..d5f778fefe --- /dev/null +++ b/drivers/net/idpf/idpf_logs.h @@ -0,0 +1,56 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2022 Intel Corporation + */ + +#ifndef _IDPF_LOGS_H_ +#define _IDPF_LOGS_H_ + +#include + +extern int idpf_logtype_init; +extern int idpf_logtype_driver; + +#define PMD_INIT_LOG(level, ...) \ + rte_log(RTE_LOG_ ## level, \ + idpf_logtype_init, \ + RTE_FMT("%s(): " \ + RTE_FMT_HEAD(__VA_ARGS__,) "\n", \ + __func__, \ + RTE_FMT_TAIL(__VA_ARGS__,))) + +#define PMD_DRV_LOG_RAW(level, ...) \ + rte_log(RTE_LOG_ ## level, \ + idpf_logtype_driver, \ + RTE_FMT("%s(): " \ + RTE_FMT_HEAD(__VA_ARGS__,) "\n", \ + __func__, \ + RTE_FMT_TAIL(__VA_ARGS__,))) + +#define PMD_DRV_LOG(level, fmt, args...) \ + PMD_DRV_LOG_RAW(level, fmt "\n", ## args) + +#ifdef RTE_LIBRTE_IDPF_DEBUG_RX +#define PMD_RX_LOG(level, ...) \ + RTE_LOG(level, \ + PMD, \ + RTE_FMT("%s(): " \ + RTE_FMT_HEAD(__VA_ARGS__,) "\n", \ + __func__, \ + RTE_FMT_TAIL(__VA_ARGS__,))) +#else +#define PMD_RX_LOG(level, fmt, args...) do { } while (0) +#endif + +#ifdef RTE_LIBRTE_IDPF_DEBUG_TX +#define PMD_TX_LOG(level, ...) \ + RTE_LOG(level, \ + PMD, \ + RTE_FMT("%s(): " \ + RTE_FMT_HEAD(__VA_ARGS__,) "\n", \ + __func__, \ + RTE_FMT_TAIL(__VA_ARGS__,))) +#else +#define PMD_TX_LOG(level, fmt, args...) do { } while (0) +#endif + +#endif /* _IDPF_LOGS_H_ */ diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c new file mode 100644 index 0000000000..e89c2a2783 --- /dev/null +++ b/drivers/net/idpf/idpf_vchnl.c @@ -0,0 +1,416 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2022 Intel Corporation + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include + +#include "idpf_ethdev.h" + +static int +idpf_vc_clean(struct idpf_adapter *adapter) +{ + struct idpf_ctlq_msg *q_msg[IDPF_CTLQ_LEN]; + uint16_t num_q_msg = IDPF_CTLQ_LEN; + struct idpf_dma_mem *dma_mem; + int err; + uint32_t i; + + for (i = 0; i < 10; i++) { + err = idpf_ctlq_clean_sq(adapter->hw.asq, &num_q_msg, q_msg); + msleep(20); + if (num_q_msg > 0) + break; + } + if (err != 0) + return err; + + /* Empty queue is not an error */ + for (i = 0; i < num_q_msg; i++) { + dma_mem = q_msg[i]->ctx.indirect.payload; + if (dma_mem != NULL) { + idpf_free_dma_mem(&adapter->hw, dma_mem); + rte_free(dma_mem); + } + rte_free(q_msg[i]); + } + + return 0; +} + +static int +idpf_send_vc_msg(struct idpf_adapter *adapter, enum virtchnl_ops op, + uint16_t msg_size, uint8_t *msg) +{ + struct idpf_ctlq_msg *ctlq_msg; + struct idpf_dma_mem *dma_mem; + int err; + + err = idpf_vc_clean(adapter); + if (err != 0) + goto err; + + ctlq_msg = rte_zmalloc(NULL, sizeof(struct idpf_ctlq_msg), 0); + if (ctlq_msg == NULL) { + err = -ENOMEM; + goto err; + } + + dma_mem = rte_zmalloc(NULL, sizeof(struct idpf_dma_mem), 0); + if (dma_mem == NULL) { + err = -ENOMEM; + goto dma_mem_error; + } + + dma_mem->size = IDPF_DFLT_MBX_BUF_SIZE; + idpf_alloc_dma_mem(&adapter->hw, dma_mem, dma_mem->size); + if (dma_mem->va == NULL) { + err = -ENOMEM; + goto dma_alloc_error; + } + + memcpy(dma_mem->va, msg, msg_size); + + ctlq_msg->opcode = idpf_mbq_opc_send_msg_to_pf; + ctlq_msg->func_id = 0; + ctlq_msg->data_len = msg_size; + ctlq_msg->cookie.mbx.chnl_opcode = op; + ctlq_msg->cookie.mbx.chnl_retval = VIRTCHNL_STATUS_SUCCESS; + ctlq_msg->ctx.indirect.payload = dma_mem; + + err = idpf_ctlq_send(&adapter->hw, adapter->hw.asq, 1, ctlq_msg); + if (err != 0) + goto send_error; + + return 0; + +send_error: + idpf_free_dma_mem(&adapter->hw, dma_mem); +dma_alloc_error: + rte_free(dma_mem); +dma_mem_error: + rte_free(ctlq_msg); +err: + return err; +} + +static enum idpf_vc_result +idpf_read_msg_from_cp(struct idpf_adapter *adapter, uint16_t buf_len, + uint8_t *buf) +{ + struct idpf_hw *hw = &adapter->hw; + struct idpf_ctlq_msg ctlq_msg; + struct idpf_dma_mem *dma_mem = NULL; + enum idpf_vc_result result = IDPF_MSG_NON; + enum virtchnl_ops opcode; + uint16_t pending = 1; + int ret; + + ret = idpf_ctlq_recv(hw->arq, &pending, &ctlq_msg); + if (ret != 0) { + PMD_DRV_LOG(DEBUG, "Can't read msg from AQ"); + if (ret != -ENOMSG) + result = IDPF_MSG_ERR; + return result; + } + + rte_memcpy(buf, ctlq_msg.ctx.indirect.payload->va, buf_len); + + opcode = (enum virtchnl_ops)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_opcode); + adapter->cmd_retval = + (enum virtchnl_status_code)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_retval); + + PMD_DRV_LOG(DEBUG, "CQ from CP carries opcode %u, retval %d", + opcode, adapter->cmd_retval); + + if (opcode == VIRTCHNL2_OP_EVENT) { + struct virtchnl2_event *ve = + (struct virtchnl2_event *)ctlq_msg.ctx.indirect.payload->va; + + result = IDPF_MSG_SYS; + switch (ve->event) { + case VIRTCHNL2_EVENT_LINK_CHANGE: + /* TBD */ + break; + default: + PMD_DRV_LOG(ERR, "%s: Unknown event %d from CP", + __func__, ve->event); + break; + } + } else { + /* async reply msg on command issued by pf previously */ + result = IDPF_MSG_CMD; + if (opcode != adapter->pend_cmd) { + PMD_DRV_LOG(WARNING, "command mismatch, expect %u, get %u", + adapter->pend_cmd, opcode); + result = IDPF_MSG_ERR; + } + } + + if (ctlq_msg.data_len != 0) + dma_mem = ctlq_msg.ctx.indirect.payload; + else + pending = 0; + + ret = idpf_ctlq_post_rx_buffs(hw, hw->arq, &pending, &dma_mem); + if (ret != 0 && dma_mem != NULL) + idpf_free_dma_mem(hw, dma_mem); + + return result; +} + +#define MAX_TRY_TIMES 200 +#define ASQ_DELAY_MS 10 + +int +idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops, uint16_t buf_len, + uint8_t *buf) +{ + int err = 0; + int i = 0; + int ret; + + do { + ret = idpf_read_msg_from_cp(adapter, buf_len, buf); + if (ret == IDPF_MSG_CMD) + break; + rte_delay_ms(ASQ_DELAY_MS); + } while (i++ < MAX_TRY_TIMES); + if (i >= MAX_TRY_TIMES || + adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) { + err = -EBUSY; + PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d", + adapter->cmd_retval, ops); + } + + return err; +} + +static int +idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args) +{ + int err = 0; + int i = 0; + int ret; + + if (atomic_set_cmd(adapter, args->ops)) + return -EINVAL; + + ret = idpf_send_vc_msg(adapter, args->ops, args->in_args_size, args->in_args); + if (ret != 0) { + PMD_DRV_LOG(ERR, "fail to send cmd %d", args->ops); + clear_cmd(adapter); + return ret; + } + + switch (args->ops) { + case VIRTCHNL_OP_VERSION: + case VIRTCHNL2_OP_GET_CAPS: + case VIRTCHNL2_OP_CREATE_VPORT: + case VIRTCHNL2_OP_DESTROY_VPORT: + case VIRTCHNL2_OP_ENABLE_VPORT: + case VIRTCHNL2_OP_DISABLE_VPORT: + /* for init virtchnl ops, need to poll the response */ + err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer); + clear_cmd(adapter); + break; + default: + /* For other virtchnl ops in running time, + * wait for the cmd done flag. + */ + do { + if (adapter->pend_cmd == VIRTCHNL_OP_UNKNOWN) + break; + rte_delay_ms(ASQ_DELAY_MS); + /* If don't read msg or read sys event, continue */ + } while (i++ < MAX_TRY_TIMES); + /* If there's no response is received, clear command */ + if (i >= MAX_TRY_TIMES || + adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) { + err = -EBUSY; + PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d", + adapter->cmd_retval, args->ops); + clear_cmd(adapter); + } + break; + } + + return err; +} + +int +idpf_vc_check_api_version(struct idpf_adapter *adapter) +{ + struct virtchnl2_version_info version, *pver; + struct idpf_cmd_info args; + int err; + + memset(&version, 0, sizeof(struct virtchnl_version_info)); + version.major = VIRTCHNL2_VERSION_MAJOR_2; + version.minor = VIRTCHNL2_VERSION_MINOR_0; + + args.ops = VIRTCHNL_OP_VERSION; + args.in_args = (uint8_t *)&version; + args.in_args_size = sizeof(version); + args.out_buffer = adapter->mbx_resp; + args.out_size = IDPF_DFLT_MBX_BUF_SIZE; + + err = idpf_execute_vc_cmd(adapter, &args); + if (err != 0) { + PMD_DRV_LOG(ERR, + "Failed to execute command of VIRTCHNL_OP_VERSION"); + return err; + } + + pver = (struct virtchnl2_version_info *)args.out_buffer; + adapter->virtchnl_version = *pver; + + if (adapter->virtchnl_version.major != VIRTCHNL2_VERSION_MAJOR_2 || + adapter->virtchnl_version.minor != VIRTCHNL2_VERSION_MINOR_0) { + PMD_INIT_LOG(ERR, "VIRTCHNL API version mismatch:(%u.%u)-(%u.%u)", + adapter->virtchnl_version.major, + adapter->virtchnl_version.minor, + VIRTCHNL2_VERSION_MAJOR_2, + VIRTCHNL2_VERSION_MINOR_0); + return -EINVAL; + } + + return 0; +} + +int +idpf_vc_get_caps(struct idpf_adapter *adapter) +{ + struct virtchnl2_get_capabilities caps_msg; + struct idpf_cmd_info args; + int err; + + memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities)); + + args.ops = VIRTCHNL2_OP_GET_CAPS; + args.in_args = (uint8_t *)&caps_msg; + args.in_args_size = sizeof(caps_msg); + args.out_buffer = adapter->mbx_resp; + args.out_size = IDPF_DFLT_MBX_BUF_SIZE; + + err = idpf_execute_vc_cmd(adapter, &args); + if (err != 0) { + PMD_DRV_LOG(ERR, + "Failed to execute command of VIRTCHNL2_OP_GET_CAPS"); + return err; + } + + rte_memcpy(adapter->caps, args.out_buffer, sizeof(caps_msg)); + + return 0; +} + +int +idpf_vc_create_vport(struct idpf_adapter *adapter) +{ + uint16_t idx = adapter->cur_vport_idx; + struct virtchnl2_create_vport *vport_req_info = + (struct virtchnl2_create_vport *)adapter->vport_req_info[idx]; + struct virtchnl2_create_vport vport_msg; + struct idpf_cmd_info args; + int err = -1; + + memset(&vport_msg, 0, sizeof(struct virtchnl2_create_vport)); + vport_msg.vport_type = vport_req_info->vport_type; + vport_msg.txq_model = vport_req_info->txq_model; + vport_msg.rxq_model = vport_req_info->rxq_model; + vport_msg.num_tx_q = vport_req_info->num_tx_q; + vport_msg.num_tx_complq = vport_req_info->num_tx_complq; + vport_msg.num_rx_q = vport_req_info->num_rx_q; + vport_msg.num_rx_bufq = vport_req_info->num_rx_bufq; + + memset(&args, 0, sizeof(args)); + args.ops = VIRTCHNL2_OP_CREATE_VPORT; + args.in_args = (uint8_t *)&vport_msg; + args.in_args_size = sizeof(vport_msg); + args.out_buffer = adapter->mbx_resp; + args.out_size = IDPF_DFLT_MBX_BUF_SIZE; + + err = idpf_execute_vc_cmd(adapter, &args); + if (err != 0) { + PMD_DRV_LOG(ERR, + "Failed to execute command of VIRTCHNL2_OP_CREATE_VPORT"); + return err; + } + + if (adapter->vport_recv_info[idx] == NULL) { + adapter->vport_recv_info[idx] = rte_zmalloc(NULL, + IDPF_DFLT_MBX_BUF_SIZE, 0); + if (adapter->vport_recv_info[idx] == NULL) { + PMD_INIT_LOG(ERR, "Failed to alloc vport_recv_info."); + return -ENOMEM; + } + } + rte_memcpy(adapter->vport_recv_info[idx], args.out_buffer, + IDPF_DFLT_MBX_BUF_SIZE); + return 0; +} + +int +idpf_vc_destroy_vport(struct idpf_vport *vport) +{ + struct idpf_adapter *adapter = vport->adapter; + struct virtchnl2_vport vc_vport; + struct idpf_cmd_info args; + int err; + + vc_vport.vport_id = vport->vport_id; + + memset(&args, 0, sizeof(args)); + args.ops = VIRTCHNL2_OP_DESTROY_VPORT; + args.in_args = (uint8_t *)&vc_vport; + args.in_args_size = sizeof(vc_vport); + args.out_buffer = adapter->mbx_resp; + args.out_size = IDPF_DFLT_MBX_BUF_SIZE; + + err = idpf_execute_vc_cmd(adapter, &args); + if (err != 0) + PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_DESTROY_VPORT"); + + return err; +} + +int +idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable) +{ + struct idpf_adapter *adapter = vport->adapter; + struct virtchnl2_vport vc_vport; + struct idpf_cmd_info args; + int err; + + vc_vport.vport_id = vport->vport_id; + args.ops = enable ? VIRTCHNL2_OP_ENABLE_VPORT : + VIRTCHNL2_OP_DISABLE_VPORT; + args.in_args = (uint8_t *)&vc_vport; + args.in_args_size = sizeof(vc_vport); + args.out_buffer = adapter->mbx_resp; + args.out_size = IDPF_DFLT_MBX_BUF_SIZE; + + err = idpf_execute_vc_cmd(adapter, &args); + if (err != 0) { + PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_VPORT", + enable ? "ENABLE" : "DISABLE"); + } + + return err; +} diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build new file mode 100644 index 0000000000..ecf73355c3 --- /dev/null +++ b/drivers/net/idpf/meson.build @@ -0,0 +1,15 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2022 Intel Corporation + +if is_windows + build = false + reason = 'not supported on Windows' + subdir_done() +endif + +deps += ['common_idpf'] + +sources = files( + 'idpf_ethdev.c', + 'idpf_vchnl.c', +) diff --git a/drivers/net/idpf/version.map b/drivers/net/idpf/version.map new file mode 100644 index 0000000000..78c3585d7c --- /dev/null +++ b/drivers/net/idpf/version.map @@ -0,0 +1,3 @@ +DPDK_23 { + local: *; +}; diff --git a/drivers/net/meson.build b/drivers/net/meson.build index 589399400a..6470bf3636 100644 --- a/drivers/net/meson.build +++ b/drivers/net/meson.build @@ -29,6 +29,7 @@ drivers = [ 'i40e', 'iavf', 'ice', + 'idpf', 'igc', 'ionic', 'ipn3ke', From patchwork Mon Oct 31 08:33:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xing, Beilei" X-Patchwork-Id: 119343 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 244C7A00C2; Mon, 31 Oct 2022 10:04:37 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 653C540DF7; Mon, 31 Oct 2022 10:04:24 +0100 (CET) Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id C73D040695 for ; Mon, 31 Oct 2022 10:04:17 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667207058; x=1698743058; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=JtJChglb7ZQh5DYYyPAikSdTxyAHIJ4ZdFhu6Z9wMA4=; b=CRYDtvN4vFfXJnsJGrlwmfiRxCMrqR1z/GmyhIgKFGt0fHE/HTrnZfat UnoI4Gn9l4yPBmsMAKo+27PYTaUPkrq8Rhu6i5oeTT+VI/odwxvY8y6Ld v5COuKMTiuRd6xZvy55b8ZSIC/Xuhhu2+HZ1QhW7yny5ht2hc/CoH3wm4 Z+Z1qVgpM9LGhvSGABZM3E9RSIfe6qeaif5Iw53jQ0jFRfOl7pLyizOkM 3prUrpgdGgWdnbSv9SYSlkrbYuaQ5SdZn1KmgLVv3Gr+z6bxgy0kBIi/c Jt67L6ZhS/LOEu/4NTWUCgrPcrBGSgsuQ+EiqiFVD9dndqhS74x7eRYXi A==; X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="370926272" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="370926272" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Oct 2022 02:04:04 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="878664790" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="878664790" Received: from dpdk-beileix-3.sh.intel.com ([10.67.110.253]) by fmsmga006.fm.intel.com with ESMTP; 31 Oct 2022 02:04:02 -0700 From: beilei.xing@intel.com To: jingjing.wu@intel.com, beilei.xing@intel.com Cc: dev@dpdk.org, Junfeng Guo , Xiaoyun Li Subject: [PATCH v18 03/18] net/idpf: add Tx queue setup Date: Mon, 31 Oct 2022 08:33:31 +0000 Message-Id: <20221031083346.16558-4-beilei.xing@intel.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20221031083346.16558-1-beilei.xing@intel.com> References: <20221031051556.98549-1-beilei.xing@intel.com> <20221031083346.16558-1-beilei.xing@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Junfeng Guo Add support for tx_queue_setup ops. In the single queue model, the same descriptor queue is used by SW to post buffer descriptors to HW and by HW to post completed descriptors to SW. In the split queue model, "RX buffer queues" are used to pass descriptor buffers from SW to HW while Rx queues are used only to pass the descriptor completions, that is, descriptors that point to completed buffers, from HW to SW. This is contrary to the single queue model in which Rx queues are used for both purposes. Signed-off-by: Beilei Xing Signed-off-by: Xiaoyun Li Signed-off-by: Junfeng Guo --- drivers/net/idpf/idpf_ethdev.c | 13 ++ drivers/net/idpf/idpf_rxtx.c | 364 +++++++++++++++++++++++++++++++++ drivers/net/idpf/idpf_rxtx.h | 70 +++++++ drivers/net/idpf/meson.build | 1 + 4 files changed, 448 insertions(+) create mode 100644 drivers/net/idpf/idpf_rxtx.c create mode 100644 drivers/net/idpf/idpf_rxtx.h diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c index 035f563275..54f20d30ca 100644 --- a/drivers/net/idpf/idpf_ethdev.c +++ b/drivers/net/idpf/idpf_ethdev.c @@ -11,6 +11,7 @@ #include #include "idpf_ethdev.h" +#include "idpf_rxtx.h" #define IDPF_TX_SINGLE_Q "tx_single" #define IDPF_RX_SINGLE_Q "rx_single" @@ -42,6 +43,17 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD; dev_info->min_mtu = RTE_ETHER_MIN_MTU; + dev_info->default_txconf = (struct rte_eth_txconf) { + .tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH, + .tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH, + }; + + dev_info->tx_desc_lim = (struct rte_eth_desc_lim) { + .nb_max = IDPF_MAX_RING_DESC, + .nb_min = IDPF_MIN_RING_DESC, + .nb_align = IDPF_ALIGN_RING_DESC, + }; + return 0; } @@ -631,6 +643,7 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter) static const struct eth_dev_ops idpf_eth_dev_ops = { .dev_configure = idpf_dev_configure, .dev_close = idpf_dev_close, + .tx_queue_setup = idpf_tx_queue_setup, .dev_infos_get = idpf_dev_info_get, }; diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c new file mode 100644 index 0000000000..4afa0a2560 --- /dev/null +++ b/drivers/net/idpf/idpf_rxtx.c @@ -0,0 +1,364 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2022 Intel Corporation + */ + +#include +#include + +#include "idpf_ethdev.h" +#include "idpf_rxtx.h" + +static int +check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh, + uint16_t tx_free_thresh) +{ + /* TX descriptors will have their RS bit set after tx_rs_thresh + * descriptors have been used. The TX descriptor ring will be cleaned + * after tx_free_thresh descriptors are used or if the number of + * descriptors required to transmit a packet is greater than the + * number of free TX descriptors. + * + * The following constraints must be satisfied: + * - tx_rs_thresh must be less than the size of the ring minus 2. + * - tx_free_thresh must be less than the size of the ring minus 3. + * - tx_rs_thresh must be less than or equal to tx_free_thresh. + * - tx_rs_thresh must be a divisor of the ring size. + * + * One descriptor in the TX ring is used as a sentinel to avoid a H/W + * race condition, hence the maximum threshold constraints. When set + * to zero use default values. + */ + if (tx_rs_thresh >= (nb_desc - 2)) { + PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than the " + "number of TX descriptors (%u) minus 2", + tx_rs_thresh, nb_desc); + return -EINVAL; + } + if (tx_free_thresh >= (nb_desc - 3)) { + PMD_INIT_LOG(ERR, "tx_free_thresh (%u) must be less than the " + "number of TX descriptors (%u) minus 3.", + tx_free_thresh, nb_desc); + return -EINVAL; + } + if (tx_rs_thresh > tx_free_thresh) { + PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than or " + "equal to tx_free_thresh (%u).", + tx_rs_thresh, tx_free_thresh); + return -EINVAL; + } + if ((nb_desc % tx_rs_thresh) != 0) { + PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be a divisor of the " + "number of TX descriptors (%u).", + tx_rs_thresh, nb_desc); + return -EINVAL; + } + + return 0; +} + +static void +reset_split_tx_descq(struct idpf_tx_queue *txq) +{ + struct idpf_tx_entry *txe; + uint32_t i, size; + uint16_t prev; + + if (txq == NULL) { + PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL"); + return; + } + + size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc; + for (i = 0; i < size; i++) + ((volatile char *)txq->desc_ring)[i] = 0; + + txe = txq->sw_ring; + prev = (uint16_t)(txq->sw_nb_desc - 1); + for (i = 0; i < txq->sw_nb_desc; i++) { + txe[i].mbuf = NULL; + txe[i].last_id = i; + txe[prev].next_id = i; + prev = i; + } + + txq->tx_tail = 0; + txq->nb_used = 0; + + /* Use this as next to clean for split desc queue */ + txq->last_desc_cleaned = 0; + txq->sw_tail = 0; + txq->nb_free = txq->nb_tx_desc - 1; +} + +static void +reset_split_tx_complq(struct idpf_tx_queue *cq) +{ + uint32_t i, size; + + if (cq == NULL) { + PMD_DRV_LOG(DEBUG, "Pointer to complq is NULL"); + return; + } + + size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc; + for (i = 0; i < size; i++) + ((volatile char *)cq->compl_ring)[i] = 0; + + cq->tx_tail = 0; + cq->expected_gen_id = 1; +} + +static void +reset_single_tx_queue(struct idpf_tx_queue *txq) +{ + struct idpf_tx_entry *txe; + uint32_t i, size; + uint16_t prev; + + if (txq == NULL) { + PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL"); + return; + } + + txe = txq->sw_ring; + size = sizeof(struct idpf_flex_tx_desc) * txq->nb_tx_desc; + for (i = 0; i < size; i++) + ((volatile char *)txq->tx_ring)[i] = 0; + + prev = (uint16_t)(txq->nb_tx_desc - 1); + for (i = 0; i < txq->nb_tx_desc; i++) { + txq->tx_ring[i].qw1.cmd_dtype = + rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE); + txe[i].mbuf = NULL; + txe[i].last_id = i; + txe[prev].next_id = i; + prev = i; + } + + txq->tx_tail = 0; + txq->nb_used = 0; + + txq->last_desc_cleaned = txq->nb_tx_desc - 1; + txq->nb_free = txq->nb_tx_desc - 1; + + txq->next_dd = txq->rs_thresh - 1; + txq->next_rs = txq->rs_thresh - 1; +} + +static int +idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, + uint16_t nb_desc, unsigned int socket_id, + const struct rte_eth_txconf *tx_conf) +{ + struct idpf_vport *vport = dev->data->dev_private; + struct idpf_adapter *adapter = vport->adapter; + uint16_t tx_rs_thresh, tx_free_thresh; + struct idpf_hw *hw = &adapter->hw; + struct idpf_tx_queue *txq, *cq; + const struct rte_memzone *mz; + uint32_t ring_size; + uint64_t offloads; + int ret; + + offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads; + + tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh != 0) ? + tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH); + tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh != 0) ? + tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH); + if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0) + return -EINVAL; + + /* Allocate the TX queue data structure. */ + txq = rte_zmalloc_socket("idpf split txq", + sizeof(struct idpf_tx_queue), + RTE_CACHE_LINE_SIZE, + socket_id); + if (txq == NULL) { + PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure"); + return -ENOMEM; + } + + txq->nb_tx_desc = nb_desc; + txq->rs_thresh = tx_rs_thresh; + txq->free_thresh = tx_free_thresh; + txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx; + txq->port_id = dev->data->port_id; + txq->offloads = offloads; + + /* Allocate software ring */ + txq->sw_nb_desc = 2 * nb_desc; + txq->sw_ring = + rte_zmalloc_socket("idpf split tx sw ring", + sizeof(struct idpf_tx_entry) * + txq->sw_nb_desc, + RTE_CACHE_LINE_SIZE, + socket_id); + if (txq->sw_ring == NULL) { + PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring"); + ret = -ENOMEM; + goto err_txq_sw_ring; + } + + /* Allocate TX hardware ring descriptors. */ + ring_size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc; + ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN); + mz = rte_eth_dma_zone_reserve(dev, "split_tx_ring", queue_idx, + ring_size, IDPF_RING_BASE_ALIGN, + socket_id); + if (mz == NULL) { + PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX"); + ret = -ENOMEM; + goto err_txq_mz; + } + txq->tx_ring_phys_addr = mz->iova; + txq->desc_ring = mz->addr; + + txq->mz = mz; + reset_split_tx_descq(txq); + txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start + + queue_idx * vport->chunks_info.tx_qtail_spacing); + + /* Allocate the TX completion queue data structure. */ + txq->complq = rte_zmalloc_socket("idpf splitq cq", + sizeof(struct idpf_tx_queue), + RTE_CACHE_LINE_SIZE, + socket_id); + cq = txq->complq; + if (cq == NULL) { + PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure"); + ret = -ENOMEM; + goto err_cq; + } + cq->nb_tx_desc = 2 * nb_desc; + cq->queue_id = vport->chunks_info.tx_compl_start_qid + queue_idx; + cq->port_id = dev->data->port_id; + cq->txqs = dev->data->tx_queues; + cq->tx_start_qid = vport->chunks_info.tx_start_qid; + + ring_size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc; + ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN); + mz = rte_eth_dma_zone_reserve(dev, "tx_split_compl_ring", queue_idx, + ring_size, IDPF_RING_BASE_ALIGN, + socket_id); + if (mz == NULL) { + PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX completion queue"); + ret = -ENOMEM; + goto err_cq_mz; + } + cq->tx_ring_phys_addr = mz->iova; + cq->compl_ring = mz->addr; + cq->mz = mz; + reset_split_tx_complq(cq); + + txq->q_set = true; + dev->data->tx_queues[queue_idx] = txq; + + return 0; + +err_cq_mz: + rte_free(cq); +err_cq: + rte_memzone_free(txq->mz); +err_txq_mz: + rte_free(txq->sw_ring); +err_txq_sw_ring: + rte_free(txq); + + return ret; +} + +static int +idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, + uint16_t nb_desc, unsigned int socket_id, + const struct rte_eth_txconf *tx_conf) +{ + struct idpf_vport *vport = dev->data->dev_private; + struct idpf_adapter *adapter = vport->adapter; + uint16_t tx_rs_thresh, tx_free_thresh; + struct idpf_hw *hw = &adapter->hw; + const struct rte_memzone *mz; + struct idpf_tx_queue *txq; + uint32_t ring_size; + uint64_t offloads; + + offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads; + + tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh > 0) ? + tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH); + tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh > 0) ? + tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH); + if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0) + return -EINVAL; + + /* Allocate the TX queue data structure. */ + txq = rte_zmalloc_socket("idpf txq", + sizeof(struct idpf_tx_queue), + RTE_CACHE_LINE_SIZE, + socket_id); + if (txq == NULL) { + PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure"); + return -ENOMEM; + } + + /* TODO: vlan offload */ + + txq->nb_tx_desc = nb_desc; + txq->rs_thresh = tx_rs_thresh; + txq->free_thresh = tx_free_thresh; + txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx; + txq->port_id = dev->data->port_id; + txq->offloads = offloads; + + /* Allocate software ring */ + txq->sw_ring = + rte_zmalloc_socket("idpf tx sw ring", + sizeof(struct idpf_tx_entry) * nb_desc, + RTE_CACHE_LINE_SIZE, + socket_id); + if (txq->sw_ring == NULL) { + PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring"); + rte_free(txq); + return -ENOMEM; + } + + /* Allocate TX hardware ring descriptors. */ + ring_size = sizeof(struct idpf_flex_tx_desc) * nb_desc; + ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN); + mz = rte_eth_dma_zone_reserve(dev, "tx_ring", queue_idx, + ring_size, IDPF_RING_BASE_ALIGN, + socket_id); + if (mz == NULL) { + PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX"); + rte_free(txq->sw_ring); + rte_free(txq); + return -ENOMEM; + } + + txq->tx_ring_phys_addr = mz->iova; + txq->tx_ring = mz->addr; + + txq->mz = mz; + reset_single_tx_queue(txq); + txq->q_set = true; + dev->data->tx_queues[queue_idx] = txq; + txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start + + queue_idx * vport->chunks_info.tx_qtail_spacing); + + return 0; +} + +int +idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, + uint16_t nb_desc, unsigned int socket_id, + const struct rte_eth_txconf *tx_conf) +{ + struct idpf_vport *vport = dev->data->dev_private; + + if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) + return idpf_tx_single_queue_setup(dev, queue_idx, nb_desc, + socket_id, tx_conf); + else + return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc, + socket_id, tx_conf); +} diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h new file mode 100644 index 0000000000..c7ba15b058 --- /dev/null +++ b/drivers/net/idpf/idpf_rxtx.h @@ -0,0 +1,70 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2022 Intel Corporation + */ + +#ifndef _IDPF_RXTX_H_ +#define _IDPF_RXTX_H_ + +#include "idpf_ethdev.h" + +/* In QLEN must be whole number of 32 descriptors. */ +#define IDPF_ALIGN_RING_DESC 32 +#define IDPF_MIN_RING_DESC 32 +#define IDPF_MAX_RING_DESC 4096 +#define IDPF_DMA_MEM_ALIGN 4096 +/* Base address of the HW descriptor ring should be 128B aligned. */ +#define IDPF_RING_BASE_ALIGN 128 + +#define IDPF_DEFAULT_TX_RS_THRESH 32 +#define IDPF_DEFAULT_TX_FREE_THRESH 32 + +struct idpf_tx_entry { + struct rte_mbuf *mbuf; + uint16_t next_id; + uint16_t last_id; +}; + +/* Structure associated with each TX queue. */ +struct idpf_tx_queue { + const struct rte_memzone *mz; /* memzone for Tx ring */ + volatile struct idpf_flex_tx_desc *tx_ring; /* Tx ring virtual address */ + volatile union { + struct idpf_flex_tx_sched_desc *desc_ring; + struct idpf_splitq_tx_compl_desc *compl_ring; + }; + uint64_t tx_ring_phys_addr; /* Tx ring DMA address */ + struct idpf_tx_entry *sw_ring; /* address array of SW ring */ + + uint16_t nb_tx_desc; /* ring length */ + uint16_t tx_tail; /* current value of tail */ + volatile uint8_t *qtx_tail; /* register address of tail */ + /* number of used desc since RS bit set */ + uint16_t nb_used; + uint16_t nb_free; + uint16_t last_desc_cleaned; /* last desc have been cleaned*/ + uint16_t free_thresh; + uint16_t rs_thresh; + + uint16_t port_id; + uint16_t queue_id; + uint64_t offloads; + uint16_t next_dd; /* next to set RS, for VPMD */ + uint16_t next_rs; /* next to check DD, for VPMD */ + + bool q_set; /* if tx queue has been configured */ + bool q_started; /* if tx queue has been started */ + + /* only valid for split queue mode */ + uint16_t sw_nb_desc; + uint16_t sw_tail; + void **txqs; + uint32_t tx_start_qid; + uint8_t expected_gen_id; + struct idpf_tx_queue *complq; +}; + +int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, + uint16_t nb_desc, unsigned int socket_id, + const struct rte_eth_txconf *tx_conf); + +#endif /* _IDPF_RXTX_H_ */ diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build index ecf73355c3..b632b76656 100644 --- a/drivers/net/idpf/meson.build +++ b/drivers/net/idpf/meson.build @@ -11,5 +11,6 @@ deps += ['common_idpf'] sources = files( 'idpf_ethdev.c', + 'idpf_rxtx.c', 'idpf_vchnl.c', ) From patchwork Mon Oct 31 08:33:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xing, Beilei" X-Patchwork-Id: 119345 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 204F9A00C2; Mon, 31 Oct 2022 10:04:49 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 060D140E28; Mon, 31 Oct 2022 10:04:26 +0100 (CET) Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id 241C94069C for ; Mon, 31 Oct 2022 10:04:17 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667207058; x=1698743058; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=vG64RNfNtek09LOAkcxucCgnpuavxybPYqBguwgfsZk=; b=kkbdx4moWJ5gTOwMsKa/AKEV20h4pzzm4pvxdkynXA5DcRI3CMRqI8If qd+1gRR430Y6KIALkp8viE/dJuvbb13yIWI1o5acReOpr9ZKtYcZvjYZa +q/hF2KnTr2L0BAG8Vg9GFcyOdK01F9RIP85CGHOmTF4rKz4hcNYFfjT3 9MzqE4rn+ezM7nsePA8xtbSReBxplD5nQ1Y5czSLPWBsrmDyVwAX4mpKg 6UHo6VpMqGruMXxoxQxjrmM5Mta5h5XX9slPa7ne6BDXDtXbVrVWDjkfC w6IT1hjtiyoEZqVfB7OQcAS2Nh7xbiafhwGjQMRVmuiU5RKdSCPHm088i Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="370926281" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="370926281" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Oct 2022 02:04:06 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="878664801" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="878664801" Received: from dpdk-beileix-3.sh.intel.com ([10.67.110.253]) by fmsmga006.fm.intel.com with ESMTP; 31 Oct 2022 02:04:04 -0700 From: beilei.xing@intel.com To: jingjing.wu@intel.com, beilei.xing@intel.com Cc: dev@dpdk.org, Junfeng Guo , Xiaoyun Li Subject: [PATCH v18 04/18] net/idpf: add Rx queue setup Date: Mon, 31 Oct 2022 08:33:32 +0000 Message-Id: <20221031083346.16558-5-beilei.xing@intel.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20221031083346.16558-1-beilei.xing@intel.com> References: <20221031051556.98549-1-beilei.xing@intel.com> <20221031083346.16558-1-beilei.xing@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Junfeng Guo Add support for rx_queue_setup ops. Signed-off-by: Beilei Xing Signed-off-by: Xiaoyun Li Signed-off-by: Junfeng Guo --- drivers/net/idpf/idpf_ethdev.c | 11 + drivers/net/idpf/idpf_rxtx.c | 400 +++++++++++++++++++++++++++++++++ drivers/net/idpf/idpf_rxtx.h | 46 ++++ 3 files changed, 457 insertions(+) diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c index 54f20d30ca..fb5cd1b111 100644 --- a/drivers/net/idpf/idpf_ethdev.c +++ b/drivers/net/idpf/idpf_ethdev.c @@ -48,12 +48,22 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) .tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH, }; + dev_info->default_rxconf = (struct rte_eth_rxconf) { + .rx_free_thresh = IDPF_DEFAULT_RX_FREE_THRESH, + }; + dev_info->tx_desc_lim = (struct rte_eth_desc_lim) { .nb_max = IDPF_MAX_RING_DESC, .nb_min = IDPF_MIN_RING_DESC, .nb_align = IDPF_ALIGN_RING_DESC, }; + dev_info->rx_desc_lim = (struct rte_eth_desc_lim) { + .nb_max = IDPF_MAX_RING_DESC, + .nb_min = IDPF_MIN_RING_DESC, + .nb_align = IDPF_ALIGN_RING_DESC, + }; + return 0; } @@ -643,6 +653,7 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter) static const struct eth_dev_ops idpf_eth_dev_ops = { .dev_configure = idpf_dev_configure, .dev_close = idpf_dev_close, + .rx_queue_setup = idpf_rx_queue_setup, .tx_queue_setup = idpf_tx_queue_setup, .dev_infos_get = idpf_dev_info_get, }; diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c index 4afa0a2560..25dd5d85d5 100644 --- a/drivers/net/idpf/idpf_rxtx.c +++ b/drivers/net/idpf/idpf_rxtx.c @@ -8,6 +8,21 @@ #include "idpf_ethdev.h" #include "idpf_rxtx.h" +static int +check_rx_thresh(uint16_t nb_desc, uint16_t thresh) +{ + /* The following constraints must be satisfied: + * thresh < rxq->nb_rx_desc + */ + if (thresh >= nb_desc) { + PMD_INIT_LOG(ERR, "rx_free_thresh (%u) must be less than %u", + thresh, nb_desc); + return -EINVAL; + } + + return 0; +} + static int check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh, uint16_t tx_free_thresh) @@ -56,6 +71,87 @@ check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh, return 0; } +static void +reset_split_rx_descq(struct idpf_rx_queue *rxq) +{ + uint16_t len; + uint32_t i; + + if (rxq == NULL) + return; + + len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST; + + for (i = 0; i < len * sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3); + i++) + ((volatile char *)rxq->rx_ring)[i] = 0; + + rxq->rx_tail = 0; + rxq->expected_gen_id = 1; +} + +static void +reset_split_rx_bufq(struct idpf_rx_queue *rxq) +{ + uint16_t len; + uint32_t i; + + if (rxq == NULL) + return; + + len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST; + + for (i = 0; i < len * sizeof(struct virtchnl2_splitq_rx_buf_desc); + i++) + ((volatile char *)rxq->rx_ring)[i] = 0; + + memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf)); + + for (i = 0; i < IDPF_RX_MAX_BURST; i++) + rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf; + + /* The next descriptor id which can be received. */ + rxq->rx_next_avail = 0; + + /* The next descriptor id which can be refilled. */ + rxq->rx_tail = 0; + /* The number of descriptors which can be refilled. */ + rxq->nb_rx_hold = rxq->nb_rx_desc - 1; + + rxq->bufq1 = NULL; + rxq->bufq2 = NULL; +} + +static void +reset_single_rx_queue(struct idpf_rx_queue *rxq) +{ + uint16_t len; + uint32_t i; + + if (rxq == NULL) + return; + + len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST; + + for (i = 0; i < len * sizeof(struct virtchnl2_singleq_rx_buf_desc); + i++) + ((volatile char *)rxq->rx_ring)[i] = 0; + + memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf)); + + for (i = 0; i < IDPF_RX_MAX_BURST; i++) + rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf; + + rxq->rx_tail = 0; + rxq->nb_rx_hold = 0; + + if (rxq->pkt_first_seg != NULL) + rte_pktmbuf_free(rxq->pkt_first_seg); + + rxq->pkt_first_seg = NULL; + rxq->pkt_last_seg = NULL; +} + static void reset_split_tx_descq(struct idpf_tx_queue *txq) { @@ -145,6 +241,310 @@ reset_single_tx_queue(struct idpf_tx_queue *txq) txq->next_rs = txq->rs_thresh - 1; } +static int +idpf_rx_split_bufq_setup(struct rte_eth_dev *dev, struct idpf_rx_queue *bufq, + uint16_t queue_idx, uint16_t rx_free_thresh, + uint16_t nb_desc, unsigned int socket_id, + struct rte_mempool *mp) +{ + struct idpf_vport *vport = dev->data->dev_private; + struct idpf_adapter *adapter = vport->adapter; + struct idpf_hw *hw = &adapter->hw; + const struct rte_memzone *mz; + uint32_t ring_size; + uint16_t len; + + bufq->mp = mp; + bufq->nb_rx_desc = nb_desc; + bufq->rx_free_thresh = rx_free_thresh; + bufq->queue_id = vport->chunks_info.rx_buf_start_qid + queue_idx; + bufq->port_id = dev->data->port_id; + bufq->rx_hdr_len = 0; + bufq->adapter = adapter; + + len = rte_pktmbuf_data_room_size(bufq->mp) - RTE_PKTMBUF_HEADROOM; + bufq->rx_buf_len = len; + + /* Allocate the software ring. */ + len = nb_desc + IDPF_RX_MAX_BURST; + bufq->sw_ring = + rte_zmalloc_socket("idpf rx bufq sw ring", + sizeof(struct rte_mbuf *) * len, + RTE_CACHE_LINE_SIZE, + socket_id); + if (bufq->sw_ring == NULL) { + PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring"); + return -ENOMEM; + } + + /* Allocate a liitle more to support bulk allocate. */ + len = nb_desc + IDPF_RX_MAX_BURST; + ring_size = RTE_ALIGN(len * + sizeof(struct virtchnl2_splitq_rx_buf_desc), + IDPF_DMA_MEM_ALIGN); + mz = rte_eth_dma_zone_reserve(dev, "rx_buf_ring", queue_idx, + ring_size, IDPF_RING_BASE_ALIGN, + socket_id); + if (mz == NULL) { + PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue."); + rte_free(bufq->sw_ring); + return -ENOMEM; + } + + /* Zero all the descriptors in the ring. */ + memset(mz->addr, 0, ring_size); + bufq->rx_ring_phys_addr = mz->iova; + bufq->rx_ring = mz->addr; + + bufq->mz = mz; + reset_split_rx_bufq(bufq); + bufq->q_set = true; + bufq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_buf_qtail_start + + queue_idx * vport->chunks_info.rx_buf_qtail_spacing); + + /* TODO: allow bulk or vec */ + + return 0; +} + +static int +idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, + uint16_t nb_desc, unsigned int socket_id, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp) +{ + struct idpf_vport *vport = dev->data->dev_private; + struct idpf_adapter *adapter = vport->adapter; + struct idpf_rx_queue *bufq1, *bufq2; + const struct rte_memzone *mz; + struct idpf_rx_queue *rxq; + uint16_t rx_free_thresh; + uint32_t ring_size; + uint64_t offloads; + uint16_t qid; + uint16_t len; + int ret; + + offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads; + + /* Check free threshold */ + rx_free_thresh = (rx_conf->rx_free_thresh == 0) ? + IDPF_DEFAULT_RX_FREE_THRESH : + rx_conf->rx_free_thresh; + if (check_rx_thresh(nb_desc, rx_free_thresh) != 0) + return -EINVAL; + + /* Setup Rx description queue */ + rxq = rte_zmalloc_socket("idpf rxq", + sizeof(struct idpf_rx_queue), + RTE_CACHE_LINE_SIZE, + socket_id); + if (rxq == NULL) { + PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure"); + return -ENOMEM; + } + + rxq->mp = mp; + rxq->nb_rx_desc = nb_desc; + rxq->rx_free_thresh = rx_free_thresh; + rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx; + rxq->port_id = dev->data->port_id; + rxq->rx_hdr_len = 0; + rxq->adapter = adapter; + rxq->offloads = offloads; + + len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM; + rxq->rx_buf_len = len; + + len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST; + ring_size = RTE_ALIGN(len * + sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3), + IDPF_DMA_MEM_ALIGN); + mz = rte_eth_dma_zone_reserve(dev, "rx_cpmpl_ring", queue_idx, + ring_size, IDPF_RING_BASE_ALIGN, + socket_id); + + if (mz == NULL) { + PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX"); + ret = -ENOMEM; + goto free_rxq; + } + + /* Zero all the descriptors in the ring. */ + memset(mz->addr, 0, ring_size); + rxq->rx_ring_phys_addr = mz->iova; + rxq->rx_ring = mz->addr; + + rxq->mz = mz; + reset_split_rx_descq(rxq); + + /* TODO: allow bulk or vec */ + + /* setup Rx buffer queue */ + bufq1 = rte_zmalloc_socket("idpf bufq1", + sizeof(struct idpf_rx_queue), + RTE_CACHE_LINE_SIZE, + socket_id); + if (bufq1 == NULL) { + PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 1."); + ret = -ENOMEM; + goto free_mz; + } + qid = 2 * queue_idx; + ret = idpf_rx_split_bufq_setup(dev, bufq1, qid, rx_free_thresh, + nb_desc, socket_id, mp); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Failed to setup buffer queue 1"); + ret = -EINVAL; + goto free_bufq1; + } + rxq->bufq1 = bufq1; + + bufq2 = rte_zmalloc_socket("idpf bufq2", + sizeof(struct idpf_rx_queue), + RTE_CACHE_LINE_SIZE, + socket_id); + if (bufq2 == NULL) { + PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 2."); + rte_free(bufq1->sw_ring); + rte_memzone_free(bufq1->mz); + ret = -ENOMEM; + goto free_bufq1; + } + qid = 2 * queue_idx + 1; + ret = idpf_rx_split_bufq_setup(dev, bufq2, qid, rx_free_thresh, + nb_desc, socket_id, mp); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Failed to setup buffer queue 2"); + rte_free(bufq1->sw_ring); + rte_memzone_free(bufq1->mz); + ret = -EINVAL; + goto free_bufq2; + } + rxq->bufq2 = bufq2; + + rxq->q_set = true; + dev->data->rx_queues[queue_idx] = rxq; + + return 0; + +free_bufq2: + rte_free(bufq2); +free_bufq1: + rte_free(bufq1); +free_mz: + rte_memzone_free(mz); +free_rxq: + rte_free(rxq); + + return ret; +} + +static int +idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, + uint16_t nb_desc, unsigned int socket_id, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp) +{ + struct idpf_vport *vport = dev->data->dev_private; + struct idpf_adapter *adapter = vport->adapter; + struct idpf_hw *hw = &adapter->hw; + const struct rte_memzone *mz; + struct idpf_rx_queue *rxq; + uint16_t rx_free_thresh; + uint32_t ring_size; + uint64_t offloads; + uint16_t len; + + offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads; + + /* Check free threshold */ + rx_free_thresh = (rx_conf->rx_free_thresh == 0) ? + IDPF_DEFAULT_RX_FREE_THRESH : + rx_conf->rx_free_thresh; + if (check_rx_thresh(nb_desc, rx_free_thresh) != 0) + return -EINVAL; + + /* Setup Rx description queue */ + rxq = rte_zmalloc_socket("idpf rxq", + sizeof(struct idpf_rx_queue), + RTE_CACHE_LINE_SIZE, + socket_id); + if (rxq == NULL) { + PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure"); + return -ENOMEM; + } + + rxq->mp = mp; + rxq->nb_rx_desc = nb_desc; + rxq->rx_free_thresh = rx_free_thresh; + rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx; + rxq->port_id = dev->data->port_id; + rxq->rx_hdr_len = 0; + rxq->adapter = adapter; + rxq->offloads = offloads; + + len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM; + rxq->rx_buf_len = len; + + len = nb_desc + IDPF_RX_MAX_BURST; + rxq->sw_ring = + rte_zmalloc_socket("idpf rxq sw ring", + sizeof(struct rte_mbuf *) * len, + RTE_CACHE_LINE_SIZE, + socket_id); + if (rxq->sw_ring == NULL) { + PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring"); + rte_free(rxq); + return -ENOMEM; + } + + /* Allocate a liitle more to support bulk allocate. */ + len = nb_desc + IDPF_RX_MAX_BURST; + ring_size = RTE_ALIGN(len * + sizeof(struct virtchnl2_singleq_rx_buf_desc), + IDPF_DMA_MEM_ALIGN); + mz = rte_eth_dma_zone_reserve(dev, "rx ring", queue_idx, + ring_size, IDPF_RING_BASE_ALIGN, + socket_id); + if (mz == NULL) { + PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue."); + rte_free(rxq->sw_ring); + rte_free(rxq); + return -ENOMEM; + } + + /* Zero all the descriptors in the ring. */ + memset(mz->addr, 0, ring_size); + rxq->rx_ring_phys_addr = mz->iova; + rxq->rx_ring = mz->addr; + + rxq->mz = mz; + reset_single_rx_queue(rxq); + rxq->q_set = true; + dev->data->rx_queues[queue_idx] = rxq; + rxq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_qtail_start + + queue_idx * vport->chunks_info.rx_qtail_spacing); + + return 0; +} + +int +idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, + uint16_t nb_desc, unsigned int socket_id, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp) +{ + struct idpf_vport *vport = dev->data->dev_private; + + if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) + return idpf_rx_single_queue_setup(dev, queue_idx, nb_desc, + socket_id, rx_conf, mp); + else + return idpf_rx_split_queue_setup(dev, queue_idx, nb_desc, + socket_id, rx_conf, mp); +} + static int idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, uint16_t nb_desc, unsigned int socket_id, diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h index c7ba15b058..3f3932c3eb 100644 --- a/drivers/net/idpf/idpf_rxtx.h +++ b/drivers/net/idpf/idpf_rxtx.h @@ -15,9 +15,51 @@ /* Base address of the HW descriptor ring should be 128B aligned. */ #define IDPF_RING_BASE_ALIGN 128 +#define IDPF_RX_MAX_BURST 32 +#define IDPF_DEFAULT_RX_FREE_THRESH 32 + #define IDPF_DEFAULT_TX_RS_THRESH 32 #define IDPF_DEFAULT_TX_FREE_THRESH 32 +struct idpf_rx_queue { + struct idpf_adapter *adapter; /* the adapter this queue belongs to */ + struct rte_mempool *mp; /* mbuf pool to populate Rx ring */ + const struct rte_memzone *mz; /* memzone for Rx ring */ + volatile void *rx_ring; + struct rte_mbuf **sw_ring; /* address of SW ring */ + uint64_t rx_ring_phys_addr; /* Rx ring DMA address */ + + uint16_t nb_rx_desc; /* ring length */ + uint16_t rx_tail; /* current value of tail */ + volatile uint8_t *qrx_tail; /* register address of tail */ + uint16_t rx_free_thresh; /* max free RX desc to hold */ + uint16_t nb_rx_hold; /* number of held free RX desc */ + struct rte_mbuf *pkt_first_seg; /* first segment of current packet */ + struct rte_mbuf *pkt_last_seg; /* last segment of current packet */ + struct rte_mbuf fake_mbuf; /* dummy mbuf */ + + uint16_t rx_nb_avail; + uint16_t rx_next_avail; + + uint16_t port_id; /* device port ID */ + uint16_t queue_id; /* Rx queue index */ + uint16_t rx_buf_len; /* The packet buffer size */ + uint16_t rx_hdr_len; /* The header buffer size */ + uint16_t max_pkt_len; /* Maximum packet length */ + uint8_t rxdid; + + bool q_set; /* if rx queue has been configured */ + bool q_started; /* if rx queue has been started */ + + /* only valid for split queue mode */ + uint8_t expected_gen_id; + struct idpf_rx_queue *bufq1; + struct idpf_rx_queue *bufq2; + + uint64_t offloads; + uint32_t hw_register_set; +}; + struct idpf_tx_entry { struct rte_mbuf *mbuf; uint16_t next_id; @@ -63,6 +105,10 @@ struct idpf_tx_queue { struct idpf_tx_queue *complq; }; +int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, + uint16_t nb_desc, unsigned int socket_id, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp); int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, uint16_t nb_desc, unsigned int socket_id, const struct rte_eth_txconf *tx_conf); From patchwork Mon Oct 31 08:33:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xing, Beilei" X-Patchwork-Id: 119344 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 15ACFA00C2; Mon, 31 Oct 2022 10:04:43 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2DACE40F16; Mon, 31 Oct 2022 10:04:25 +0100 (CET) Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id 3D4A740151 for ; Mon, 31 Oct 2022 10:04:18 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667207058; x=1698743058; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=oMQRVc+ustbPDp3T1l0Vo8SltdOE8k7HBV1G87h4uKM=; b=SDZY/WdzEl4SpQUzWu9mxWz45tBHnxQ/LbWy9Xhuj1cL+kjHs1+/GPJP QwQ1gi10sDzsRX/7+5u+gCuNdHgEh+B9R5nP0qLbtUn1IIfcvnzf97h1N 5NjPYko6PT0G5XE1yY4lXMmH/YlIS8ya7rmOPuathg67m6J0PPWymJDU5 a88X8rBweKIvAaPohgGNTHZmQLc5e13IRMiXwxLCWxgvsKm1Y4Yswx29Y kLox5MAPIVEhcLtrpc7C1+5UIjlHXulQQVeGn3Eq/5nm1TRDbNw0jZByG zPd2mdzz6m8uF1gqjduEo9RX2EEFx3N+hqB2t1nheBDNv68R9VSgH7NxG w==; X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="370926291" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="370926291" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Oct 2022 02:04:08 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="878664809" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="878664809" Received: from dpdk-beileix-3.sh.intel.com ([10.67.110.253]) by fmsmga006.fm.intel.com with ESMTP; 31 Oct 2022 02:04:06 -0700 From: beilei.xing@intel.com To: jingjing.wu@intel.com, beilei.xing@intel.com Cc: dev@dpdk.org, Junfeng Guo , Xiaoyun Li Subject: [PATCH v18 05/18] net/idpf: add support for device start and stop Date: Mon, 31 Oct 2022 08:33:33 +0000 Message-Id: <20221031083346.16558-6-beilei.xing@intel.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20221031083346.16558-1-beilei.xing@intel.com> References: <20221031051556.98549-1-beilei.xing@intel.com> <20221031083346.16558-1-beilei.xing@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Junfeng Guo Add dev ops dev_start, dev_stop and link_update. Signed-off-by: Beilei Xing Signed-off-by: Xiaoyun Li Signed-off-by: Junfeng Guo --- drivers/net/idpf/idpf_ethdev.c | 55 ++++++++++++++++++++++++++++++++++ drivers/net/idpf/idpf_rxtx.c | 20 +++++++++++++ 2 files changed, 75 insertions(+) diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c index fb5cd1b111..621bf9aad5 100644 --- a/drivers/net/idpf/idpf_ethdev.c +++ b/drivers/net/idpf/idpf_ethdev.c @@ -29,6 +29,22 @@ static const char * const idpf_valid_args[] = { NULL }; +static int +idpf_dev_link_update(struct rte_eth_dev *dev, + __rte_unused int wait_to_complete) +{ + struct rte_eth_link new_link; + + memset(&new_link, 0, sizeof(new_link)); + + new_link.link_speed = RTE_ETH_SPEED_NUM_NONE; + new_link.link_duplex = RTE_ETH_LINK_FULL_DUPLEX; + new_link.link_autoneg = !(dev->data->dev_conf.link_speeds & + RTE_ETH_LINK_SPEED_FIXED); + + return rte_eth_linkstatus_set(dev, &new_link); +} + static int idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) { @@ -267,6 +283,42 @@ idpf_dev_configure(struct rte_eth_dev *dev) return 0; } +static int +idpf_dev_start(struct rte_eth_dev *dev) +{ + struct idpf_vport *vport = dev->data->dev_private; + int ret; + + if (dev->data->mtu > vport->max_mtu) { + PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu); + return -EINVAL; + } + + vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD; + + /* TODO: start queues */ + + ret = idpf_vc_ena_dis_vport(vport, true); + if (ret != 0) { + PMD_DRV_LOG(ERR, "Failed to enable vport"); + return ret; + } + + return 0; +} + +static int +idpf_dev_stop(struct rte_eth_dev *dev) +{ + struct idpf_vport *vport = dev->data->dev_private; + + idpf_vc_ena_dis_vport(vport, false); + + /* TODO: stop queues */ + + return 0; +} + static int idpf_dev_close(struct rte_eth_dev *dev) { @@ -656,6 +708,9 @@ static const struct eth_dev_ops idpf_eth_dev_ops = { .rx_queue_setup = idpf_rx_queue_setup, .tx_queue_setup = idpf_tx_queue_setup, .dev_infos_get = idpf_dev_info_get, + .dev_start = idpf_dev_start, + .dev_stop = idpf_dev_stop, + .link_update = idpf_dev_link_update, }; static uint16_t diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c index 25dd5d85d5..3528d2f2c7 100644 --- a/drivers/net/idpf/idpf_rxtx.c +++ b/drivers/net/idpf/idpf_rxtx.c @@ -334,6 +334,11 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, if (check_rx_thresh(nb_desc, rx_free_thresh) != 0) return -EINVAL; + if (rx_conf->rx_deferred_start) { + PMD_INIT_LOG(ERR, "Queue start is not supported currently."); + return -EINVAL; + } + /* Setup Rx description queue */ rxq = rte_zmalloc_socket("idpf rxq", sizeof(struct idpf_rx_queue), @@ -465,6 +470,11 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, if (check_rx_thresh(nb_desc, rx_free_thresh) != 0) return -EINVAL; + if (rx_conf->rx_deferred_start) { + PMD_INIT_LOG(ERR, "Queue start is not supported currently."); + return -EINVAL; + } + /* Setup Rx description queue */ rxq = rte_zmalloc_socket("idpf rxq", sizeof(struct idpf_rx_queue), @@ -569,6 +579,11 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0) return -EINVAL; + if (tx_conf->tx_deferred_start) { + PMD_INIT_LOG(ERR, "Queue start is not supported currently."); + return -EINVAL; + } + /* Allocate the TX queue data structure. */ txq = rte_zmalloc_socket("idpf split txq", sizeof(struct idpf_tx_queue), @@ -691,6 +706,11 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0) return -EINVAL; + if (tx_conf->tx_deferred_start) { + PMD_INIT_LOG(ERR, "Queue start is not supported currently."); + return -EINVAL; + } + /* Allocate the TX queue data structure. */ txq = rte_zmalloc_socket("idpf txq", sizeof(struct idpf_tx_queue), From patchwork Mon Oct 31 08:33:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xing, Beilei" X-Patchwork-Id: 119346 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7A4A1A00C2; Mon, 31 Oct 2022 10:04:57 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CAE10410F2; Mon, 31 Oct 2022 10:04:26 +0100 (CET) Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id AE99940695 for ; Mon, 31 Oct 2022 10:04:18 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667207058; x=1698743058; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=hx6WVIXKDFS6BU+paLJ+gw71y0cKpv8hzOsE/RvihrY=; b=jnxHrPzmIQvjtzigWz8vvL8GsatZgqqcHEFyP/QiVG2P7T6+S9EqGXx+ HGmYvbVt+vIK4kS8jvYL/rdmQXnGltKNlfM7Xb6iqf+djyimpCpPiJAzU bveydBrQ7p/gFfk60x5cAZWhKHXkod8B9LFMrkq/X290V5ABmxhllAcjD HaE0MqWAoS+ZfazmliquSrYU7u5CvUeWfn3ar/qjyR6Pog9DAnwq8+pUx grOq/SnwTD4U8VfcrdSv1ZAwUwIaIq88wr+O4Ar/ue4l7IkkbsB1WPBxZ mUtTT1dD0fPQ9AS3Gv/gWco1AKlH9DAorr65tHiJw5vCL0XkHjZBDxOCf g==; X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="370926299" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="370926299" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Oct 2022 02:04:10 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="878664826" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="878664826" Received: from dpdk-beileix-3.sh.intel.com ([10.67.110.253]) by fmsmga006.fm.intel.com with ESMTP; 31 Oct 2022 02:04:08 -0700 From: beilei.xing@intel.com To: jingjing.wu@intel.com, beilei.xing@intel.com Cc: dev@dpdk.org, Junfeng Guo , Xiaoyun Li Subject: [PATCH v18 06/18] net/idpf: add support for queue start Date: Mon, 31 Oct 2022 08:33:34 +0000 Message-Id: <20221031083346.16558-7-beilei.xing@intel.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20221031083346.16558-1-beilei.xing@intel.com> References: <20221031051556.98549-1-beilei.xing@intel.com> <20221031083346.16558-1-beilei.xing@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Junfeng Guo Add support for these device ops: - rx_queue_start - tx_queue_start Signed-off-by: Beilei Xing Signed-off-by: Xiaoyun Li Signed-off-by: Junfeng Guo --- drivers/net/idpf/idpf_ethdev.c | 42 +++- drivers/net/idpf/idpf_ethdev.h | 9 + drivers/net/idpf/idpf_rxtx.c | 237 +++++++++++++++-- drivers/net/idpf/idpf_rxtx.h | 6 + drivers/net/idpf/idpf_vchnl.c | 447 +++++++++++++++++++++++++++++++++ 5 files changed, 720 insertions(+), 21 deletions(-) diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c index 621bf9aad5..0400ed611f 100644 --- a/drivers/net/idpf/idpf_ethdev.c +++ b/drivers/net/idpf/idpf_ethdev.c @@ -283,6 +283,39 @@ idpf_dev_configure(struct rte_eth_dev *dev) return 0; } +static int +idpf_start_queues(struct rte_eth_dev *dev) +{ + struct idpf_rx_queue *rxq; + struct idpf_tx_queue *txq; + int err = 0; + int i; + + for (i = 0; i < dev->data->nb_tx_queues; i++) { + txq = dev->data->tx_queues[i]; + if (txq == NULL || txq->tx_deferred_start) + continue; + err = idpf_tx_queue_start(dev, i); + if (err != 0) { + PMD_DRV_LOG(ERR, "Fail to start Tx queue %u", i); + return err; + } + } + + for (i = 0; i < dev->data->nb_rx_queues; i++) { + rxq = dev->data->rx_queues[i]; + if (rxq == NULL || rxq->rx_deferred_start) + continue; + err = idpf_rx_queue_start(dev, i); + if (err != 0) { + PMD_DRV_LOG(ERR, "Fail to start Rx queue %u", i); + return err; + } + } + + return err; +} + static int idpf_dev_start(struct rte_eth_dev *dev) { @@ -296,11 +329,16 @@ idpf_dev_start(struct rte_eth_dev *dev) vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD; - /* TODO: start queues */ + ret = idpf_start_queues(dev); + if (ret != 0) { + PMD_DRV_LOG(ERR, "Failed to start queues"); + return ret; + } ret = idpf_vc_ena_dis_vport(vport, true); if (ret != 0) { PMD_DRV_LOG(ERR, "Failed to enable vport"); + /* TODO: stop queues */ return ret; } @@ -711,6 +749,8 @@ static const struct eth_dev_ops idpf_eth_dev_ops = { .dev_start = idpf_dev_start, .dev_stop = idpf_dev_stop, .link_update = idpf_dev_link_update, + .rx_queue_start = idpf_rx_queue_start, + .tx_queue_start = idpf_tx_queue_start, }; static uint16_t diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h index 84ae6641e2..96c22009e9 100644 --- a/drivers/net/idpf/idpf_ethdev.h +++ b/drivers/net/idpf/idpf_ethdev.h @@ -24,7 +24,9 @@ #define IDPF_DEFAULT_TXQ_NUM 16 #define IDPF_INVALID_VPORT_IDX 0xffff +#define IDPF_TXQ_PER_GRP 1 #define IDPF_TX_COMPLQ_PER_GRP 1 +#define IDPF_RXQ_PER_GRP 1 #define IDPF_RX_BUFQ_PER_GRP 2 #define IDPF_CTLQ_ID -1 @@ -182,6 +184,13 @@ int idpf_vc_check_api_version(struct idpf_adapter *adapter); int idpf_vc_get_caps(struct idpf_adapter *adapter); int idpf_vc_create_vport(struct idpf_adapter *adapter); int idpf_vc_destroy_vport(struct idpf_vport *vport); +int idpf_vc_config_rxqs(struct idpf_vport *vport); +int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id); +int idpf_vc_config_txqs(struct idpf_vport *vport); +int idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id); +int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid, + bool rx, bool on); +int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable); int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable); int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops, uint16_t buf_len, uint8_t *buf); diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c index 3528d2f2c7..6d954afd9d 100644 --- a/drivers/net/idpf/idpf_rxtx.c +++ b/drivers/net/idpf/idpf_rxtx.c @@ -334,11 +334,6 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, if (check_rx_thresh(nb_desc, rx_free_thresh) != 0) return -EINVAL; - if (rx_conf->rx_deferred_start) { - PMD_INIT_LOG(ERR, "Queue start is not supported currently."); - return -EINVAL; - } - /* Setup Rx description queue */ rxq = rte_zmalloc_socket("idpf rxq", sizeof(struct idpf_rx_queue), @@ -354,6 +349,7 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, rxq->rx_free_thresh = rx_free_thresh; rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx; rxq->port_id = dev->data->port_id; + rxq->rx_deferred_start = rx_conf->rx_deferred_start; rxq->rx_hdr_len = 0; rxq->adapter = adapter; rxq->offloads = offloads; @@ -470,11 +466,6 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, if (check_rx_thresh(nb_desc, rx_free_thresh) != 0) return -EINVAL; - if (rx_conf->rx_deferred_start) { - PMD_INIT_LOG(ERR, "Queue start is not supported currently."); - return -EINVAL; - } - /* Setup Rx description queue */ rxq = rte_zmalloc_socket("idpf rxq", sizeof(struct idpf_rx_queue), @@ -490,6 +481,7 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, rxq->rx_free_thresh = rx_free_thresh; rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx; rxq->port_id = dev->data->port_id; + rxq->rx_deferred_start = rx_conf->rx_deferred_start; rxq->rx_hdr_len = 0; rxq->adapter = adapter; rxq->offloads = offloads; @@ -579,11 +571,6 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0) return -EINVAL; - if (tx_conf->tx_deferred_start) { - PMD_INIT_LOG(ERR, "Queue start is not supported currently."); - return -EINVAL; - } - /* Allocate the TX queue data structure. */ txq = rte_zmalloc_socket("idpf split txq", sizeof(struct idpf_tx_queue), @@ -600,6 +587,7 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx; txq->port_id = dev->data->port_id; txq->offloads = offloads; + txq->tx_deferred_start = tx_conf->tx_deferred_start; /* Allocate software ring */ txq->sw_nb_desc = 2 * nb_desc; @@ -706,11 +694,6 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0) return -EINVAL; - if (tx_conf->tx_deferred_start) { - PMD_INIT_LOG(ERR, "Queue start is not supported currently."); - return -EINVAL; - } - /* Allocate the TX queue data structure. */ txq = rte_zmalloc_socket("idpf txq", sizeof(struct idpf_tx_queue), @@ -729,6 +712,7 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx; txq->port_id = dev->data->port_id; txq->offloads = offloads; + txq->tx_deferred_start = tx_conf->tx_deferred_start; /* Allocate software ring */ txq->sw_ring = @@ -782,3 +766,216 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc, socket_id, tx_conf); } +static int +idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq) +{ + volatile struct virtchnl2_singleq_rx_buf_desc *rxd; + struct rte_mbuf *mbuf = NULL; + uint64_t dma_addr; + uint16_t i; + + for (i = 0; i < rxq->nb_rx_desc; i++) { + mbuf = rte_mbuf_raw_alloc(rxq->mp); + if (unlikely(mbuf == NULL)) { + PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX"); + return -ENOMEM; + } + + rte_mbuf_refcnt_set(mbuf, 1); + mbuf->next = NULL; + mbuf->data_off = RTE_PKTMBUF_HEADROOM; + mbuf->nb_segs = 1; + mbuf->port = rxq->port_id; + + dma_addr = + rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf)); + + rxd = &((volatile struct virtchnl2_singleq_rx_buf_desc *)(rxq->rx_ring))[i]; + rxd->pkt_addr = dma_addr; + rxd->hdr_addr = 0; + rxd->rsvd1 = 0; + rxd->rsvd2 = 0; + rxq->sw_ring[i] = mbuf; + } + + return 0; +} + +static int +idpf_alloc_split_rxq_mbufs(struct idpf_rx_queue *rxq) +{ + volatile struct virtchnl2_splitq_rx_buf_desc *rxd; + struct rte_mbuf *mbuf = NULL; + uint64_t dma_addr; + uint16_t i; + + for (i = 0; i < rxq->nb_rx_desc; i++) { + mbuf = rte_mbuf_raw_alloc(rxq->mp); + if (unlikely(mbuf == NULL)) { + PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX"); + return -ENOMEM; + } + + rte_mbuf_refcnt_set(mbuf, 1); + mbuf->next = NULL; + mbuf->data_off = RTE_PKTMBUF_HEADROOM; + mbuf->nb_segs = 1; + mbuf->port = rxq->port_id; + + dma_addr = + rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf)); + + rxd = &((volatile struct virtchnl2_splitq_rx_buf_desc *)(rxq->rx_ring))[i]; + rxd->qword0.buf_id = i; + rxd->qword0.rsvd0 = 0; + rxd->qword0.rsvd1 = 0; + rxd->pkt_addr = dma_addr; + rxd->hdr_addr = 0; + rxd->rsvd2 = 0; + + rxq->sw_ring[i] = mbuf; + } + + rxq->nb_rx_hold = 0; + rxq->rx_tail = rxq->nb_rx_desc - 1; + + return 0; +} + +int +idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id) +{ + struct idpf_rx_queue *rxq; + int err; + + if (rx_queue_id >= dev->data->nb_rx_queues) + return -EINVAL; + + rxq = dev->data->rx_queues[rx_queue_id]; + + if (rxq == NULL || !rxq->q_set) { + PMD_DRV_LOG(ERR, "RX queue %u not available or setup", + rx_queue_id); + return -EINVAL; + } + + if (rxq->bufq1 == NULL) { + /* Single queue */ + err = idpf_alloc_single_rxq_mbufs(rxq); + if (err != 0) { + PMD_DRV_LOG(ERR, "Failed to allocate RX queue mbuf"); + return err; + } + + rte_wmb(); + + /* Init the RX tail register. */ + IDPF_PCI_REG_WRITE(rxq->qrx_tail, rxq->nb_rx_desc - 1); + } else { + /* Split queue */ + err = idpf_alloc_split_rxq_mbufs(rxq->bufq1); + if (err != 0) { + PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf"); + return err; + } + err = idpf_alloc_split_rxq_mbufs(rxq->bufq2); + if (err != 0) { + PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf"); + return err; + } + + rte_wmb(); + + /* Init the RX tail register. */ + IDPF_PCI_REG_WRITE(rxq->bufq1->qrx_tail, rxq->bufq1->rx_tail); + IDPF_PCI_REG_WRITE(rxq->bufq2->qrx_tail, rxq->bufq2->rx_tail); + } + + return err; +} + +int +idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id) +{ + struct idpf_vport *vport = dev->data->dev_private; + struct idpf_rx_queue *rxq = + dev->data->rx_queues[rx_queue_id]; + int err = 0; + + err = idpf_vc_config_rxq(vport, rx_queue_id); + if (err != 0) { + PMD_DRV_LOG(ERR, "Fail to configure Rx queue %u", rx_queue_id); + return err; + } + + err = idpf_rx_queue_init(dev, rx_queue_id); + if (err != 0) { + PMD_DRV_LOG(ERR, "Failed to init RX queue %u", + rx_queue_id); + return err; + } + + /* Ready to switch the queue on */ + err = idpf_switch_queue(vport, rx_queue_id, true, true); + if (err != 0) { + PMD_DRV_LOG(ERR, "Failed to switch RX queue %u on", + rx_queue_id); + } else { + rxq->q_started = true; + dev->data->rx_queue_state[rx_queue_id] = + RTE_ETH_QUEUE_STATE_STARTED; + } + + return err; +} + +int +idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id) +{ + struct idpf_tx_queue *txq; + + if (tx_queue_id >= dev->data->nb_tx_queues) + return -EINVAL; + + txq = dev->data->tx_queues[tx_queue_id]; + + /* Init the RX tail register. */ + IDPF_PCI_REG_WRITE(txq->qtx_tail, 0); + + return 0; +} + +int +idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id) +{ + struct idpf_vport *vport = dev->data->dev_private; + struct idpf_tx_queue *txq = + dev->data->tx_queues[tx_queue_id]; + int err = 0; + + err = idpf_vc_config_txq(vport, tx_queue_id); + if (err != 0) { + PMD_DRV_LOG(ERR, "Fail to configure Tx queue %u", tx_queue_id); + return err; + } + + err = idpf_tx_queue_init(dev, tx_queue_id); + if (err != 0) { + PMD_DRV_LOG(ERR, "Failed to init TX queue %u", + tx_queue_id); + return err; + } + + /* Ready to switch the queue on */ + err = idpf_switch_queue(vport, tx_queue_id, false, true); + if (err != 0) { + PMD_DRV_LOG(ERR, "Failed to switch TX queue %u on", + tx_queue_id); + } else { + txq->q_started = true; + dev->data->tx_queue_state[tx_queue_id] = + RTE_ETH_QUEUE_STATE_STARTED; + } + + return err; +} diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h index 3f3932c3eb..ab9b3830fd 100644 --- a/drivers/net/idpf/idpf_rxtx.h +++ b/drivers/net/idpf/idpf_rxtx.h @@ -50,6 +50,7 @@ struct idpf_rx_queue { bool q_set; /* if rx queue has been configured */ bool q_started; /* if rx queue has been started */ + bool rx_deferred_start; /* don't start this queue in dev start */ /* only valid for split queue mode */ uint8_t expected_gen_id; @@ -95,6 +96,7 @@ struct idpf_tx_queue { bool q_set; /* if tx queue has been configured */ bool q_started; /* if tx queue has been started */ + bool tx_deferred_start; /* don't start this queue in dev start */ /* only valid for split queue mode */ uint16_t sw_nb_desc; @@ -109,8 +111,12 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, uint16_t nb_desc, unsigned int socket_id, const struct rte_eth_rxconf *rx_conf, struct rte_mempool *mp); +int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id); +int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id); int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, uint16_t nb_desc, unsigned int socket_id, const struct rte_eth_txconf *tx_conf); +int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id); +int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id); #endif /* _IDPF_RXTX_H_ */ diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c index e89c2a2783..08fb9061c2 100644 --- a/drivers/net/idpf/idpf_vchnl.c +++ b/drivers/net/idpf/idpf_vchnl.c @@ -21,6 +21,7 @@ #include #include "idpf_ethdev.h" +#include "idpf_rxtx.h" static int idpf_vc_clean(struct idpf_adapter *adapter) @@ -223,6 +224,10 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args) case VIRTCHNL2_OP_GET_CAPS: case VIRTCHNL2_OP_CREATE_VPORT: case VIRTCHNL2_OP_DESTROY_VPORT: + case VIRTCHNL2_OP_CONFIG_RX_QUEUES: + case VIRTCHNL2_OP_CONFIG_TX_QUEUES: + case VIRTCHNL2_OP_ENABLE_QUEUES: + case VIRTCHNL2_OP_DISABLE_QUEUES: case VIRTCHNL2_OP_ENABLE_VPORT: case VIRTCHNL2_OP_DISABLE_VPORT: /* for init virtchnl ops, need to poll the response */ @@ -390,6 +395,448 @@ idpf_vc_destroy_vport(struct idpf_vport *vport) return err; } +#define IDPF_RX_BUF_STRIDE 64 +int +idpf_vc_config_rxqs(struct idpf_vport *vport) +{ + struct idpf_adapter *adapter = vport->adapter; + struct idpf_rx_queue **rxq = + (struct idpf_rx_queue **)vport->dev_data->rx_queues; + struct virtchnl2_config_rx_queues *vc_rxqs = NULL; + struct virtchnl2_rxq_info *rxq_info; + struct idpf_cmd_info args; + uint16_t total_qs, num_qs; + int size, i, j; + int err = 0; + int k = 0; + + total_qs = vport->num_rx_q + vport->num_rx_bufq; + while (total_qs) { + if (total_qs > adapter->max_rxq_per_msg) { + num_qs = adapter->max_rxq_per_msg; + total_qs -= adapter->max_rxq_per_msg; + } else { + num_qs = total_qs; + total_qs = 0; + } + + size = sizeof(*vc_rxqs) + (num_qs - 1) * + sizeof(struct virtchnl2_rxq_info); + vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0); + if (vc_rxqs == NULL) { + PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues"); + err = -ENOMEM; + break; + } + vc_rxqs->vport_id = vport->vport_id; + vc_rxqs->num_qinfo = num_qs; + if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) { + for (i = 0; i < num_qs; i++, k++) { + rxq_info = &vc_rxqs->qinfo[i]; + rxq_info->dma_ring_addr = rxq[k]->rx_ring_phys_addr; + rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX; + rxq_info->queue_id = rxq[k]->queue_id; + rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE; + rxq_info->data_buffer_size = rxq[k]->rx_buf_len; + rxq_info->max_pkt_size = vport->max_pkt_len; + + rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M; + rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE; + + rxq_info->ring_len = rxq[k]->nb_rx_desc; + } + } else { + for (i = 0; i < num_qs / 3; i++, k++) { + /* Rx queue */ + rxq_info = &vc_rxqs->qinfo[i * 3]; + rxq_info->dma_ring_addr = + rxq[k]->rx_ring_phys_addr; + rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX; + rxq_info->queue_id = rxq[k]->queue_id; + rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT; + rxq_info->data_buffer_size = rxq[k]->rx_buf_len; + rxq_info->max_pkt_size = vport->max_pkt_len; + + rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M; + rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE; + + rxq_info->ring_len = rxq[k]->nb_rx_desc; + rxq_info->rx_bufq1_id = rxq[k]->bufq1->queue_id; + rxq_info->rx_bufq2_id = rxq[k]->bufq2->queue_id; + rxq_info->rx_buffer_low_watermark = 64; + + /* Buffer queue */ + for (j = 1; j <= IDPF_RX_BUFQ_PER_GRP; j++) { + struct idpf_rx_queue *bufq = j == 1 ? + rxq[k]->bufq1 : rxq[k]->bufq2; + rxq_info = &vc_rxqs->qinfo[i * 3 + j]; + rxq_info->dma_ring_addr = + bufq->rx_ring_phys_addr; + rxq_info->type = + VIRTCHNL2_QUEUE_TYPE_RX_BUFFER; + rxq_info->queue_id = bufq->queue_id; + rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT; + rxq_info->data_buffer_size = bufq->rx_buf_len; + rxq_info->desc_ids = + VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M; + rxq_info->ring_len = bufq->nb_rx_desc; + + rxq_info->buffer_notif_stride = + IDPF_RX_BUF_STRIDE; + rxq_info->rx_buffer_low_watermark = 64; + } + } + } + memset(&args, 0, sizeof(args)); + args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES; + args.in_args = (uint8_t *)vc_rxqs; + args.in_args_size = size; + args.out_buffer = adapter->mbx_resp; + args.out_size = IDPF_DFLT_MBX_BUF_SIZE; + + err = idpf_execute_vc_cmd(adapter, &args); + rte_free(vc_rxqs); + if (err != 0) { + PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES"); + break; + } + } + + return err; +} + +int +idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id) +{ + struct idpf_adapter *adapter = vport->adapter; + struct idpf_rx_queue **rxq = + (struct idpf_rx_queue **)vport->dev_data->rx_queues; + struct virtchnl2_config_rx_queues *vc_rxqs = NULL; + struct virtchnl2_rxq_info *rxq_info; + struct idpf_cmd_info args; + uint16_t num_qs; + int size, err, i; + + if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) + num_qs = IDPF_RXQ_PER_GRP; + else + num_qs = IDPF_RXQ_PER_GRP + IDPF_RX_BUFQ_PER_GRP; + + size = sizeof(*vc_rxqs) + (num_qs - 1) * + sizeof(struct virtchnl2_rxq_info); + vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0); + if (vc_rxqs == NULL) { + PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues"); + err = -ENOMEM; + return err; + } + vc_rxqs->vport_id = vport->vport_id; + vc_rxqs->num_qinfo = num_qs; + if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) { + rxq_info = &vc_rxqs->qinfo[0]; + rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr; + rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX; + rxq_info->queue_id = rxq[rxq_id]->queue_id; + rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE; + rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len; + rxq_info->max_pkt_size = vport->max_pkt_len; + + rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M; + rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE; + + rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc; + } else { + /* Rx queue */ + rxq_info = &vc_rxqs->qinfo[0]; + rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr; + rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX; + rxq_info->queue_id = rxq[rxq_id]->queue_id; + rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT; + rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len; + rxq_info->max_pkt_size = vport->max_pkt_len; + + rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M; + rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE; + + rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc; + rxq_info->rx_bufq1_id = rxq[rxq_id]->bufq1->queue_id; + rxq_info->rx_bufq2_id = rxq[rxq_id]->bufq2->queue_id; + rxq_info->rx_buffer_low_watermark = 64; + + /* Buffer queue */ + for (i = 1; i <= IDPF_RX_BUFQ_PER_GRP; i++) { + struct idpf_rx_queue *bufq = + i == 1 ? rxq[rxq_id]->bufq1 : rxq[rxq_id]->bufq2; + rxq_info = &vc_rxqs->qinfo[i]; + rxq_info->dma_ring_addr = bufq->rx_ring_phys_addr; + rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER; + rxq_info->queue_id = bufq->queue_id; + rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT; + rxq_info->data_buffer_size = bufq->rx_buf_len; + rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M; + rxq_info->ring_len = bufq->nb_rx_desc; + + rxq_info->buffer_notif_stride = IDPF_RX_BUF_STRIDE; + rxq_info->rx_buffer_low_watermark = 64; + } + } + + memset(&args, 0, sizeof(args)); + args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES; + args.in_args = (uint8_t *)vc_rxqs; + args.in_args_size = size; + args.out_buffer = adapter->mbx_resp; + args.out_size = IDPF_DFLT_MBX_BUF_SIZE; + + err = idpf_execute_vc_cmd(adapter, &args); + rte_free(vc_rxqs); + if (err != 0) + PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES"); + + return err; +} + +int +idpf_vc_config_txqs(struct idpf_vport *vport) +{ + struct idpf_adapter *adapter = vport->adapter; + struct idpf_tx_queue **txq = + (struct idpf_tx_queue **)vport->dev_data->tx_queues; + struct virtchnl2_config_tx_queues *vc_txqs = NULL; + struct virtchnl2_txq_info *txq_info; + struct idpf_cmd_info args; + uint16_t total_qs, num_qs; + int size, i; + int err = 0; + int k = 0; + + total_qs = vport->num_tx_q + vport->num_tx_complq; + while (total_qs) { + if (total_qs > adapter->max_txq_per_msg) { + num_qs = adapter->max_txq_per_msg; + total_qs -= adapter->max_txq_per_msg; + } else { + num_qs = total_qs; + total_qs = 0; + } + size = sizeof(*vc_txqs) + (num_qs - 1) * + sizeof(struct virtchnl2_txq_info); + vc_txqs = rte_zmalloc("cfg_txqs", size, 0); + if (vc_txqs == NULL) { + PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues"); + err = -ENOMEM; + break; + } + vc_txqs->vport_id = vport->vport_id; + vc_txqs->num_qinfo = num_qs; + if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) { + for (i = 0; i < num_qs; i++, k++) { + txq_info = &vc_txqs->qinfo[i]; + txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr; + txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX; + txq_info->queue_id = txq[k]->queue_id; + txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE; + txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE; + txq_info->ring_len = txq[k]->nb_tx_desc; + } + } else { + for (i = 0; i < num_qs / 2; i++, k++) { + /* txq info */ + txq_info = &vc_txqs->qinfo[2 * i]; + txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr; + txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX; + txq_info->queue_id = txq[k]->queue_id; + txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT; + txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW; + txq_info->ring_len = txq[k]->nb_tx_desc; + txq_info->tx_compl_queue_id = + txq[k]->complq->queue_id; + txq_info->relative_queue_id = txq_info->queue_id; + + /* tx completion queue info */ + txq_info = &vc_txqs->qinfo[2 * i + 1]; + txq_info->dma_ring_addr = + txq[k]->complq->tx_ring_phys_addr; + txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION; + txq_info->queue_id = txq[k]->complq->queue_id; + txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT; + txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW; + txq_info->ring_len = txq[k]->complq->nb_tx_desc; + } + } + + memset(&args, 0, sizeof(args)); + args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES; + args.in_args = (uint8_t *)vc_txqs; + args.in_args_size = size; + args.out_buffer = adapter->mbx_resp; + args.out_size = IDPF_DFLT_MBX_BUF_SIZE; + + err = idpf_execute_vc_cmd(adapter, &args); + rte_free(vc_txqs); + if (err != 0) { + PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES"); + break; + } + } + + return err; +} + +int +idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id) +{ + struct idpf_adapter *adapter = vport->adapter; + struct idpf_tx_queue **txq = + (struct idpf_tx_queue **)vport->dev_data->tx_queues; + struct virtchnl2_config_tx_queues *vc_txqs = NULL; + struct virtchnl2_txq_info *txq_info; + struct idpf_cmd_info args; + uint16_t num_qs; + int size, err; + + if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) + num_qs = IDPF_TXQ_PER_GRP; + else + num_qs = IDPF_TXQ_PER_GRP + IDPF_TX_COMPLQ_PER_GRP; + + size = sizeof(*vc_txqs) + (num_qs - 1) * + sizeof(struct virtchnl2_txq_info); + vc_txqs = rte_zmalloc("cfg_txqs", size, 0); + if (vc_txqs == NULL) { + PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues"); + err = -ENOMEM; + return err; + } + vc_txqs->vport_id = vport->vport_id; + vc_txqs->num_qinfo = num_qs; + + if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) { + txq_info = &vc_txqs->qinfo[0]; + txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr; + txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX; + txq_info->queue_id = txq[txq_id]->queue_id; + txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE; + txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE; + txq_info->ring_len = txq[txq_id]->nb_tx_desc; + } else { + /* txq info */ + txq_info = &vc_txqs->qinfo[0]; + txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr; + txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX; + txq_info->queue_id = txq[txq_id]->queue_id; + txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT; + txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW; + txq_info->ring_len = txq[txq_id]->nb_tx_desc; + txq_info->tx_compl_queue_id = txq[txq_id]->complq->queue_id; + txq_info->relative_queue_id = txq_info->queue_id; + + /* tx completion queue info */ + txq_info = &vc_txqs->qinfo[1]; + txq_info->dma_ring_addr = txq[txq_id]->complq->tx_ring_phys_addr; + txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION; + txq_info->queue_id = txq[txq_id]->complq->queue_id; + txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT; + txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW; + txq_info->ring_len = txq[txq_id]->complq->nb_tx_desc; + } + + memset(&args, 0, sizeof(args)); + args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES; + args.in_args = (uint8_t *)vc_txqs; + args.in_args_size = size; + args.out_buffer = adapter->mbx_resp; + args.out_size = IDPF_DFLT_MBX_BUF_SIZE; + + err = idpf_execute_vc_cmd(adapter, &args); + rte_free(vc_txqs); + if (err != 0) + PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES"); + + return err; +} + +static int +idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid, + uint32_t type, bool on) +{ + struct idpf_adapter *adapter = vport->adapter; + struct virtchnl2_del_ena_dis_queues *queue_select; + struct virtchnl2_queue_chunk *queue_chunk; + struct idpf_cmd_info args; + int err, len; + + len = sizeof(struct virtchnl2_del_ena_dis_queues); + queue_select = rte_zmalloc("queue_select", len, 0); + if (queue_select == NULL) + return -ENOMEM; + + queue_chunk = queue_select->chunks.chunks; + queue_select->chunks.num_chunks = 1; + queue_select->vport_id = vport->vport_id; + + queue_chunk->type = type; + queue_chunk->start_queue_id = qid; + queue_chunk->num_queues = 1; + + args.ops = on ? VIRTCHNL2_OP_ENABLE_QUEUES : + VIRTCHNL2_OP_DISABLE_QUEUES; + args.in_args = (u8 *)queue_select; + args.in_args_size = len; + args.out_buffer = adapter->mbx_resp; + args.out_size = IDPF_DFLT_MBX_BUF_SIZE; + err = idpf_execute_vc_cmd(adapter, &args); + if (err != 0) + PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES", + on ? "ENABLE" : "DISABLE"); + + rte_free(queue_select); + return err; +} + +int +idpf_switch_queue(struct idpf_vport *vport, uint16_t qid, + bool rx, bool on) +{ + uint32_t type; + int err, queue_id; + + /* switch txq/rxq */ + type = rx ? VIRTCHNL2_QUEUE_TYPE_RX : VIRTCHNL2_QUEUE_TYPE_TX; + + if (type == VIRTCHNL2_QUEUE_TYPE_RX) + queue_id = vport->chunks_info.rx_start_qid + qid; + else + queue_id = vport->chunks_info.tx_start_qid + qid; + err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on); + if (err != 0) + return err; + + /* switch tx completion queue */ + if (!rx && vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) { + type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION; + queue_id = vport->chunks_info.tx_compl_start_qid + qid; + err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on); + if (err != 0) + return err; + } + + /* switch rx buffer queue */ + if (rx && vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) { + type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER; + queue_id = vport->chunks_info.rx_buf_start_qid + 2 * qid; + err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on); + if (err != 0) + return err; + queue_id++; + err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on); + if (err != 0) + return err; + } + + return err; +} + int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable) { From patchwork Mon Oct 31 08:33:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xing, Beilei" X-Patchwork-Id: 119347 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DE17CA00C2; Mon, 31 Oct 2022 10:05:06 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A80CE415D7; Mon, 31 Oct 2022 10:04:27 +0100 (CET) Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id DFF5140151 for ; Mon, 31 Oct 2022 10:04:18 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667207059; x=1698743059; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=y08YtbO4TeqJ2Wo+MBL8MYtTEl/MAjnB0i5EgMvfBEQ=; b=dlFjkV68MZpLlcUfGPjaxcvBp6ANcRZpsSnpESCZVNzKEb4QZ0vppFpu Yr2EOj1mvmsYrJ4JikbGBrHDWG9xYkGRFP1Ob+Jy9stMwCUGPhJAcCW/H kHu8WVqJI7uZngvv64QWRpOOLwlA2E5kYM/eujDM+27Mhthp00vtYjbVm c68qSnoVdXajVyFCBSTTd1YEIK3GOfV9YgNTmju2E2tbk+6Cyt8NMiFPR W+GupZh5SJG3pyc5lBjMGtkuIP1hgtotpvuOWiY4grj57694LtBMuJJlm SndTeSY8FksmIzGg1mx/SL7QDOmgluIoZgpSyS3RPTqs6d1swwcVQNWsx A==; X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="370926307" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="370926307" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Oct 2022 02:04:12 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="878664836" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="878664836" Received: from dpdk-beileix-3.sh.intel.com ([10.67.110.253]) by fmsmga006.fm.intel.com with ESMTP; 31 Oct 2022 02:04:10 -0700 From: beilei.xing@intel.com To: jingjing.wu@intel.com, beilei.xing@intel.com Cc: dev@dpdk.org, Junfeng Guo , Xiaoyun Li Subject: [PATCH v18 07/18] net/idpf: add support for queue stop Date: Mon, 31 Oct 2022 08:33:35 +0000 Message-Id: <20221031083346.16558-8-beilei.xing@intel.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20221031083346.16558-1-beilei.xing@intel.com> References: <20221031051556.98549-1-beilei.xing@intel.com> <20221031083346.16558-1-beilei.xing@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Junfeng Guo Add support for these device ops: - rx_queue_stop - tx_queue_stop Signed-off-by: Beilei Xing Signed-off-by: Xiaoyun Li Signed-off-by: Junfeng Guo --- drivers/net/idpf/idpf_ethdev.c | 17 ++-- drivers/net/idpf/idpf_rxtx.c | 148 +++++++++++++++++++++++++++++++++ drivers/net/idpf/idpf_rxtx.h | 13 +++ drivers/net/idpf/idpf_vchnl.c | 69 +++++++++++++++ 4 files changed, 242 insertions(+), 5 deletions(-) diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c index 0400ed611f..9f1e1e6a18 100644 --- a/drivers/net/idpf/idpf_ethdev.c +++ b/drivers/net/idpf/idpf_ethdev.c @@ -324,7 +324,8 @@ idpf_dev_start(struct rte_eth_dev *dev) if (dev->data->mtu > vport->max_mtu) { PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu); - return -EINVAL; + ret = -EINVAL; + goto err_mtu; } vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD; @@ -332,17 +333,21 @@ idpf_dev_start(struct rte_eth_dev *dev) ret = idpf_start_queues(dev); if (ret != 0) { PMD_DRV_LOG(ERR, "Failed to start queues"); - return ret; + goto err_mtu; } ret = idpf_vc_ena_dis_vport(vport, true); if (ret != 0) { PMD_DRV_LOG(ERR, "Failed to enable vport"); - /* TODO: stop queues */ - return ret; + goto err_vport; } return 0; + +err_vport: + idpf_stop_queues(dev); +err_mtu: + return ret; } static int @@ -352,7 +357,7 @@ idpf_dev_stop(struct rte_eth_dev *dev) idpf_vc_ena_dis_vport(vport, false); - /* TODO: stop queues */ + idpf_stop_queues(dev); return 0; } @@ -751,6 +756,8 @@ static const struct eth_dev_ops idpf_eth_dev_ops = { .link_update = idpf_dev_link_update, .rx_queue_start = idpf_rx_queue_start, .tx_queue_start = idpf_tx_queue_start, + .rx_queue_stop = idpf_rx_queue_stop, + .tx_queue_stop = idpf_tx_queue_stop, }; static uint16_t diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c index 6d954afd9d..8d5ec41a1f 100644 --- a/drivers/net/idpf/idpf_rxtx.c +++ b/drivers/net/idpf/idpf_rxtx.c @@ -71,6 +71,55 @@ check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh, return 0; } +static void +release_rxq_mbufs(struct idpf_rx_queue *rxq) +{ + uint16_t i; + + if (rxq->sw_ring == NULL) + return; + + for (i = 0; i < rxq->nb_rx_desc; i++) { + if (rxq->sw_ring[i] != NULL) { + rte_pktmbuf_free_seg(rxq->sw_ring[i]); + rxq->sw_ring[i] = NULL; + } + } +} + +static void +release_txq_mbufs(struct idpf_tx_queue *txq) +{ + uint16_t nb_desc, i; + + if (txq == NULL || txq->sw_ring == NULL) { + PMD_DRV_LOG(DEBUG, "Pointer to rxq or sw_ring is NULL"); + return; + } + + if (txq->sw_nb_desc != 0) { + /* For split queue model, descriptor ring */ + nb_desc = txq->sw_nb_desc; + } else { + /* For single queue model */ + nb_desc = txq->nb_tx_desc; + } + for (i = 0; i < nb_desc; i++) { + if (txq->sw_ring[i].mbuf != NULL) { + rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf); + txq->sw_ring[i].mbuf = NULL; + } + } +} + +static const struct idpf_rxq_ops def_rxq_ops = { + .release_mbufs = release_rxq_mbufs, +}; + +static const struct idpf_txq_ops def_txq_ops = { + .release_mbufs = release_txq_mbufs, +}; + static void reset_split_rx_descq(struct idpf_rx_queue *rxq) { @@ -122,6 +171,14 @@ reset_split_rx_bufq(struct idpf_rx_queue *rxq) rxq->bufq2 = NULL; } +static inline void +reset_split_rx_queue(struct idpf_rx_queue *rxq) +{ + reset_split_rx_descq(rxq); + reset_split_rx_bufq(rxq->bufq1); + reset_split_rx_bufq(rxq->bufq2); +} + static void reset_single_rx_queue(struct idpf_rx_queue *rxq) { @@ -301,6 +358,7 @@ idpf_rx_split_bufq_setup(struct rte_eth_dev *dev, struct idpf_rx_queue *bufq, bufq->q_set = true; bufq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_buf_qtail_start + queue_idx * vport->chunks_info.rx_buf_qtail_spacing); + bufq->ops = &def_rxq_ops; /* TODO: allow bulk or vec */ @@ -527,6 +585,7 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, dev->data->rx_queues[queue_idx] = rxq; rxq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_qtail_start + queue_idx * vport->chunks_info.rx_qtail_spacing); + rxq->ops = &def_rxq_ops; return 0; } @@ -621,6 +680,7 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, reset_split_tx_descq(txq); txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start + queue_idx * vport->chunks_info.tx_qtail_spacing); + txq->ops = &def_txq_ops; /* Allocate the TX completion queue data structure. */ txq->complq = rte_zmalloc_socket("idpf splitq cq", @@ -748,6 +808,7 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, dev->data->tx_queues[queue_idx] = txq; txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start + queue_idx * vport->chunks_info.tx_qtail_spacing); + txq->ops = &def_txq_ops; return 0; } @@ -979,3 +1040,90 @@ idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id) return err; } + +int +idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id) +{ + struct idpf_vport *vport = dev->data->dev_private; + struct idpf_rx_queue *rxq; + int err; + + if (rx_queue_id >= dev->data->nb_rx_queues) + return -EINVAL; + + err = idpf_switch_queue(vport, rx_queue_id, true, false); + if (err != 0) { + PMD_DRV_LOG(ERR, "Failed to switch RX queue %u off", + rx_queue_id); + return err; + } + + rxq = dev->data->rx_queues[rx_queue_id]; + if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) { + rxq->ops->release_mbufs(rxq); + reset_single_rx_queue(rxq); + } else { + rxq->bufq1->ops->release_mbufs(rxq->bufq1); + rxq->bufq2->ops->release_mbufs(rxq->bufq2); + reset_split_rx_queue(rxq); + } + dev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED; + + return 0; +} + +int +idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id) +{ + struct idpf_vport *vport = dev->data->dev_private; + struct idpf_tx_queue *txq; + int err; + + if (tx_queue_id >= dev->data->nb_tx_queues) + return -EINVAL; + + err = idpf_switch_queue(vport, tx_queue_id, false, false); + if (err != 0) { + PMD_DRV_LOG(ERR, "Failed to switch TX queue %u off", + tx_queue_id); + return err; + } + + txq = dev->data->tx_queues[tx_queue_id]; + txq->ops->release_mbufs(txq); + if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) { + reset_single_tx_queue(txq); + } else { + reset_split_tx_descq(txq); + reset_split_tx_complq(txq->complq); + } + dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED; + + return 0; +} + +void +idpf_stop_queues(struct rte_eth_dev *dev) +{ + struct idpf_rx_queue *rxq; + struct idpf_tx_queue *txq; + int i; + + for (i = 0; i < dev->data->nb_rx_queues; i++) { + rxq = dev->data->rx_queues[i]; + if (rxq == NULL) + continue; + + if (idpf_rx_queue_stop(dev, i) != 0) + PMD_DRV_LOG(WARNING, "Fail to stop Rx queue %d", i); + } + + for (i = 0; i < dev->data->nb_tx_queues; i++) { + txq = dev->data->tx_queues[i]; + if (txq == NULL) + continue; + + if (idpf_tx_queue_stop(dev, i) != 0) + PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i); + } +} diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h index ab9b3830fd..b959638489 100644 --- a/drivers/net/idpf/idpf_rxtx.h +++ b/drivers/net/idpf/idpf_rxtx.h @@ -51,6 +51,7 @@ struct idpf_rx_queue { bool q_set; /* if rx queue has been configured */ bool q_started; /* if rx queue has been started */ bool rx_deferred_start; /* don't start this queue in dev start */ + const struct idpf_rxq_ops *ops; /* only valid for split queue mode */ uint8_t expected_gen_id; @@ -97,6 +98,7 @@ struct idpf_tx_queue { bool q_set; /* if tx queue has been configured */ bool q_started; /* if tx queue has been started */ bool tx_deferred_start; /* don't start this queue in dev start */ + const struct idpf_txq_ops *ops; /* only valid for split queue mode */ uint16_t sw_nb_desc; @@ -107,16 +109,27 @@ struct idpf_tx_queue { struct idpf_tx_queue *complq; }; +struct idpf_rxq_ops { + void (*release_mbufs)(struct idpf_rx_queue *rxq); +}; + +struct idpf_txq_ops { + void (*release_mbufs)(struct idpf_tx_queue *txq); +}; + int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, uint16_t nb_desc, unsigned int socket_id, const struct rte_eth_rxconf *rx_conf, struct rte_mempool *mp); int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id); int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id); +int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id); int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, uint16_t nb_desc, unsigned int socket_id, const struct rte_eth_txconf *tx_conf); int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id); int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id); +int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id); +void idpf_stop_queues(struct rte_eth_dev *dev); #endif /* _IDPF_RXTX_H_ */ diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c index 08fb9061c2..f92141b54f 100644 --- a/drivers/net/idpf/idpf_vchnl.c +++ b/drivers/net/idpf/idpf_vchnl.c @@ -837,6 +837,75 @@ idpf_switch_queue(struct idpf_vport *vport, uint16_t qid, return err; } +#define IDPF_RXTX_QUEUE_CHUNKS_NUM 2 +int +idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable) +{ + struct idpf_adapter *adapter = vport->adapter; + struct virtchnl2_del_ena_dis_queues *queue_select; + struct virtchnl2_queue_chunk *queue_chunk; + uint32_t type; + struct idpf_cmd_info args; + uint16_t num_chunks; + int err, len; + + num_chunks = IDPF_RXTX_QUEUE_CHUNKS_NUM; + if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) + num_chunks++; + if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) + num_chunks++; + + len = sizeof(struct virtchnl2_del_ena_dis_queues) + + sizeof(struct virtchnl2_queue_chunk) * (num_chunks - 1); + queue_select = rte_zmalloc("queue_select", len, 0); + if (queue_select == NULL) + return -ENOMEM; + + queue_chunk = queue_select->chunks.chunks; + queue_select->chunks.num_chunks = num_chunks; + queue_select->vport_id = vport->vport_id; + + type = VIRTCHNL_QUEUE_TYPE_RX; + queue_chunk[type].type = type; + queue_chunk[type].start_queue_id = vport->chunks_info.rx_start_qid; + queue_chunk[type].num_queues = vport->num_rx_q; + + type = VIRTCHNL2_QUEUE_TYPE_TX; + queue_chunk[type].type = type; + queue_chunk[type].start_queue_id = vport->chunks_info.tx_start_qid; + queue_chunk[type].num_queues = vport->num_tx_q; + + if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) { + type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER; + queue_chunk[type].type = type; + queue_chunk[type].start_queue_id = + vport->chunks_info.rx_buf_start_qid; + queue_chunk[type].num_queues = vport->num_rx_bufq; + } + + if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) { + type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION; + queue_chunk[type].type = type; + queue_chunk[type].start_queue_id = + vport->chunks_info.tx_compl_start_qid; + queue_chunk[type].num_queues = vport->num_tx_complq; + } + + args.ops = enable ? VIRTCHNL2_OP_ENABLE_QUEUES : + VIRTCHNL2_OP_DISABLE_QUEUES; + args.in_args = (u8 *)queue_select; + args.in_args_size = len; + args.out_buffer = adapter->mbx_resp; + args.out_size = IDPF_DFLT_MBX_BUF_SIZE; + err = idpf_execute_vc_cmd(adapter, &args); + if (err != 0) + PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES", + enable ? "ENABLE" : "DISABLE"); + + rte_free(queue_select); + return err; +} + int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable) { From patchwork Mon Oct 31 08:33:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xing, Beilei" X-Patchwork-Id: 119349 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4B337A00C2; Mon, 31 Oct 2022 10:05:23 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 48CD842802; Mon, 31 Oct 2022 10:04:29 +0100 (CET) Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id 89C5F40695 for ; Mon, 31 Oct 2022 10:04:19 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667207059; x=1698743059; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2zEfZ2NguX4B8X2zR8hct6OWu/kXWrDb9rA/yKLvToQ=; b=dUz0STIHDhNzTl01s7As1CzyzIenaLG7aYTV+oAyfLFb8PBviLZFiNYU voy7ORXKIXjjrqL4dBkRt12832jcMd8EshcKSytCtKnUC5v41YhSyU2WI fzMiliHUj6j4R/noD9Qyz9Foylw5SnsnrimsgF3GiNTVOYYXzzdOFkniO jAdPaftQ4ZUcgiAJizxga5hc9OxexP33aFjytGFYRCoHktS70G/yf4ilD bkHURFP2GI0cAujmGZ+Co7UHNw7jtM9vrNIGjyNBFSAGnrt9S8DStGL+V 1hmsLfd5U9kmT90PrVMLMsJUGFqbDg3d5wgMbeThZeWBeaW0NXJaoyDF5 A==; X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="370926317" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="370926317" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Oct 2022 02:04:14 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="878664851" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="878664851" Received: from dpdk-beileix-3.sh.intel.com ([10.67.110.253]) by fmsmga006.fm.intel.com with ESMTP; 31 Oct 2022 02:04:12 -0700 From: beilei.xing@intel.com To: jingjing.wu@intel.com, beilei.xing@intel.com Cc: dev@dpdk.org, Junfeng Guo , Xiaoyun Li Subject: [PATCH v18 08/18] net/idpf: add queue release Date: Mon, 31 Oct 2022 08:33:36 +0000 Message-Id: <20221031083346.16558-9-beilei.xing@intel.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20221031083346.16558-1-beilei.xing@intel.com> References: <20221031051556.98549-1-beilei.xing@intel.com> <20221031083346.16558-1-beilei.xing@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Junfeng Guo Add support for queue operations: - rx_queue_release - tx_queue_release Signed-off-by: Beilei Xing Signed-off-by: Xiaoyun Li Signed-off-by: Junfeng Guo --- drivers/net/idpf/idpf_ethdev.c | 2 + drivers/net/idpf/idpf_rxtx.c | 81 ++++++++++++++++++++++++++++++++++ drivers/net/idpf/idpf_rxtx.h | 3 ++ 3 files changed, 86 insertions(+) diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c index 9f1e1e6a18..1485f40e71 100644 --- a/drivers/net/idpf/idpf_ethdev.c +++ b/drivers/net/idpf/idpf_ethdev.c @@ -758,6 +758,8 @@ static const struct eth_dev_ops idpf_eth_dev_ops = { .tx_queue_start = idpf_tx_queue_start, .rx_queue_stop = idpf_rx_queue_stop, .tx_queue_stop = idpf_tx_queue_stop, + .rx_queue_release = idpf_dev_rx_queue_release, + .tx_queue_release = idpf_dev_tx_queue_release, }; static uint16_t diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c index 8d5ec41a1f..053409b99a 100644 --- a/drivers/net/idpf/idpf_rxtx.c +++ b/drivers/net/idpf/idpf_rxtx.c @@ -171,6 +171,51 @@ reset_split_rx_bufq(struct idpf_rx_queue *rxq) rxq->bufq2 = NULL; } +static void +idpf_rx_queue_release(void *rxq) +{ + struct idpf_rx_queue *q = rxq; + + if (q == NULL) + return; + + /* Split queue */ + if (q->bufq1 != NULL && q->bufq2 != NULL) { + q->bufq1->ops->release_mbufs(q->bufq1); + rte_free(q->bufq1->sw_ring); + rte_memzone_free(q->bufq1->mz); + rte_free(q->bufq1); + q->bufq2->ops->release_mbufs(q->bufq2); + rte_free(q->bufq2->sw_ring); + rte_memzone_free(q->bufq2->mz); + rte_free(q->bufq2); + rte_memzone_free(q->mz); + rte_free(q); + return; + } + + /* Single queue */ + q->ops->release_mbufs(q); + rte_free(q->sw_ring); + rte_memzone_free(q->mz); + rte_free(q); +} + +static void +idpf_tx_queue_release(void *txq) +{ + struct idpf_tx_queue *q = txq; + + if (q == NULL) + return; + + rte_free(q->complq); + q->ops->release_mbufs(q); + rte_free(q->sw_ring); + rte_memzone_free(q->mz); + rte_free(q); +} + static inline void reset_split_rx_queue(struct idpf_rx_queue *rxq) { @@ -392,6 +437,12 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, if (check_rx_thresh(nb_desc, rx_free_thresh) != 0) return -EINVAL; + /* Free memory if needed */ + if (dev->data->rx_queues[queue_idx] != NULL) { + idpf_rx_queue_release(dev->data->rx_queues[queue_idx]); + dev->data->rx_queues[queue_idx] = NULL; + } + /* Setup Rx description queue */ rxq = rte_zmalloc_socket("idpf rxq", sizeof(struct idpf_rx_queue), @@ -524,6 +575,12 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, if (check_rx_thresh(nb_desc, rx_free_thresh) != 0) return -EINVAL; + /* Free memory if needed */ + if (dev->data->rx_queues[queue_idx] != NULL) { + idpf_rx_queue_release(dev->data->rx_queues[queue_idx]); + dev->data->rx_queues[queue_idx] = NULL; + } + /* Setup Rx description queue */ rxq = rte_zmalloc_socket("idpf rxq", sizeof(struct idpf_rx_queue), @@ -630,6 +687,12 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0) return -EINVAL; + /* Free memory if needed. */ + if (dev->data->tx_queues[queue_idx] != NULL) { + idpf_tx_queue_release(dev->data->tx_queues[queue_idx]); + dev->data->tx_queues[queue_idx] = NULL; + } + /* Allocate the TX queue data structure. */ txq = rte_zmalloc_socket("idpf split txq", sizeof(struct idpf_tx_queue), @@ -754,6 +817,12 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0) return -EINVAL; + /* Free memory if needed. */ + if (dev->data->tx_queues[queue_idx] != NULL) { + idpf_tx_queue_release(dev->data->tx_queues[queue_idx]); + dev->data->tx_queues[queue_idx] = NULL; + } + /* Allocate the TX queue data structure. */ txq = rte_zmalloc_socket("idpf txq", sizeof(struct idpf_tx_queue), @@ -1102,6 +1171,18 @@ idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id) return 0; } +void +idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid) +{ + idpf_rx_queue_release(dev->data->rx_queues[qid]); +} + +void +idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid) +{ + idpf_tx_queue_release(dev->data->tx_queues[qid]); +} + void idpf_stop_queues(struct rte_eth_dev *dev) { diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h index b959638489..37944edcac 100644 --- a/drivers/net/idpf/idpf_rxtx.h +++ b/drivers/net/idpf/idpf_rxtx.h @@ -124,12 +124,15 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id); int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id); int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id); +void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid); + int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, uint16_t nb_desc, unsigned int socket_id, const struct rte_eth_txconf *tx_conf); int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id); int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id); int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id); +void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid); void idpf_stop_queues(struct rte_eth_dev *dev); #endif /* _IDPF_RXTX_H_ */ From patchwork Mon Oct 31 08:33:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xing, Beilei" X-Patchwork-Id: 119348 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D26DAA00C2; Mon, 31 Oct 2022 10:05:14 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 82CD1427F0; Mon, 31 Oct 2022 10:04:28 +0100 (CET) Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id 3264D4069C for ; Mon, 31 Oct 2022 10:04:19 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667207059; x=1698743059; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=FClQ5w/Mk2hiM5LjmdmDKQGn+U+zyfYc8YYh1s1BujM=; b=R9pMxO+5cVtZ1V/TYcqDhWkkqHhv1wTrbSFI+lGW7wUJP/L8Mpg6P2so uh/HC0z+9YJugvw1sp8lThMfnnedA//+3gCNg0jC29a2YBgaz2/00vGFC fp2R843ZX6YlqUCq/r0/5W1/yNRt0Mr2/84lMUYG9I8A6CiEdLWHSt0Ju H6vvAJOfss+9QwR9LPmSCjanGNWN2n54qBZ7CO8IqHSmur+xEcZvAimO9 R/jvVU6RqrP0UK8FIap4xDOt4lBKJGsUOr8UR9NfRxu665kzf80VQKVCx 7ZV7aJ/HN0pgSfwLdwAvLbB43/cL7Md2ca/0qDCiaNZGhef7ewKanqLlj Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="370926324" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="370926324" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Oct 2022 02:04:15 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="878664859" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="878664859" Received: from dpdk-beileix-3.sh.intel.com ([10.67.110.253]) by fmsmga006.fm.intel.com with ESMTP; 31 Oct 2022 02:04:13 -0700 From: beilei.xing@intel.com To: jingjing.wu@intel.com, beilei.xing@intel.com Cc: dev@dpdk.org, Junfeng Guo Subject: [PATCH v18 09/18] net/idpf: add support for MTU configuration Date: Mon, 31 Oct 2022 08:33:37 +0000 Message-Id: <20221031083346.16558-10-beilei.xing@intel.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20221031083346.16558-1-beilei.xing@intel.com> References: <20221031051556.98549-1-beilei.xing@intel.com> <20221031083346.16558-1-beilei.xing@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Junfeng Guo Add dev ops mtu_set. Signed-off-by: Beilei Xing Signed-off-by: Junfeng Guo --- doc/guides/nics/features/idpf.ini | 1 + drivers/net/idpf/idpf_ethdev.c | 13 +++++++++++++ 2 files changed, 14 insertions(+) diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini index 46aab2eb61..d722c49fde 100644 --- a/doc/guides/nics/features/idpf.ini +++ b/doc/guides/nics/features/idpf.ini @@ -4,6 +4,7 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +MTU update = Y Linux = Y x86-32 = Y x86-64 = Y diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c index 1485f40e71..856f3d7266 100644 --- a/drivers/net/idpf/idpf_ethdev.c +++ b/drivers/net/idpf/idpf_ethdev.c @@ -83,6 +83,18 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) return 0; } +static int +idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu __rte_unused) +{ + /* mtu setting is forbidden if port is start */ + if (dev->data->dev_started) { + PMD_DRV_LOG(ERR, "port must be stopped before configuration"); + return -EBUSY; + } + + return 0; +} + static int idpf_init_vport_req_info(struct rte_eth_dev *dev) { @@ -760,6 +772,7 @@ static const struct eth_dev_ops idpf_eth_dev_ops = { .tx_queue_stop = idpf_tx_queue_stop, .rx_queue_release = idpf_dev_rx_queue_release, .tx_queue_release = idpf_dev_tx_queue_release, + .mtu_set = idpf_dev_mtu_set, }; static uint16_t From patchwork Mon Oct 31 08:33:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xing, Beilei" X-Patchwork-Id: 119350 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E5DB6A00C2; Mon, 31 Oct 2022 10:05:30 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 334F04280E; Mon, 31 Oct 2022 10:04:30 +0100 (CET) Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id BE28540151 for ; Mon, 31 Oct 2022 10:04:19 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667207059; x=1698743059; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=X5780iKEyX4idL1oA45PI2Ib31V4TojLXr3JkPW1CSA=; b=SW/uUMCJbv8D8EVqjdIG1GInuckBWp4nLgGGaGYJlxbFMRHkYN4cmDsG T455A0GcgEfS8Yos3ZMn7M0y3bcsjIUsiTcR1y/RYPvkffUFFngH3TNFg PDcXcfw/pzyx+VdmoUB2Dih7D0vugtcdbmJ2OMo7YnwRM1B45MahsZZ42 SVaB6szafKeFC+WMw2AfAUY3nE4ezwMqZAOiGnJBpLMnSda57J74aYMzB 5Y4VPjqUqaY1LjzYwjcNqWlU2kQKG6vlL4EKj5MqNwzlupHQ50MR7XmIQ T8XyFYz7dC9UIeU9mBY4+k3448lW345tJS4GacU1pvXAv45pPh2/ns5RQ g==; X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="370926325" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="370926325" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Oct 2022 02:04:17 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="878664869" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="878664869" Received: from dpdk-beileix-3.sh.intel.com ([10.67.110.253]) by fmsmga006.fm.intel.com with ESMTP; 31 Oct 2022 02:04:15 -0700 From: beilei.xing@intel.com To: jingjing.wu@intel.com, beilei.xing@intel.com Cc: dev@dpdk.org, Junfeng Guo , Xiaoyun Li Subject: [PATCH v18 10/18] net/idpf: add support for basic Rx datapath Date: Mon, 31 Oct 2022 08:33:38 +0000 Message-Id: <20221031083346.16558-11-beilei.xing@intel.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20221031083346.16558-1-beilei.xing@intel.com> References: <20221031051556.98549-1-beilei.xing@intel.com> <20221031083346.16558-1-beilei.xing@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Junfeng Guo Add basic Rx support in split queue mode and single queue mode. Signed-off-by: Beilei Xing Signed-off-by: Xiaoyun Li Signed-off-by: Junfeng Guo --- drivers/net/idpf/idpf_ethdev.c | 2 + drivers/net/idpf/idpf_rxtx.c | 273 +++++++++++++++++++++++++++++++++ drivers/net/idpf/idpf_rxtx.h | 7 +- 3 files changed, 281 insertions(+), 1 deletion(-) diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c index 856f3d7266..2f1f9eeee5 100644 --- a/drivers/net/idpf/idpf_ethdev.c +++ b/drivers/net/idpf/idpf_ethdev.c @@ -348,6 +348,8 @@ idpf_dev_start(struct rte_eth_dev *dev) goto err_mtu; } + idpf_set_rx_function(dev); + ret = idpf_vc_ena_dis_vport(vport, true); if (ret != 0) { PMD_DRV_LOG(ERR, "Failed to enable vport"); diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c index 053409b99a..ea499c4d37 100644 --- a/drivers/net/idpf/idpf_rxtx.c +++ b/drivers/net/idpf/idpf_rxtx.c @@ -1208,3 +1208,276 @@ idpf_stop_queues(struct rte_eth_dev *dev) PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i); } } + +static void +idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq) +{ + volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_ring; + volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_desc; + uint16_t nb_refill = rx_bufq->rx_free_thresh; + uint16_t nb_desc = rx_bufq->nb_rx_desc; + uint16_t next_avail = rx_bufq->rx_tail; + struct rte_mbuf *nmb[rx_bufq->rx_free_thresh]; + struct rte_eth_dev *dev; + uint64_t dma_addr; + uint16_t delta; + int i; + + if (rx_bufq->nb_rx_hold < rx_bufq->rx_free_thresh) + return; + + rx_buf_ring = rx_bufq->rx_ring; + delta = nb_desc - next_avail; + if (unlikely(delta < nb_refill)) { + if (likely(rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, delta) == 0)) { + for (i = 0; i < delta; i++) { + rx_buf_desc = &rx_buf_ring[next_avail + i]; + rx_bufq->sw_ring[next_avail + i] = nmb[i]; + dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i])); + rx_buf_desc->hdr_addr = 0; + rx_buf_desc->pkt_addr = dma_addr; + } + nb_refill -= delta; + next_avail = 0; + rx_bufq->nb_rx_hold -= delta; + } else { + dev = &rte_eth_devices[rx_bufq->port_id]; + dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail; + PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u", + rx_bufq->port_id, rx_bufq->queue_id); + return; + } + } + + if (nb_desc - next_avail >= nb_refill) { + if (likely(rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, nb_refill) == 0)) { + for (i = 0; i < nb_refill; i++) { + rx_buf_desc = &rx_buf_ring[next_avail + i]; + rx_bufq->sw_ring[next_avail + i] = nmb[i]; + dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i])); + rx_buf_desc->hdr_addr = 0; + rx_buf_desc->pkt_addr = dma_addr; + } + next_avail += nb_refill; + rx_bufq->nb_rx_hold -= nb_refill; + } else { + dev = &rte_eth_devices[rx_bufq->port_id]; + dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail; + PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u", + rx_bufq->port_id, rx_bufq->queue_id); + } + } + + IDPF_PCI_REG_WRITE(rx_bufq->qrx_tail, next_avail); + + rx_bufq->rx_tail = next_avail; +} + +uint16_t +idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t nb_pkts) +{ + volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc_ring; + volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc; + uint16_t pktlen_gen_bufq_id; + struct idpf_rx_queue *rxq; + struct rte_mbuf *rxm; + uint16_t rx_id_bufq1; + uint16_t rx_id_bufq2; + uint16_t pkt_len; + uint16_t bufq_id; + uint16_t gen_id; + uint16_t rx_id; + uint16_t nb_rx; + + nb_rx = 0; + rxq = rx_queue; + + if (unlikely(rxq == NULL) || unlikely(!rxq->q_started)) + return nb_rx; + + rx_id = rxq->rx_tail; + rx_id_bufq1 = rxq->bufq1->rx_next_avail; + rx_id_bufq2 = rxq->bufq2->rx_next_avail; + rx_desc_ring = rxq->rx_ring; + + while (nb_rx < nb_pkts) { + rx_desc = &rx_desc_ring[rx_id]; + + pktlen_gen_bufq_id = + rte_le_to_cpu_16(rx_desc->pktlen_gen_bufq_id); + gen_id = (pktlen_gen_bufq_id & + VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) >> + VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S; + if (gen_id != rxq->expected_gen_id) + break; + + pkt_len = (pktlen_gen_bufq_id & + VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M) >> + VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S; + if (pkt_len == 0) + PMD_RX_LOG(ERR, "Packet length is 0"); + + rx_id++; + if (unlikely(rx_id == rxq->nb_rx_desc)) { + rx_id = 0; + rxq->expected_gen_id ^= 1; + } + + bufq_id = (pktlen_gen_bufq_id & + VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M) >> + VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S; + if (bufq_id == 0) { + rxm = rxq->bufq1->sw_ring[rx_id_bufq1]; + rx_id_bufq1++; + if (unlikely(rx_id_bufq1 == rxq->bufq1->nb_rx_desc)) + rx_id_bufq1 = 0; + rxq->bufq1->nb_rx_hold++; + } else { + rxm = rxq->bufq2->sw_ring[rx_id_bufq2]; + rx_id_bufq2++; + if (unlikely(rx_id_bufq2 == rxq->bufq2->nb_rx_desc)) + rx_id_bufq2 = 0; + rxq->bufq2->nb_rx_hold++; + } + + rxm->pkt_len = pkt_len; + rxm->data_len = pkt_len; + rxm->data_off = RTE_PKTMBUF_HEADROOM; + rxm->next = NULL; + rxm->nb_segs = 1; + rxm->port = rxq->port_id; + + rx_pkts[nb_rx++] = rxm; + } + + if (nb_rx > 0) { + rxq->rx_tail = rx_id; + if (rx_id_bufq1 != rxq->bufq1->rx_next_avail) + rxq->bufq1->rx_next_avail = rx_id_bufq1; + if (rx_id_bufq2 != rxq->bufq2->rx_next_avail) + rxq->bufq2->rx_next_avail = rx_id_bufq2; + + idpf_split_rx_bufq_refill(rxq->bufq1); + idpf_split_rx_bufq_refill(rxq->bufq2); + } + + return nb_rx; +} + +static inline void +idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold, + uint16_t rx_id) +{ + nb_hold = (uint16_t)(nb_hold + rxq->nb_rx_hold); + + if (nb_hold > rxq->rx_free_thresh) { + PMD_RX_LOG(DEBUG, + "port_id=%u queue_id=%u rx_tail=%u nb_hold=%u", + rxq->port_id, rxq->queue_id, rx_id, nb_hold); + rx_id = (uint16_t)((rx_id == 0) ? + (rxq->nb_rx_desc - 1) : (rx_id - 1)); + IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id); + nb_hold = 0; + } + rxq->nb_rx_hold = nb_hold; +} + +uint16_t +idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t nb_pkts) +{ + volatile union virtchnl2_rx_desc *rx_ring; + volatile union virtchnl2_rx_desc *rxdp; + union virtchnl2_rx_desc rxd; + struct idpf_rx_queue *rxq; + uint16_t rx_id, nb_hold; + struct rte_eth_dev *dev; + uint16_t rx_packet_len; + struct rte_mbuf *rxm; + struct rte_mbuf *nmb; + uint16_t rx_status0; + uint64_t dma_addr; + uint16_t nb_rx; + + nb_rx = 0; + nb_hold = 0; + rxq = rx_queue; + + if (unlikely(rxq == NULL) || unlikely(!rxq->q_started)) + return nb_rx; + + rx_id = rxq->rx_tail; + rx_ring = rxq->rx_ring; + + while (nb_rx < nb_pkts) { + rxdp = &rx_ring[rx_id]; + rx_status0 = rte_le_to_cpu_16(rxdp->flex_nic_wb.status_error0); + + /* Check the DD bit first */ + if ((rx_status0 & (1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)) == 0) + break; + + nmb = rte_mbuf_raw_alloc(rxq->mp); + if (unlikely(nmb == NULL)) { + dev = &rte_eth_devices[rxq->port_id]; + dev->data->rx_mbuf_alloc_failed++; + PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u " + "queue_id=%u", rxq->port_id, rxq->queue_id); + break; + } + rxd = *rxdp; /* copy descriptor in ring to temp variable*/ + + nb_hold++; + rxm = rxq->sw_ring[rx_id]; + rxq->sw_ring[rx_id] = nmb; + rx_id++; + if (unlikely(rx_id == rxq->nb_rx_desc)) + rx_id = 0; + + /* Prefetch next mbuf */ + rte_prefetch0(rxq->sw_ring[rx_id]); + + /* When next RX descriptor is on a cache line boundary, + * prefetch the next 4 RX descriptors and next 8 pointers + * to mbufs. + */ + if ((rx_id & 0x3) == 0) { + rte_prefetch0(&rx_ring[rx_id]); + rte_prefetch0(rxq->sw_ring[rx_id]); + } + dma_addr = + rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb)); + rxdp->read.hdr_addr = 0; + rxdp->read.pkt_addr = dma_addr; + + rx_packet_len = (rte_cpu_to_le_16(rxd.flex_nic_wb.pkt_len) & + VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M); + + rxm->data_off = RTE_PKTMBUF_HEADROOM; + rte_prefetch0(RTE_PTR_ADD(rxm->buf_addr, RTE_PKTMBUF_HEADROOM)); + rxm->nb_segs = 1; + rxm->next = NULL; + rxm->pkt_len = rx_packet_len; + rxm->data_len = rx_packet_len; + rxm->port = rxq->port_id; + + rx_pkts[nb_rx++] = rxm; + } + rxq->rx_tail = rx_id; + + idpf_update_rx_tail(rxq, nb_hold, rx_id); + + return nb_rx; +} + +void +idpf_set_rx_function(struct rte_eth_dev *dev) +{ + struct idpf_vport *vport = dev->data->dev_private; + + if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) + dev->rx_pkt_burst = idpf_splitq_recv_pkts; + else + dev->rx_pkt_burst = idpf_singleq_recv_pkts; +} diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h index 37944edcac..650c6c1c3a 100644 --- a/drivers/net/idpf/idpf_rxtx.h +++ b/drivers/net/idpf/idpf_rxtx.h @@ -133,6 +133,11 @@ int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id); int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id); int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id); void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid); - +uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t nb_pkts); +uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t nb_pkts); void idpf_stop_queues(struct rte_eth_dev *dev); + +void idpf_set_rx_function(struct rte_eth_dev *dev); #endif /* _IDPF_RXTX_H_ */ From patchwork Mon Oct 31 08:33:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xing, Beilei" X-Patchwork-Id: 119351 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A417EA00C2; Mon, 31 Oct 2022 10:05:38 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 216E642826; Mon, 31 Oct 2022 10:04:31 +0100 (CET) Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id 048FB40223 for ; Mon, 31 Oct 2022 10:04:19 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667207060; x=1698743060; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=kWafxfNArycwneEVpS/bi/yBwHh0WHmimCWzkKGLcW4=; b=c6MPjcT+thMuHAnFATqjsZOmz4yOgTFCN0s/S2X0inEqe96f8wjrOoMq H6smIJkWJdE7NkZe+6NHpVFpwJYO0hdvi7p/pfrZJQnhaGhY8vzEv50mP xJ9bZkeHoRN3tVCm3o56UU7NKlIxCPSt/nIc+32JSbFRBbrVvHReW8FxB mtPfAkka4NjLcr3uuH3m3sj5jnCrpf7Nx3QokvDd8rkBsCfBFPMzAi4hl zlVXMAUv+LMRYtXQnf6dqG1gFfMlDwonphSOr12QZWIoiIZWoZVthFdde Mp7MEI1QRVva02p1wIYVUDJRvp+B8qa2EmeSgsNHBk6dWcp3CuVXQw6kH A==; X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="370926333" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="370926333" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Oct 2022 02:04:19 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="878664877" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="878664877" Received: from dpdk-beileix-3.sh.intel.com ([10.67.110.253]) by fmsmga006.fm.intel.com with ESMTP; 31 Oct 2022 02:04:17 -0700 From: beilei.xing@intel.com To: jingjing.wu@intel.com, beilei.xing@intel.com Cc: dev@dpdk.org, Junfeng Guo , Xiaoyun Li Subject: [PATCH v18 11/18] net/idpf: add support for basic Tx datapath Date: Mon, 31 Oct 2022 08:33:39 +0000 Message-Id: <20221031083346.16558-12-beilei.xing@intel.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20221031083346.16558-1-beilei.xing@intel.com> References: <20221031051556.98549-1-beilei.xing@intel.com> <20221031083346.16558-1-beilei.xing@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Junfeng Guo Add basic Tx support in split queue mode and single queue mode. Signed-off-by: Beilei Xing Signed-off-by: Xiaoyun Li Signed-off-by: Junfeng Guo --- drivers/net/idpf/idpf_ethdev.c | 3 + drivers/net/idpf/idpf_ethdev.h | 1 + drivers/net/idpf/idpf_rxtx.c | 357 +++++++++++++++++++++++++++++++++ drivers/net/idpf/idpf_rxtx.h | 10 + 4 files changed, 371 insertions(+) diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c index 2f1f9eeee5..f9f6fe1162 100644 --- a/drivers/net/idpf/idpf_ethdev.c +++ b/drivers/net/idpf/idpf_ethdev.c @@ -59,6 +59,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD; dev_info->min_mtu = RTE_ETHER_MIN_MTU; + dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS; + dev_info->default_txconf = (struct rte_eth_txconf) { .tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH, .tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH, @@ -349,6 +351,7 @@ idpf_dev_start(struct rte_eth_dev *dev) } idpf_set_rx_function(dev); + idpf_set_tx_function(dev); ret = idpf_vc_ena_dis_vport(vport, true); if (ret != 0) { diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h index 96c22009e9..af0a8e2970 100644 --- a/drivers/net/idpf/idpf_ethdev.h +++ b/drivers/net/idpf/idpf_ethdev.h @@ -35,6 +35,7 @@ #define IDPF_MIN_BUF_SIZE 1024 #define IDPF_MAX_FRAME_SIZE 9728 +#define IDPF_MIN_FRAME_SIZE 14 #define IDPF_NUM_MACADDR_MAX 64 diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c index ea499c4d37..f55d2143b9 100644 --- a/drivers/net/idpf/idpf_rxtx.c +++ b/drivers/net/idpf/idpf_rxtx.c @@ -1365,6 +1365,148 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, return nb_rx; } +static inline void +idpf_split_tx_free(struct idpf_tx_queue *cq) +{ + volatile struct idpf_splitq_tx_compl_desc *compl_ring = cq->compl_ring; + volatile struct idpf_splitq_tx_compl_desc *txd; + uint16_t next = cq->tx_tail; + struct idpf_tx_entry *txe; + struct idpf_tx_queue *txq; + uint16_t gen, qid, q_head; + uint8_t ctype; + + txd = &compl_ring[next]; + gen = (rte_le_to_cpu_16(txd->qid_comptype_gen) & + IDPF_TXD_COMPLQ_GEN_M) >> IDPF_TXD_COMPLQ_GEN_S; + if (gen != cq->expected_gen_id) + return; + + ctype = (rte_le_to_cpu_16(txd->qid_comptype_gen) & + IDPF_TXD_COMPLQ_COMPL_TYPE_M) >> IDPF_TXD_COMPLQ_COMPL_TYPE_S; + qid = (rte_le_to_cpu_16(txd->qid_comptype_gen) & + IDPF_TXD_COMPLQ_QID_M) >> IDPF_TXD_COMPLQ_QID_S; + q_head = rte_le_to_cpu_16(txd->q_head_compl_tag.compl_tag); + txq = cq->txqs[qid - cq->tx_start_qid]; + + switch (ctype) { + case IDPF_TXD_COMPLT_RE: + if (q_head == 0) + txq->last_desc_cleaned = txq->nb_tx_desc - 1; + else + txq->last_desc_cleaned = q_head - 1; + if (unlikely((txq->last_desc_cleaned % 32) == 0)) { + PMD_DRV_LOG(ERR, "unexpected desc (head = %u) completion.", + q_head); + return; + } + + break; + case IDPF_TXD_COMPLT_RS: + txq->nb_free++; + txq->nb_used--; + txe = &txq->sw_ring[q_head]; + if (txe->mbuf != NULL) { + rte_pktmbuf_free_seg(txe->mbuf); + txe->mbuf = NULL; + } + break; + default: + PMD_DRV_LOG(ERR, "unknown completion type."); + return; + } + + if (++next == cq->nb_tx_desc) { + next = 0; + cq->expected_gen_id ^= 1; + } + + cq->tx_tail = next; +} + +uint16_t +idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t nb_pkts) +{ + struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue; + volatile struct idpf_flex_tx_sched_desc *txr; + volatile struct idpf_flex_tx_sched_desc *txd; + struct idpf_tx_entry *sw_ring; + struct idpf_tx_entry *txe, *txn; + uint16_t nb_used, tx_id, sw_id; + struct rte_mbuf *tx_pkt; + uint16_t nb_to_clean; + uint16_t nb_tx = 0; + + if (unlikely(txq == NULL) || unlikely(!txq->q_started)) + return nb_tx; + + txr = txq->desc_ring; + sw_ring = txq->sw_ring; + tx_id = txq->tx_tail; + sw_id = txq->sw_tail; + txe = &sw_ring[sw_id]; + + for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) { + tx_pkt = tx_pkts[nb_tx]; + + if (txq->nb_free <= txq->free_thresh) { + /* TODO: Need to refine + * 1. free and clean: Better to decide a clean destination instead of + * loop times. And don't free mbuf when RS got immediately, free when + * transmit or according to the clean destination. + * Now, just ignore the RE write back, free mbuf when get RS + * 2. out-of-order rewrite back haven't be supported, SW head and HW head + * need to be separated. + **/ + nb_to_clean = 2 * txq->rs_thresh; + while (nb_to_clean--) + idpf_split_tx_free(txq->complq); + } + + if (txq->nb_free < tx_pkt->nb_segs) + break; + nb_used = tx_pkt->nb_segs; + + do { + txd = &txr[tx_id]; + txn = &sw_ring[txe->next_id]; + txe->mbuf = tx_pkt; + + /* Setup TX descriptor */ + txd->buf_addr = + rte_cpu_to_le_64(rte_mbuf_data_iova(tx_pkt)); + txd->qw1.cmd_dtype = + rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE); + txd->qw1.rxr_bufsize = tx_pkt->data_len; + txd->qw1.compl_tag = sw_id; + tx_id++; + if (tx_id == txq->nb_tx_desc) + tx_id = 0; + sw_id = txe->next_id; + txe = txn; + tx_pkt = tx_pkt->next; + } while (tx_pkt); + + /* fill the last descriptor with End of Packet (EOP) bit */ + txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_EOP; + + if (unlikely((tx_id % 32) == 0)) + txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_RE; + txq->nb_free = (uint16_t)(txq->nb_free - nb_used); + txq->nb_used = (uint16_t)(txq->nb_used + nb_used); + } + + /* update the tail pointer if any packets were processed */ + if (likely(nb_tx > 0)) { + IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id); + txq->tx_tail = tx_id; + txq->sw_tail = sw_id; + } + + return nb_tx; +} + static inline void idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold, uint16_t rx_id) @@ -1471,6 +1613,208 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, return nb_rx; } +static inline int +idpf_xmit_cleanup(struct idpf_tx_queue *txq) +{ + uint16_t last_desc_cleaned = txq->last_desc_cleaned; + struct idpf_tx_entry *sw_ring = txq->sw_ring; + uint16_t nb_tx_desc = txq->nb_tx_desc; + uint16_t desc_to_clean_to; + uint16_t nb_tx_to_clean; + uint16_t i; + + volatile struct idpf_flex_tx_desc *txd = txq->tx_ring; + + desc_to_clean_to = (uint16_t)(last_desc_cleaned + txq->rs_thresh); + if (desc_to_clean_to >= nb_tx_desc) + desc_to_clean_to = (uint16_t)(desc_to_clean_to - nb_tx_desc); + + desc_to_clean_to = sw_ring[desc_to_clean_to].last_id; + /* In the writeback Tx desccriptor, the only significant fields are the 4-bit DTYPE */ + if ((txd[desc_to_clean_to].qw1.cmd_dtype & + rte_cpu_to_le_16(IDPF_TXD_QW1_DTYPE_M)) != + rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE)) { + PMD_TX_LOG(DEBUG, "TX descriptor %4u is not done " + "(port=%d queue=%d)", desc_to_clean_to, + txq->port_id, txq->queue_id); + return -1; + } + + if (last_desc_cleaned > desc_to_clean_to) + nb_tx_to_clean = (uint16_t)((nb_tx_desc - last_desc_cleaned) + + desc_to_clean_to); + else + nb_tx_to_clean = (uint16_t)(desc_to_clean_to - + last_desc_cleaned); + + txd[desc_to_clean_to].qw1.cmd_dtype = 0; + txd[desc_to_clean_to].qw1.buf_size = 0; + for (i = 0; i < RTE_DIM(txd[desc_to_clean_to].qw1.flex.raw); i++) + txd[desc_to_clean_to].qw1.flex.raw[i] = 0; + + txq->last_desc_cleaned = desc_to_clean_to; + txq->nb_free = (uint16_t)(txq->nb_free + nb_tx_to_clean); + + return 0; +} + +/* TX function */ +uint16_t +idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t nb_pkts) +{ + volatile struct idpf_flex_tx_desc *txd; + volatile struct idpf_flex_tx_desc *txr; + struct idpf_tx_entry *txe, *txn; + struct idpf_tx_entry *sw_ring; + struct idpf_tx_queue *txq; + struct rte_mbuf *tx_pkt; + struct rte_mbuf *m_seg; + uint64_t buf_dma_addr; + uint16_t tx_last; + uint16_t nb_used; + uint16_t td_cmd; + uint16_t tx_id; + uint16_t nb_tx; + uint16_t slen; + + nb_tx = 0; + txq = tx_queue; + + if (unlikely(txq == NULL) || unlikely(!txq->q_started)) + return nb_tx; + + sw_ring = txq->sw_ring; + txr = txq->tx_ring; + tx_id = txq->tx_tail; + txe = &sw_ring[tx_id]; + + /* Check if the descriptor ring needs to be cleaned. */ + if (txq->nb_free < txq->free_thresh) + (void)idpf_xmit_cleanup(txq); + + for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) { + td_cmd = 0; + + tx_pkt = *tx_pkts++; + RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf); + + /* The number of descriptors that must be allocated for + * a packet equals to the number of the segments of that + * packet plus 1 context descriptor if needed. + */ + nb_used = (uint16_t)(tx_pkt->nb_segs); + tx_last = (uint16_t)(tx_id + nb_used - 1); + + /* Circular ring */ + if (tx_last >= txq->nb_tx_desc) + tx_last = (uint16_t)(tx_last - txq->nb_tx_desc); + + PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u" + " tx_first=%u tx_last=%u", + txq->port_id, txq->queue_id, tx_id, tx_last); + + if (nb_used > txq->nb_free) { + if (idpf_xmit_cleanup(txq) != 0) { + if (nb_tx == 0) + return 0; + goto end_of_tx; + } + if (unlikely(nb_used > txq->rs_thresh)) { + while (nb_used > txq->nb_free) { + if (idpf_xmit_cleanup(txq) != 0) { + if (nb_tx == 0) + return 0; + goto end_of_tx; + } + } + } + } + + m_seg = tx_pkt; + do { + txd = &txr[tx_id]; + txn = &sw_ring[txe->next_id]; + + if (txe->mbuf != NULL) + rte_pktmbuf_free_seg(txe->mbuf); + txe->mbuf = m_seg; + + /* Setup TX Descriptor */ + slen = m_seg->data_len; + buf_dma_addr = rte_mbuf_data_iova(m_seg); + txd->buf_addr = rte_cpu_to_le_64(buf_dma_addr); + txd->qw1.buf_size = slen; + txd->qw1.cmd_dtype = rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_DATA << + IDPF_FLEX_TXD_QW1_DTYPE_S); + + txe->last_id = tx_last; + tx_id = txe->next_id; + txe = txn; + m_seg = m_seg->next; + } while (m_seg); + + /* The last packet data descriptor needs End Of Packet (EOP) */ + td_cmd |= IDPF_TX_FLEX_DESC_CMD_EOP; + txq->nb_used = (uint16_t)(txq->nb_used + nb_used); + txq->nb_free = (uint16_t)(txq->nb_free - nb_used); + + if (txq->nb_used >= txq->rs_thresh) { + PMD_TX_LOG(DEBUG, "Setting RS bit on TXD id=" + "%4u (port=%d queue=%d)", + tx_last, txq->port_id, txq->queue_id); + + td_cmd |= IDPF_TX_FLEX_DESC_CMD_RS; + + /* Update txq RS bit counters */ + txq->nb_used = 0; + } + + txd->qw1.cmd_dtype |= rte_cpu_to_le_16(td_cmd << IDPF_FLEX_TXD_QW1_CMD_S); + } + +end_of_tx: + rte_wmb(); + + PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u tx_tail=%u nb_tx=%u", + txq->port_id, txq->queue_id, tx_id, nb_tx); + + IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id); + txq->tx_tail = tx_id; + + return nb_tx; +} + +/* TX prep functions */ +uint16_t +idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t nb_pkts) +{ + int i; + uint64_t ol_flags; + struct rte_mbuf *m; + + for (i = 0; i < nb_pkts; i++) { + m = tx_pkts[i]; + ol_flags = m->ol_flags; + + /* Check condition for nb_segs > IDPF_TX_MAX_MTU_SEG. */ + if ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) == 0) { + if (m->nb_segs > IDPF_TX_MAX_MTU_SEG) { + rte_errno = EINVAL; + return i; + } + } + + if (m->pkt_len < IDPF_MIN_FRAME_SIZE) { + rte_errno = EINVAL; + return i; + } + } + + return i; +} + void idpf_set_rx_function(struct rte_eth_dev *dev) { @@ -1481,3 +1825,16 @@ idpf_set_rx_function(struct rte_eth_dev *dev) else dev->rx_pkt_burst = idpf_singleq_recv_pkts; } + +void +idpf_set_tx_function(struct rte_eth_dev *dev) +{ + struct idpf_vport *vport = dev->data->dev_private; + if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) { + dev->tx_pkt_burst = idpf_splitq_xmit_pkts; + dev->tx_pkt_prepare = idpf_prep_pkts; + } else { + dev->tx_pkt_burst = idpf_singleq_xmit_pkts; + dev->tx_pkt_prepare = idpf_prep_pkts; + } +} diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h index 650c6c1c3a..30dc94b3dd 100644 --- a/drivers/net/idpf/idpf_rxtx.h +++ b/drivers/net/idpf/idpf_rxtx.h @@ -21,6 +21,8 @@ #define IDPF_DEFAULT_TX_RS_THRESH 32 #define IDPF_DEFAULT_TX_FREE_THRESH 32 +#define IDPF_TX_MAX_MTU_SEG 10 + struct idpf_rx_queue { struct idpf_adapter *adapter; /* the adapter this queue belongs to */ struct rte_mempool *mp; /* mbuf pool to populate Rx ring */ @@ -137,7 +139,15 @@ uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts); uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts); +uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t nb_pkts); +uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t nb_pkts); +uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t nb_pkts); + void idpf_stop_queues(struct rte_eth_dev *dev); void idpf_set_rx_function(struct rte_eth_dev *dev); +void idpf_set_tx_function(struct rte_eth_dev *dev); #endif /* _IDPF_RXTX_H_ */ From patchwork Mon Oct 31 08:33:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xing, Beilei" X-Patchwork-Id: 119352 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AC11CA00C2; Mon, 31 Oct 2022 10:06:02 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B41FC42B8F; Mon, 31 Oct 2022 10:04:53 +0100 (CET) Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id 7680740151 for ; Mon, 31 Oct 2022 10:04:21 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667207061; x=1698743061; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=T+GsSQEqqEMDaJEvuCfGSOUpK3LCmEilLFL8oaIvhAg=; b=TNw0BrVeXV254BcMWF2ROHK4H4+Bz11Sh8n+52qc3UbCIdcYOZRLrgdd XIq+LVHZvLHbo+qdlbKbg24VAAW9Y1Ql9P7USBwwt0I0jRf5HDRgOKR8z Syj2rnob969YdXM1Y0hqDo0y8pOUM7w7Vzs7tkiyKUvgXmdBsIRgjXWlJ 39qOWjJ1Wu4uxzsfzBydDqyqzjSsKKC9p7rPZRXHr3oPzGliPRSXjIDqe NfQ4Ps2KgvIFfhvvgjdBbVEsoKTGu0HS+CT+hnSbL6F0nchx23OELdeZ7 ixGSWzG7VSsnbjgSwRMEQDX796LOtoIfRKGqBbmefrKLD4lBE1QwtNaIn g==; X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="370926335" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="370926335" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Oct 2022 02:04:20 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="878664884" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="878664884" Received: from dpdk-beileix-3.sh.intel.com ([10.67.110.253]) by fmsmga006.fm.intel.com with ESMTP; 31 Oct 2022 02:04:19 -0700 From: beilei.xing@intel.com To: jingjing.wu@intel.com, beilei.xing@intel.com Cc: dev@dpdk.org, Junfeng Guo , Wenjun Wu Subject: [PATCH v18 12/18] net/idpf: support parsing packet type Date: Mon, 31 Oct 2022 08:33:40 +0000 Message-Id: <20221031083346.16558-13-beilei.xing@intel.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20221031083346.16558-1-beilei.xing@intel.com> References: <20221031051556.98549-1-beilei.xing@intel.com> <20221031083346.16558-1-beilei.xing@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Junfeng Guo Parse packet type during receiving packets. Signed-off-by: Wenjun Wu Signed-off-by: Junfeng Guo --- drivers/net/idpf/idpf_ethdev.c | 6 + drivers/net/idpf/idpf_ethdev.h | 6 + drivers/net/idpf/idpf_rxtx.c | 11 ++ drivers/net/idpf/idpf_rxtx.h | 5 + drivers/net/idpf/idpf_vchnl.c | 240 +++++++++++++++++++++++++++++++++ 5 files changed, 268 insertions(+) diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c index f9f6fe1162..d0821ec3f3 100644 --- a/drivers/net/idpf/idpf_ethdev.c +++ b/drivers/net/idpf/idpf_ethdev.c @@ -686,6 +686,12 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter) goto err_api; } + ret = idpf_get_pkt_type(adapter); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Failed to set ptype table"); + goto err_api; + } + adapter->caps = rte_zmalloc("idpf_caps", sizeof(struct virtchnl2_get_capabilities), 0); if (adapter->caps == NULL) { diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h index af0a8e2970..db9af58f72 100644 --- a/drivers/net/idpf/idpf_ethdev.h +++ b/drivers/net/idpf/idpf_ethdev.h @@ -39,6 +39,8 @@ #define IDPF_NUM_MACADDR_MAX 64 +#define IDPF_MAX_PKT_TYPE 1024 + #define IDPF_VLAN_TAG_SIZE 4 #define IDPF_ETH_OVERHEAD \ (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2) @@ -125,6 +127,8 @@ struct idpf_adapter { /* Max config queue number per VC message */ uint32_t max_rxq_per_msg; uint32_t max_txq_per_msg; + + uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned; }; TAILQ_HEAD(idpf_adapter_list, idpf_adapter); @@ -182,6 +186,7 @@ atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops) struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev); void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev); int idpf_vc_check_api_version(struct idpf_adapter *adapter); +int idpf_get_pkt_type(struct idpf_adapter *adapter); int idpf_vc_get_caps(struct idpf_adapter *adapter); int idpf_vc_create_vport(struct idpf_adapter *adapter); int idpf_vc_destroy_vport(struct idpf_vport *vport); @@ -193,6 +198,7 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid, bool rx, bool on); int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable); int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable); +int idpf_vc_query_ptype_info(struct idpf_adapter *adapter); int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops, uint16_t buf_len, uint8_t *buf); diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c index f55d2143b9..a980714060 100644 --- a/drivers/net/idpf/idpf_rxtx.c +++ b/drivers/net/idpf/idpf_rxtx.c @@ -1281,6 +1281,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc; uint16_t pktlen_gen_bufq_id; struct idpf_rx_queue *rxq; + const uint32_t *ptype_tbl; struct rte_mbuf *rxm; uint16_t rx_id_bufq1; uint16_t rx_id_bufq2; @@ -1300,6 +1301,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, rx_id_bufq1 = rxq->bufq1->rx_next_avail; rx_id_bufq2 = rxq->bufq2->rx_next_avail; rx_desc_ring = rxq->rx_ring; + ptype_tbl = rxq->adapter->ptype_tbl; while (nb_rx < nb_pkts) { rx_desc = &rx_desc_ring[rx_id]; @@ -1347,6 +1349,10 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, rxm->next = NULL; rxm->nb_segs = 1; rxm->port = rxq->port_id; + rxm->packet_type = + ptype_tbl[(rte_le_to_cpu_16(rx_desc->ptype_err_fflags0) & + VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >> + VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S]; rx_pkts[nb_rx++] = rxm; } @@ -1533,6 +1539,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, volatile union virtchnl2_rx_desc *rxdp; union virtchnl2_rx_desc rxd; struct idpf_rx_queue *rxq; + const uint32_t *ptype_tbl; uint16_t rx_id, nb_hold; struct rte_eth_dev *dev; uint16_t rx_packet_len; @@ -1551,6 +1558,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, rx_id = rxq->rx_tail; rx_ring = rxq->rx_ring; + ptype_tbl = rxq->adapter->ptype_tbl; while (nb_rx < nb_pkts) { rxdp = &rx_ring[rx_id]; @@ -1603,6 +1611,9 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, rxm->pkt_len = rx_packet_len; rxm->data_len = rx_packet_len; rxm->port = rxq->port_id; + rxm->packet_type = + ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) & + VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)]; rx_pkts[nb_rx++] = rxm; } diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h index 30dc94b3dd..3853ed55c9 100644 --- a/drivers/net/idpf/idpf_rxtx.h +++ b/drivers/net/idpf/idpf_rxtx.h @@ -23,6 +23,10 @@ #define IDPF_TX_MAX_MTU_SEG 10 +#define IDPF_GET_PTYPE_SIZE(p) \ + (sizeof(struct virtchnl2_ptype) + \ + (((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0]))) + struct idpf_rx_queue { struct idpf_adapter *adapter; /* the adapter this queue belongs to */ struct rte_mempool *mp; /* mbuf pool to populate Rx ring */ @@ -150,4 +154,5 @@ void idpf_stop_queues(struct rte_eth_dev *dev); void idpf_set_rx_function(struct rte_eth_dev *dev); void idpf_set_tx_function(struct rte_eth_dev *dev); + #endif /* _IDPF_RXTX_H_ */ diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c index f92141b54f..ced64dfaf6 100644 --- a/drivers/net/idpf/idpf_vchnl.c +++ b/drivers/net/idpf/idpf_vchnl.c @@ -234,6 +234,11 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args) err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer); clear_cmd(adapter); break; + case VIRTCHNL2_OP_GET_PTYPE_INFO: + /* for multuple response message, + * do not handle the response here. + */ + break; default: /* For other virtchnl ops in running time, * wait for the cmd done flag. @@ -298,6 +303,215 @@ idpf_vc_check_api_version(struct idpf_adapter *adapter) return 0; } +int __rte_cold +idpf_get_pkt_type(struct idpf_adapter *adapter) +{ + struct virtchnl2_get_ptype_info *ptype_info; + uint16_t ptype_recvd = 0, ptype_offset, i, j; + int ret; + + ret = idpf_vc_query_ptype_info(adapter); + if (ret != 0) { + PMD_DRV_LOG(ERR, "Fail to query packet type information"); + return ret; + } + + ptype_info = rte_zmalloc("ptype_info", IDPF_DFLT_MBX_BUF_SIZE, 0); + if (ptype_info == NULL) + return -ENOMEM; + + while (ptype_recvd < IDPF_MAX_PKT_TYPE) { + ret = idpf_read_one_msg(adapter, VIRTCHNL2_OP_GET_PTYPE_INFO, + IDPF_DFLT_MBX_BUF_SIZE, (u8 *)ptype_info); + if (ret != 0) { + PMD_DRV_LOG(ERR, "Fail to get packet type information"); + goto free_ptype_info; + } + + ptype_recvd += ptype_info->num_ptypes; + ptype_offset = sizeof(struct virtchnl2_get_ptype_info) - + sizeof(struct virtchnl2_ptype); + + for (i = 0; i < rte_cpu_to_le_16(ptype_info->num_ptypes); i++) { + bool is_inner = false, is_ip = false; + struct virtchnl2_ptype *ptype; + uint32_t proto_hdr = 0; + + ptype = (struct virtchnl2_ptype *) + ((u8 *)ptype_info + ptype_offset); + ptype_offset += IDPF_GET_PTYPE_SIZE(ptype); + if (ptype_offset > IDPF_DFLT_MBX_BUF_SIZE) { + ret = -EINVAL; + goto free_ptype_info; + } + + if (rte_cpu_to_le_16(ptype->ptype_id_10) == 0xFFFF) + goto free_ptype_info; + + for (j = 0; j < ptype->proto_id_count; j++) { + switch (rte_cpu_to_le_16(ptype->proto_id[j])) { + case VIRTCHNL2_PROTO_HDR_GRE: + case VIRTCHNL2_PROTO_HDR_VXLAN: + proto_hdr &= ~RTE_PTYPE_L4_MASK; + proto_hdr |= RTE_PTYPE_TUNNEL_GRENAT; + is_inner = true; + break; + case VIRTCHNL2_PROTO_HDR_MAC: + if (is_inner) { + proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK; + proto_hdr |= RTE_PTYPE_INNER_L2_ETHER; + } else { + proto_hdr &= ~RTE_PTYPE_L2_MASK; + proto_hdr |= RTE_PTYPE_L2_ETHER; + } + break; + case VIRTCHNL2_PROTO_HDR_VLAN: + if (is_inner) { + proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK; + proto_hdr |= RTE_PTYPE_INNER_L2_ETHER_VLAN; + } + break; + case VIRTCHNL2_PROTO_HDR_PTP: + proto_hdr &= ~RTE_PTYPE_L2_MASK; + proto_hdr |= RTE_PTYPE_L2_ETHER_TIMESYNC; + break; + case VIRTCHNL2_PROTO_HDR_LLDP: + proto_hdr &= ~RTE_PTYPE_L2_MASK; + proto_hdr |= RTE_PTYPE_L2_ETHER_LLDP; + break; + case VIRTCHNL2_PROTO_HDR_ARP: + proto_hdr &= ~RTE_PTYPE_L2_MASK; + proto_hdr |= RTE_PTYPE_L2_ETHER_ARP; + break; + case VIRTCHNL2_PROTO_HDR_PPPOE: + proto_hdr &= ~RTE_PTYPE_L2_MASK; + proto_hdr |= RTE_PTYPE_L2_ETHER_PPPOE; + break; + case VIRTCHNL2_PROTO_HDR_IPV4: + if (!is_ip) { + proto_hdr |= RTE_PTYPE_L3_IPV4_EXT_UNKNOWN; + is_ip = true; + } else { + proto_hdr |= RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN | + RTE_PTYPE_TUNNEL_IP; + is_inner = true; + } + break; + case VIRTCHNL2_PROTO_HDR_IPV6: + if (!is_ip) { + proto_hdr |= RTE_PTYPE_L3_IPV6_EXT_UNKNOWN; + is_ip = true; + } else { + proto_hdr |= RTE_PTYPE_INNER_L3_IPV6_EXT_UNKNOWN | + RTE_PTYPE_TUNNEL_IP; + is_inner = true; + } + break; + case VIRTCHNL2_PROTO_HDR_IPV4_FRAG: + case VIRTCHNL2_PROTO_HDR_IPV6_FRAG: + if (is_inner) + proto_hdr |= RTE_PTYPE_INNER_L4_FRAG; + else + proto_hdr |= RTE_PTYPE_L4_FRAG; + break; + case VIRTCHNL2_PROTO_HDR_UDP: + if (is_inner) + proto_hdr |= RTE_PTYPE_INNER_L4_UDP; + else + proto_hdr |= RTE_PTYPE_L4_UDP; + break; + case VIRTCHNL2_PROTO_HDR_TCP: + if (is_inner) + proto_hdr |= RTE_PTYPE_INNER_L4_TCP; + else + proto_hdr |= RTE_PTYPE_L4_TCP; + break; + case VIRTCHNL2_PROTO_HDR_SCTP: + if (is_inner) + proto_hdr |= RTE_PTYPE_INNER_L4_SCTP; + else + proto_hdr |= RTE_PTYPE_L4_SCTP; + break; + case VIRTCHNL2_PROTO_HDR_ICMP: + if (is_inner) + proto_hdr |= RTE_PTYPE_INNER_L4_ICMP; + else + proto_hdr |= RTE_PTYPE_L4_ICMP; + break; + case VIRTCHNL2_PROTO_HDR_ICMPV6: + if (is_inner) + proto_hdr |= RTE_PTYPE_INNER_L4_ICMP; + else + proto_hdr |= RTE_PTYPE_L4_ICMP; + break; + case VIRTCHNL2_PROTO_HDR_L2TPV2: + case VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL: + case VIRTCHNL2_PROTO_HDR_L2TPV3: + is_inner = true; + proto_hdr |= RTE_PTYPE_TUNNEL_L2TP; + break; + case VIRTCHNL2_PROTO_HDR_NVGRE: + is_inner = true; + proto_hdr |= RTE_PTYPE_TUNNEL_NVGRE; + break; + case VIRTCHNL2_PROTO_HDR_GTPC_TEID: + is_inner = true; + proto_hdr |= RTE_PTYPE_TUNNEL_GTPC; + break; + case VIRTCHNL2_PROTO_HDR_GTPU: + case VIRTCHNL2_PROTO_HDR_GTPU_UL: + case VIRTCHNL2_PROTO_HDR_GTPU_DL: + is_inner = true; + proto_hdr |= RTE_PTYPE_TUNNEL_GTPU; + break; + case VIRTCHNL2_PROTO_HDR_PAY: + case VIRTCHNL2_PROTO_HDR_IPV6_EH: + case VIRTCHNL2_PROTO_HDR_PRE_MAC: + case VIRTCHNL2_PROTO_HDR_POST_MAC: + case VIRTCHNL2_PROTO_HDR_ETHERTYPE: + case VIRTCHNL2_PROTO_HDR_SVLAN: + case VIRTCHNL2_PROTO_HDR_CVLAN: + case VIRTCHNL2_PROTO_HDR_MPLS: + case VIRTCHNL2_PROTO_HDR_MMPLS: + case VIRTCHNL2_PROTO_HDR_CTRL: + case VIRTCHNL2_PROTO_HDR_ECP: + case VIRTCHNL2_PROTO_HDR_EAPOL: + case VIRTCHNL2_PROTO_HDR_PPPOD: + case VIRTCHNL2_PROTO_HDR_IGMP: + case VIRTCHNL2_PROTO_HDR_AH: + case VIRTCHNL2_PROTO_HDR_ESP: + case VIRTCHNL2_PROTO_HDR_IKE: + case VIRTCHNL2_PROTO_HDR_NATT_KEEP: + case VIRTCHNL2_PROTO_HDR_GTP: + case VIRTCHNL2_PROTO_HDR_GTP_EH: + case VIRTCHNL2_PROTO_HDR_GTPCV2: + case VIRTCHNL2_PROTO_HDR_ECPRI: + case VIRTCHNL2_PROTO_HDR_VRRP: + case VIRTCHNL2_PROTO_HDR_OSPF: + case VIRTCHNL2_PROTO_HDR_TUN: + case VIRTCHNL2_PROTO_HDR_VXLAN_GPE: + case VIRTCHNL2_PROTO_HDR_GENEVE: + case VIRTCHNL2_PROTO_HDR_NSH: + case VIRTCHNL2_PROTO_HDR_QUIC: + case VIRTCHNL2_PROTO_HDR_PFCP: + case VIRTCHNL2_PROTO_HDR_PFCP_NODE: + case VIRTCHNL2_PROTO_HDR_PFCP_SESSION: + case VIRTCHNL2_PROTO_HDR_RTP: + case VIRTCHNL2_PROTO_HDR_NO_PROTO: + default: + continue; + } + adapter->ptype_tbl[ptype->ptype_id_10] = proto_hdr; + } + } + } + +free_ptype_info: + rte_free(ptype_info); + clear_cmd(adapter); + return ret; +} + int idpf_vc_get_caps(struct idpf_adapter *adapter) { @@ -930,3 +1144,29 @@ idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable) return err; } + +int +idpf_vc_query_ptype_info(struct idpf_adapter *adapter) +{ + struct virtchnl2_get_ptype_info *ptype_info; + struct idpf_cmd_info args; + int len, err; + + len = sizeof(struct virtchnl2_get_ptype_info); + ptype_info = rte_zmalloc("ptype_info", len, 0); + if (ptype_info == NULL) + return -ENOMEM; + + ptype_info->start_ptype_id = 0; + ptype_info->num_ptypes = IDPF_MAX_PKT_TYPE; + args.ops = VIRTCHNL2_OP_GET_PTYPE_INFO; + args.in_args = (u8 *)ptype_info; + args.in_args_size = len; + + err = idpf_execute_vc_cmd(adapter, &args); + if (err != 0) + PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_GET_PTYPE_INFO"); + + rte_free(ptype_info); + return err; +} From patchwork Mon Oct 31 08:33:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xing, Beilei" X-Patchwork-Id: 119353 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BCEBDA00C2; Mon, 31 Oct 2022 10:06:11 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7E07942B94; Mon, 31 Oct 2022 10:04:54 +0100 (CET) Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id 2031040A7F for ; Mon, 31 Oct 2022 10:04:22 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667207063; x=1698743063; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=GbRgf3KxVwqxNXMu5onCQGVUyygPfaKVJ68CuOXrr2g=; b=CwKXtLV091vJ3kL2aDYlh7D4+mv8iVuM9pdd6iAvFRPjnOhTTRKaX88e 09xVZYCiT2aCcAH6AcONH9IKI+Q1zv5tdy8+ipSZxZdzWInYjHe1IrqhT Znxkv8YTe1Bq+uA4hL8mPwmRy40D5igmVvs+0Hx2QSfeg6lfkMrV4kd20 ko0KXeK5BCyvTnZD6oSZJ0w2Y5WQXrpki4qyFohx8z0fdcvLJb7P1Bfda q7M7b9Rw+tDkBj8TooCiwFbC/dzg9LMyAgpj8lNBekosqQgLW0UenST8P smnuaya0CNO/Lp0I5j+ApFchcC2ruUtr4V+AHnYJIpnpxZYwBrnIzkSQ9 w==; X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="370926339" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="370926339" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Oct 2022 02:04:22 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="878664889" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="878664889" Received: from dpdk-beileix-3.sh.intel.com ([10.67.110.253]) by fmsmga006.fm.intel.com with ESMTP; 31 Oct 2022 02:04:20 -0700 From: beilei.xing@intel.com To: jingjing.wu@intel.com, beilei.xing@intel.com Cc: dev@dpdk.org, Junfeng Guo Subject: [PATCH v18 13/18] net/idpf: add support for write back based on ITR expire Date: Mon, 31 Oct 2022 08:33:41 +0000 Message-Id: <20221031083346.16558-14-beilei.xing@intel.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20221031083346.16558-1-beilei.xing@intel.com> References: <20221031051556.98549-1-beilei.xing@intel.com> <20221031083346.16558-1-beilei.xing@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Junfeng Guo Enable write back on ITR expire, then packets can be received one by one. Signed-off-by: Beilei Xing Signed-off-by: Junfeng Guo --- drivers/net/idpf/idpf_ethdev.c | 120 +++++++++++++++++++++++++++++++++ drivers/net/idpf/idpf_ethdev.h | 13 ++++ drivers/net/idpf/idpf_vchnl.c | 113 +++++++++++++++++++++++++++++++ 3 files changed, 246 insertions(+) diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c index d0821ec3f3..957cc10616 100644 --- a/drivers/net/idpf/idpf_ethdev.c +++ b/drivers/net/idpf/idpf_ethdev.c @@ -297,6 +297,90 @@ idpf_dev_configure(struct rte_eth_dev *dev) return 0; } +static int +idpf_config_rx_queues_irqs(struct rte_eth_dev *dev) +{ + struct idpf_vport *vport = dev->data->dev_private; + struct idpf_adapter *adapter = vport->adapter; + struct virtchnl2_queue_vector *qv_map; + struct idpf_hw *hw = &adapter->hw; + uint32_t dynctl_reg_start; + uint32_t itrn_reg_start; + uint32_t dynctl_val, itrn_val; + uint16_t i; + + qv_map = rte_zmalloc("qv_map", + dev->data->nb_rx_queues * + sizeof(struct virtchnl2_queue_vector), 0); + if (qv_map == NULL) { + PMD_DRV_LOG(ERR, "Failed to allocate %d queue-vector map", + dev->data->nb_rx_queues); + goto qv_map_alloc_err; + } + + /* Rx interrupt disabled, Map interrupt only for writeback */ + + /* The capability flags adapter->caps->other_caps should be + * compared with bit VIRTCHNL2_CAP_WB_ON_ITR here. The if + * condition should be updated when the FW can return the + * correct flag bits. + */ + dynctl_reg_start = + vport->recv_vectors->vchunks.vchunks->dynctl_reg_start; + itrn_reg_start = + vport->recv_vectors->vchunks.vchunks->itrn_reg_start; + dynctl_val = IDPF_READ_REG(hw, dynctl_reg_start); + PMD_DRV_LOG(DEBUG, "Value of dynctl_reg_start is 0x%x", + dynctl_val); + itrn_val = IDPF_READ_REG(hw, itrn_reg_start); + PMD_DRV_LOG(DEBUG, "Value of itrn_reg_start is 0x%x", itrn_val); + /* Force write-backs by setting WB_ON_ITR bit in DYN_CTL + * register. WB_ON_ITR and INTENA are mutually exclusive + * bits. Setting WB_ON_ITR bits means TX and RX Descs + * are written back based on ITR expiration irrespective + * of INTENA setting. + */ + /* TBD: need to tune INTERVAL value for better performance. */ + if (itrn_val != 0) + IDPF_WRITE_REG(hw, + dynctl_reg_start, + VIRTCHNL2_ITR_IDX_0 << + PF_GLINT_DYN_CTL_ITR_INDX_S | + PF_GLINT_DYN_CTL_WB_ON_ITR_M | + itrn_val << + PF_GLINT_DYN_CTL_INTERVAL_S); + else + IDPF_WRITE_REG(hw, + dynctl_reg_start, + VIRTCHNL2_ITR_IDX_0 << + PF_GLINT_DYN_CTL_ITR_INDX_S | + PF_GLINT_DYN_CTL_WB_ON_ITR_M | + IDPF_DFLT_INTERVAL << + PF_GLINT_DYN_CTL_INTERVAL_S); + + for (i = 0; i < dev->data->nb_rx_queues; i++) { + /* map all queues to the same vector */ + qv_map[i].queue_id = vport->chunks_info.rx_start_qid + i; + qv_map[i].vector_id = + vport->recv_vectors->vchunks.vchunks->start_vector_id; + } + vport->qv_map = qv_map; + + if (idpf_vc_config_irq_map_unmap(vport, true) != 0) { + PMD_DRV_LOG(ERR, "config interrupt mapping failed"); + goto config_irq_map_err; + } + + return 0; + +config_irq_map_err: + rte_free(vport->qv_map); + vport->qv_map = NULL; + +qv_map_alloc_err: + return -1; +} + static int idpf_start_queues(struct rte_eth_dev *dev) { @@ -334,6 +418,10 @@ static int idpf_dev_start(struct rte_eth_dev *dev) { struct idpf_vport *vport = dev->data->dev_private; + struct idpf_adapter *adapter = vport->adapter; + uint16_t num_allocated_vectors = + adapter->caps->num_allocated_vectors; + uint16_t req_vecs_num; int ret; if (dev->data->mtu > vport->max_mtu) { @@ -344,6 +432,27 @@ idpf_dev_start(struct rte_eth_dev *dev) vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD; + req_vecs_num = IDPF_DFLT_Q_VEC_NUM; + if (req_vecs_num + adapter->used_vecs_num > num_allocated_vectors) { + PMD_DRV_LOG(ERR, "The accumulated request vectors' number should be less than %d", + num_allocated_vectors); + ret = -EINVAL; + goto err_mtu; + } + + ret = idpf_vc_alloc_vectors(vport, req_vecs_num); + if (ret != 0) { + PMD_DRV_LOG(ERR, "Failed to allocate interrupt vectors"); + goto err_mtu; + } + adapter->used_vecs_num += req_vecs_num; + + ret = idpf_config_rx_queues_irqs(dev); + if (ret != 0) { + PMD_DRV_LOG(ERR, "Failed to configure irqs"); + goto err_mtu; + } + ret = idpf_start_queues(dev); if (ret != 0) { PMD_DRV_LOG(ERR, "Failed to start queues"); @@ -376,6 +485,10 @@ idpf_dev_stop(struct rte_eth_dev *dev) idpf_stop_queues(dev); + idpf_vc_config_irq_map_unmap(vport, false); + + idpf_vc_dealloc_vectors(vport); + return 0; } @@ -387,6 +500,11 @@ idpf_dev_close(struct rte_eth_dev *dev) idpf_vc_destroy_vport(vport); + rte_free(vport->recv_vectors); + vport->recv_vectors = NULL; + + rte_free(vport->qv_map); + vport->qv_map = NULL; adapter->cur_vports &= ~RTE_BIT32(vport->devarg_id); @@ -748,6 +866,8 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter) adapter->cur_vports = 0; adapter->cur_vport_nb = 0; + adapter->used_vecs_num = 0; + return ret; err_vports: diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h index db9af58f72..811240c386 100644 --- a/drivers/net/idpf/idpf_ethdev.h +++ b/drivers/net/idpf/idpf_ethdev.h @@ -33,6 +33,9 @@ #define IDPF_CTLQ_LEN 64 #define IDPF_DFLT_MBX_BUF_SIZE 4096 +#define IDPF_DFLT_Q_VEC_NUM 1 +#define IDPF_DFLT_INTERVAL 16 + #define IDPF_MIN_BUF_SIZE 1024 #define IDPF_MAX_FRAME_SIZE 9728 #define IDPF_MIN_FRAME_SIZE 14 @@ -92,6 +95,11 @@ struct idpf_vport { struct rte_eth_dev_data *dev_data; /* Pointer to the device data */ uint16_t max_pkt_len; /* Maximum packet length */ + /* MSIX info*/ + struct virtchnl2_queue_vector *qv_map; /* queue vector mapping */ + uint16_t max_vectors; + struct virtchnl2_alloc_vectors *recv_vectors; + /* Chunk info */ struct idpf_chunks_info chunks_info; @@ -124,6 +132,8 @@ struct idpf_adapter { uint16_t cur_vport_nb; uint16_t cur_vport_idx; + uint16_t used_vecs_num; + /* Max config queue number per VC message */ uint32_t max_rxq_per_msg; uint32_t max_txq_per_msg; @@ -198,6 +208,9 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid, bool rx, bool on); int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable); int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable); +int idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map); +int idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors); +int idpf_vc_dealloc_vectors(struct idpf_vport *vport); int idpf_vc_query_ptype_info(struct idpf_adapter *adapter); int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops, uint16_t buf_len, uint8_t *buf); diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c index ced64dfaf6..5088d50b3e 100644 --- a/drivers/net/idpf/idpf_vchnl.c +++ b/drivers/net/idpf/idpf_vchnl.c @@ -230,6 +230,10 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args) case VIRTCHNL2_OP_DISABLE_QUEUES: case VIRTCHNL2_OP_ENABLE_VPORT: case VIRTCHNL2_OP_DISABLE_VPORT: + case VIRTCHNL2_OP_MAP_QUEUE_VECTOR: + case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR: + case VIRTCHNL2_OP_ALLOC_VECTORS: + case VIRTCHNL2_OP_DEALLOC_VECTORS: /* for init virtchnl ops, need to poll the response */ err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer); clear_cmd(adapter); @@ -521,6 +525,8 @@ idpf_vc_get_caps(struct idpf_adapter *adapter) memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities)); + caps_msg.other_caps = VIRTCHNL2_CAP_WB_ON_ITR; + args.ops = VIRTCHNL2_OP_GET_CAPS; args.in_args = (uint8_t *)&caps_msg; args.in_args_size = sizeof(caps_msg); @@ -970,6 +976,113 @@ idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id) return err; } +int +idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map) +{ + struct idpf_adapter *adapter = vport->adapter; + struct virtchnl2_queue_vector_maps *map_info; + struct virtchnl2_queue_vector *vecmap; + uint16_t nb_rxq = vport->dev_data->nb_rx_queues; + struct idpf_cmd_info args; + int len, i, err = 0; + + len = sizeof(struct virtchnl2_queue_vector_maps) + + (nb_rxq - 1) * sizeof(struct virtchnl2_queue_vector); + + map_info = rte_zmalloc("map_info", len, 0); + if (map_info == NULL) + return -ENOMEM; + + map_info->vport_id = vport->vport_id; + map_info->num_qv_maps = nb_rxq; + for (i = 0; i < nb_rxq; i++) { + vecmap = &map_info->qv_maps[i]; + vecmap->queue_id = vport->qv_map[i].queue_id; + vecmap->vector_id = vport->qv_map[i].vector_id; + vecmap->itr_idx = VIRTCHNL2_ITR_IDX_0; + vecmap->queue_type = VIRTCHNL2_QUEUE_TYPE_RX; + } + + args.ops = map ? VIRTCHNL2_OP_MAP_QUEUE_VECTOR : + VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR; + args.in_args = (u8 *)map_info; + args.in_args_size = len; + args.out_buffer = adapter->mbx_resp; + args.out_size = IDPF_DFLT_MBX_BUF_SIZE; + err = idpf_execute_vc_cmd(adapter, &args); + if (err != 0) + PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUE_VECTOR", + map ? "MAP" : "UNMAP"); + + rte_free(map_info); + return err; +} + +int +idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors) +{ + struct idpf_adapter *adapter = vport->adapter; + struct virtchnl2_alloc_vectors *alloc_vec; + struct idpf_cmd_info args; + int err, len; + + len = sizeof(struct virtchnl2_alloc_vectors) + + (num_vectors - 1) * sizeof(struct virtchnl2_vector_chunk); + alloc_vec = rte_zmalloc("alloc_vec", len, 0); + if (alloc_vec == NULL) + return -ENOMEM; + + alloc_vec->num_vectors = num_vectors; + + args.ops = VIRTCHNL2_OP_ALLOC_VECTORS; + args.in_args = (u8 *)alloc_vec; + args.in_args_size = sizeof(struct virtchnl2_alloc_vectors); + args.out_buffer = adapter->mbx_resp; + args.out_size = IDPF_DFLT_MBX_BUF_SIZE; + err = idpf_execute_vc_cmd(adapter, &args); + if (err != 0) + PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_ALLOC_VECTORS"); + + if (vport->recv_vectors == NULL) { + vport->recv_vectors = rte_zmalloc("recv_vectors", len, 0); + if (vport->recv_vectors == NULL) { + rte_free(alloc_vec); + return -ENOMEM; + } + } + + rte_memcpy(vport->recv_vectors, args.out_buffer, len); + rte_free(alloc_vec); + return err; +} + +int +idpf_vc_dealloc_vectors(struct idpf_vport *vport) +{ + struct idpf_adapter *adapter = vport->adapter; + struct virtchnl2_alloc_vectors *alloc_vec; + struct virtchnl2_vector_chunks *vcs; + struct idpf_cmd_info args; + int err, len; + + alloc_vec = vport->recv_vectors; + vcs = &alloc_vec->vchunks; + + len = sizeof(struct virtchnl2_vector_chunks) + + (vcs->num_vchunks - 1) * sizeof(struct virtchnl2_vector_chunk); + + args.ops = VIRTCHNL2_OP_DEALLOC_VECTORS; + args.in_args = (u8 *)vcs; + args.in_args_size = len; + args.out_buffer = adapter->mbx_resp; + args.out_size = IDPF_DFLT_MBX_BUF_SIZE; + err = idpf_execute_vc_cmd(adapter, &args); + if (err != 0) + PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_DEALLOC_VECTORS"); + + return err; +} + static int idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid, uint32_t type, bool on) From patchwork Mon Oct 31 08:33:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xing, Beilei" X-Patchwork-Id: 119354 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7192CA00C2; Mon, 31 Oct 2022 10:06:19 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5557042B99; Mon, 31 Oct 2022 10:04:55 +0100 (CET) Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id 13FE540E28 for ; Mon, 31 Oct 2022 10:04:24 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667207065; x=1698743065; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=JOx5rbqurndqO4g8JwoR5QqfOWYOdpZot6zEv5sxRi8=; b=cVxCPSySJFeWwfn4O7GhL3+knXDhy0d4K5QulSliBbKbkEyXplfR+ahX iunKbHHrAKBgZc0ux4i9FuRtCV+Se0vUTyx5JCOxbjh0WAVsqEkL1lWWf Q4Xx5KltTkuwt+HQ6c++rfDppaHEP/XEmP/WRGDoYKdV/pQV4fJ2zMcKt HuTDPCECFvtQgD9R6m16zdxI2men/laRF78FrUokKzFaRsjYKcdfK59mG y6RhNaC6QUB3geDxVZKTJKGJFye/InyehZ6Gs5Czw7PW6DDe3TC5HiZ6m XDzMn+22CJs4asR/qyiKU7tHywikX/5A17f/jZcgkVDs6VmVeYg0eNa5K Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="370926342" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="370926342" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Oct 2022 02:04:24 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="878664907" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="878664907" Received: from dpdk-beileix-3.sh.intel.com ([10.67.110.253]) by fmsmga006.fm.intel.com with ESMTP; 31 Oct 2022 02:04:22 -0700 From: beilei.xing@intel.com To: jingjing.wu@intel.com, beilei.xing@intel.com Cc: dev@dpdk.org, Junfeng Guo , Xiaoyun Li Subject: [PATCH v18 14/18] net/idpf: add support for RSS Date: Mon, 31 Oct 2022 08:33:42 +0000 Message-Id: <20221031083346.16558-15-beilei.xing@intel.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20221031083346.16558-1-beilei.xing@intel.com> References: <20221031051556.98549-1-beilei.xing@intel.com> <20221031083346.16558-1-beilei.xing@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Junfeng Guo Add RSS support. Signed-off-by: Beilei Xing Signed-off-by: Xiaoyun Li Signed-off-by: Junfeng Guo --- drivers/net/idpf/idpf_ethdev.c | 120 ++++++++++++++++++++++++++++++++- drivers/net/idpf/idpf_ethdev.h | 26 +++++++ drivers/net/idpf/idpf_vchnl.c | 113 +++++++++++++++++++++++++++++++ 3 files changed, 258 insertions(+), 1 deletion(-) diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c index 957cc10616..58560ea404 100644 --- a/drivers/net/idpf/idpf_ethdev.c +++ b/drivers/net/idpf/idpf_ethdev.c @@ -59,6 +59,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD; dev_info->min_mtu = RTE_ETHER_MIN_MTU; + dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL; + dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS; dev_info->default_txconf = (struct rte_eth_txconf) { @@ -169,6 +171,8 @@ idpf_parse_devarg_id(char *name) return val; } +#define IDPF_RSS_KEY_LEN 52 + static int idpf_init_vport(struct rte_eth_dev *dev) { @@ -189,6 +193,10 @@ idpf_init_vport(struct rte_eth_dev *dev) vport->max_mtu = vport_info->max_mtu; rte_memcpy(vport->default_mac_addr, vport_info->default_mac_addr, ETH_ALEN); + vport->rss_algorithm = vport_info->rss_algorithm; + vport->rss_key_size = RTE_MIN(IDPF_RSS_KEY_LEN, + vport_info->rss_key_size); + vport->rss_lut_size = vport_info->rss_lut_size; vport->sw_idx = idx; for (i = 0; i < vport_info->chunks.num_chunks; i++) { @@ -246,17 +254,110 @@ idpf_init_vport(struct rte_eth_dev *dev) return 0; } +static int +idpf_config_rss(struct idpf_vport *vport) +{ + int ret; + + ret = idpf_vc_set_rss_key(vport); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Failed to configure RSS key"); + return ret; + } + + ret = idpf_vc_set_rss_lut(vport); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Failed to configure RSS lut"); + return ret; + } + + ret = idpf_vc_set_rss_hash(vport); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Failed to configure RSS hash"); + return ret; + } + + return ret; +} + +static int +idpf_init_rss(struct idpf_vport *vport) +{ + struct rte_eth_rss_conf *rss_conf; + uint16_t i, nb_q, lut_size; + int ret = 0; + + rss_conf = &vport->dev_data->dev_conf.rx_adv_conf.rss_conf; + nb_q = vport->dev_data->nb_rx_queues; + + vport->rss_key = rte_zmalloc("rss_key", + vport->rss_key_size, 0); + if (vport->rss_key == NULL) { + PMD_INIT_LOG(ERR, "Failed to allocate RSS key"); + ret = -ENOMEM; + goto err_alloc_key; + } + + lut_size = vport->rss_lut_size; + vport->rss_lut = rte_zmalloc("rss_lut", + sizeof(uint32_t) * lut_size, 0); + if (vport->rss_lut == NULL) { + PMD_INIT_LOG(ERR, "Failed to allocate RSS lut"); + ret = -ENOMEM; + goto err_alloc_lut; + } + + if (rss_conf->rss_key == NULL) { + for (i = 0; i < vport->rss_key_size; i++) + vport->rss_key[i] = (uint8_t)rte_rand(); + } else if (rss_conf->rss_key_len != vport->rss_key_size) { + PMD_INIT_LOG(ERR, "Invalid RSS key length in RSS configuration, should be %d", + vport->rss_key_size); + ret = -EINVAL; + goto err_cfg_key; + } else { + rte_memcpy(vport->rss_key, rss_conf->rss_key, + vport->rss_key_size); + } + + for (i = 0; i < lut_size; i++) + vport->rss_lut[i] = i % nb_q; + + vport->rss_hf = IDPF_DEFAULT_RSS_HASH_EXPANDED; + + ret = idpf_config_rss(vport); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Failed to configure RSS"); + goto err_cfg_key; + } + + return ret; + +err_cfg_key: + rte_free(vport->rss_lut); + vport->rss_lut = NULL; +err_alloc_lut: + rte_free(vport->rss_key); + vport->rss_key = NULL; +err_alloc_key: + return ret; +} + static int idpf_dev_configure(struct rte_eth_dev *dev) { + struct idpf_vport *vport = dev->data->dev_private; struct rte_eth_conf *conf = &dev->data->dev_conf; + struct idpf_adapter *adapter = vport->adapter; + int ret; if (conf->link_speeds & RTE_ETH_LINK_SPEED_FIXED) { PMD_INIT_LOG(ERR, "Setting link speed is not supported"); return -ENOTSUP; } - if (dev->data->nb_rx_queues == 1 && conf->rxmode.mq_mode != RTE_ETH_MQ_RX_NONE) { + if ((dev->data->nb_rx_queues == 1 && conf->rxmode.mq_mode != RTE_ETH_MQ_RX_NONE) || + (dev->data->nb_rx_queues > 1 && conf->rxmode.mq_mode != RTE_ETH_MQ_RX_RSS)) { PMD_INIT_LOG(ERR, "Multi-queue packet distribution mode %d is not supported", conf->rxmode.mq_mode); return -ENOTSUP; @@ -294,6 +395,17 @@ idpf_dev_configure(struct rte_eth_dev *dev) return -ENOTSUP; } + if (adapter->caps->rss_caps != 0 && dev->data->nb_rx_queues != 0) { + ret = idpf_init_rss(vport); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Failed to init rss"); + return ret; + } + } else { + PMD_INIT_LOG(ERR, "RSS is not supported."); + return -1; + } + return 0; } @@ -500,6 +612,12 @@ idpf_dev_close(struct rte_eth_dev *dev) idpf_vc_destroy_vport(vport); + rte_free(vport->rss_lut); + vport->rss_lut = NULL; + + rte_free(vport->rss_key); + vport->rss_key = NULL; + rte_free(vport->recv_vectors); vport->recv_vectors = NULL; diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h index 811240c386..8d0804f603 100644 --- a/drivers/net/idpf/idpf_ethdev.h +++ b/drivers/net/idpf/idpf_ethdev.h @@ -48,6 +48,20 @@ #define IDPF_ETH_OVERHEAD \ (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2) +#define IDPF_RSS_OFFLOAD_ALL ( \ + RTE_ETH_RSS_IPV4 | \ + RTE_ETH_RSS_FRAG_IPV4 | \ + RTE_ETH_RSS_NONFRAG_IPV4_TCP | \ + RTE_ETH_RSS_NONFRAG_IPV4_UDP | \ + RTE_ETH_RSS_NONFRAG_IPV4_SCTP | \ + RTE_ETH_RSS_NONFRAG_IPV4_OTHER | \ + RTE_ETH_RSS_IPV6 | \ + RTE_ETH_RSS_FRAG_IPV6 | \ + RTE_ETH_RSS_NONFRAG_IPV6_TCP | \ + RTE_ETH_RSS_NONFRAG_IPV6_UDP | \ + RTE_ETH_RSS_NONFRAG_IPV6_SCTP | \ + RTE_ETH_RSS_NONFRAG_IPV6_OTHER) + #define IDPF_ADAPTER_NAME_LEN (PCI_PRI_STR_SIZE + 1) /* Message type read in virtual channel from PF */ @@ -90,11 +104,20 @@ struct idpf_vport { uint16_t max_mtu; uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS]; + enum virtchnl_rss_algorithm rss_algorithm; + uint16_t rss_key_size; + uint16_t rss_lut_size; + uint16_t sw_idx; /* SW idx */ struct rte_eth_dev_data *dev_data; /* Pointer to the device data */ uint16_t max_pkt_len; /* Maximum packet length */ + /* RSS info */ + uint32_t *rss_lut; + uint8_t *rss_key; + uint64_t rss_hf; + /* MSIX info*/ struct virtchnl2_queue_vector *qv_map; /* queue vector mapping */ uint16_t max_vectors; @@ -200,6 +223,9 @@ int idpf_get_pkt_type(struct idpf_adapter *adapter); int idpf_vc_get_caps(struct idpf_adapter *adapter); int idpf_vc_create_vport(struct idpf_adapter *adapter); int idpf_vc_destroy_vport(struct idpf_vport *vport); +int idpf_vc_set_rss_key(struct idpf_vport *vport); +int idpf_vc_set_rss_lut(struct idpf_vport *vport); +int idpf_vc_set_rss_hash(struct idpf_vport *vport); int idpf_vc_config_rxqs(struct idpf_vport *vport); int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id); int idpf_vc_config_txqs(struct idpf_vport *vport); diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c index 5088d50b3e..4ef354df89 100644 --- a/drivers/net/idpf/idpf_vchnl.c +++ b/drivers/net/idpf/idpf_vchnl.c @@ -224,6 +224,9 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args) case VIRTCHNL2_OP_GET_CAPS: case VIRTCHNL2_OP_CREATE_VPORT: case VIRTCHNL2_OP_DESTROY_VPORT: + case VIRTCHNL2_OP_SET_RSS_KEY: + case VIRTCHNL2_OP_SET_RSS_LUT: + case VIRTCHNL2_OP_SET_RSS_HASH: case VIRTCHNL2_OP_CONFIG_RX_QUEUES: case VIRTCHNL2_OP_CONFIG_TX_QUEUES: case VIRTCHNL2_OP_ENABLE_QUEUES: @@ -525,6 +528,22 @@ idpf_vc_get_caps(struct idpf_adapter *adapter) memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities)); + caps_msg.rss_caps = + VIRTCHNL2_CAP_RSS_IPV4_TCP | + VIRTCHNL2_CAP_RSS_IPV4_UDP | + VIRTCHNL2_CAP_RSS_IPV4_SCTP | + VIRTCHNL2_CAP_RSS_IPV4_OTHER | + VIRTCHNL2_CAP_RSS_IPV6_TCP | + VIRTCHNL2_CAP_RSS_IPV6_UDP | + VIRTCHNL2_CAP_RSS_IPV6_SCTP | + VIRTCHNL2_CAP_RSS_IPV6_OTHER | + VIRTCHNL2_CAP_RSS_IPV4_AH | + VIRTCHNL2_CAP_RSS_IPV4_ESP | + VIRTCHNL2_CAP_RSS_IPV4_AH_ESP | + VIRTCHNL2_CAP_RSS_IPV6_AH | + VIRTCHNL2_CAP_RSS_IPV6_ESP | + VIRTCHNL2_CAP_RSS_IPV6_AH_ESP; + caps_msg.other_caps = VIRTCHNL2_CAP_WB_ON_ITR; args.ops = VIRTCHNL2_OP_GET_CAPS; @@ -615,6 +634,100 @@ idpf_vc_destroy_vport(struct idpf_vport *vport) return err; } +int +idpf_vc_set_rss_key(struct idpf_vport *vport) +{ + struct idpf_adapter *adapter = vport->adapter; + struct virtchnl2_rss_key *rss_key; + struct idpf_cmd_info args; + int len, err; + + len = sizeof(*rss_key) + sizeof(rss_key->key[0]) * + (vport->rss_key_size - 1); + rss_key = rte_zmalloc("rss_key", len, 0); + if (rss_key == NULL) + return -ENOMEM; + + rss_key->vport_id = vport->vport_id; + rss_key->key_len = vport->rss_key_size; + rte_memcpy(rss_key->key, vport->rss_key, + sizeof(rss_key->key[0]) * vport->rss_key_size); + + memset(&args, 0, sizeof(args)); + args.ops = VIRTCHNL2_OP_SET_RSS_KEY; + args.in_args = (uint8_t *)rss_key; + args.in_args_size = len; + args.out_buffer = adapter->mbx_resp; + args.out_size = IDPF_DFLT_MBX_BUF_SIZE; + + err = idpf_execute_vc_cmd(adapter, &args); + if (err != 0) + PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_KEY"); + + rte_free(rss_key); + return err; +} + +int +idpf_vc_set_rss_lut(struct idpf_vport *vport) +{ + struct idpf_adapter *adapter = vport->adapter; + struct virtchnl2_rss_lut *rss_lut; + struct idpf_cmd_info args; + int len, err; + + len = sizeof(*rss_lut) + sizeof(rss_lut->lut[0]) * + (vport->rss_lut_size - 1); + rss_lut = rte_zmalloc("rss_lut", len, 0); + if (rss_lut == NULL) + return -ENOMEM; + + rss_lut->vport_id = vport->vport_id; + rss_lut->lut_entries = vport->rss_lut_size; + rte_memcpy(rss_lut->lut, vport->rss_lut, + sizeof(rss_lut->lut[0]) * vport->rss_lut_size); + + memset(&args, 0, sizeof(args)); + args.ops = VIRTCHNL2_OP_SET_RSS_LUT; + args.in_args = (uint8_t *)rss_lut; + args.in_args_size = len; + args.out_buffer = adapter->mbx_resp; + args.out_size = IDPF_DFLT_MBX_BUF_SIZE; + + err = idpf_execute_vc_cmd(adapter, &args); + if (err != 0) + PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_LUT"); + + rte_free(rss_lut); + return err; +} + +int +idpf_vc_set_rss_hash(struct idpf_vport *vport) +{ + struct idpf_adapter *adapter = vport->adapter; + struct virtchnl2_rss_hash rss_hash; + struct idpf_cmd_info args; + int err; + + memset(&rss_hash, 0, sizeof(rss_hash)); + rss_hash.ptype_groups = vport->rss_hf; + rss_hash.vport_id = vport->vport_id; + + memset(&args, 0, sizeof(args)); + args.ops = VIRTCHNL2_OP_SET_RSS_HASH; + args.in_args = (uint8_t *)&rss_hash; + args.in_args_size = sizeof(rss_hash); + args.out_buffer = adapter->mbx_resp; + args.out_size = IDPF_DFLT_MBX_BUF_SIZE; + + err = idpf_execute_vc_cmd(adapter, &args); + if (err != 0) + PMD_DRV_LOG(ERR, "Failed to execute command of OP_SET_RSS_HASH"); + + return err; +} + #define IDPF_RX_BUF_STRIDE 64 int idpf_vc_config_rxqs(struct idpf_vport *vport) From patchwork Mon Oct 31 08:33:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xing, Beilei" X-Patchwork-Id: 119355 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7CEB8A00C2; Mon, 31 Oct 2022 10:06:26 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3C33842BA6; Mon, 31 Oct 2022 10:04:56 +0100 (CET) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id 998024282E for ; Mon, 31 Oct 2022 10:04:31 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667207071; x=1698743071; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=5rMWBxa9m5j0aPYu9kicTt9PwRayfVE/iv6Pv7zetVU=; b=ihzScHMId/119tl4o/9VUMdi+f+y1Z70coFzPNEG+vjcIfYRCtj4FXg+ yhy7zeVyc7836dKf4pJg1TKdjlKHR3H/DY2mFk9UwQ/wgIc59CAu/6PB+ VNY2jhYEc+5ktwkUUjO0YBTkDqjyz/rAbgYLlbBdPchgV1zc/GPwkWJsE UsAeVuzoSD1YDZs2lBUXQeGVnUI0BVnmzJK3vMkWHLI4FgLZcFHew7Mc+ SwLK8OrNBbSb3o93Ctst7kOvFSnFZmIwge2HoBtMYvusEa4w8NhgjkBMf eLLMcQ7g3le/rGZCwur5nbAfGvAjKn5e9jaqC2enjrkINorKZ3RdD3FF6 A==; X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="335507848" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="335507848" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Oct 2022 02:04:30 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="878664918" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="878664918" Received: from dpdk-beileix-3.sh.intel.com ([10.67.110.253]) by fmsmga006.fm.intel.com with ESMTP; 31 Oct 2022 02:04:24 -0700 From: beilei.xing@intel.com To: jingjing.wu@intel.com, beilei.xing@intel.com Cc: dev@dpdk.org, Junfeng Guo , Xiaoyun Li Subject: [PATCH v18 15/18] net/idpf: add support for Rx offloading Date: Mon, 31 Oct 2022 08:33:43 +0000 Message-Id: <20221031083346.16558-16-beilei.xing@intel.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20221031083346.16558-1-beilei.xing@intel.com> References: <20221031051556.98549-1-beilei.xing@intel.com> <20221031083346.16558-1-beilei.xing@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Junfeng Guo Add Rx offloading support: - support CHKSUM and RSS offload for split queue model - support CHKSUM offload for single queue model Signed-off-by: Beilei Xing Signed-off-by: Xiaoyun Li Signed-off-by: Junfeng Guo --- doc/guides/nics/features/idpf.ini | 5 ++ drivers/net/idpf/idpf_ethdev.c | 6 ++ drivers/net/idpf/idpf_rxtx.c | 123 ++++++++++++++++++++++++++++++ drivers/net/idpf/idpf_vchnl.c | 18 +++++ 4 files changed, 152 insertions(+) diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini index d722c49fde..868571654f 100644 --- a/doc/guides/nics/features/idpf.ini +++ b/doc/guides/nics/features/idpf.ini @@ -3,8 +3,13 @@ ; ; Refer to default.ini for the full list of available PMD features. ; +; A feature with "P" indicates only be supported when non-vector path +; is selected. +; [Features] MTU update = Y +L3 checksum offload = P +L4 checksum offload = P Linux = Y x86-32 = Y x86-64 = Y diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c index 58560ea404..a09f104425 100644 --- a/drivers/net/idpf/idpf_ethdev.c +++ b/drivers/net/idpf/idpf_ethdev.c @@ -61,6 +61,12 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL; + dev_info->rx_offload_capa = + RTE_ETH_RX_OFFLOAD_IPV4_CKSUM | + RTE_ETH_RX_OFFLOAD_UDP_CKSUM | + RTE_ETH_RX_OFFLOAD_TCP_CKSUM | + RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM; + dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS; dev_info->default_txconf = (struct rte_eth_txconf) { diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c index a980714060..f15e61a785 100644 --- a/drivers/net/idpf/idpf_rxtx.c +++ b/drivers/net/idpf/idpf_rxtx.c @@ -1209,6 +1209,73 @@ idpf_stop_queues(struct rte_eth_dev *dev) } } +#define IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S \ + (RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S) | \ + RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S) | \ + RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S) | \ + RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S)) + +static inline uint64_t +idpf_splitq_rx_csum_offload(uint8_t err) +{ + uint64_t flags = 0; + + if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S)) == 0)) + return flags; + + if (likely((err & IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S) == 0)) { + flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD | + RTE_MBUF_F_RX_L4_CKSUM_GOOD); + return flags; + } + + if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S)) != 0)) + flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD; + else + flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD; + + if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S)) != 0)) + flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD; + else + flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD; + + if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S)) != 0)) + flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD; + + if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S)) != 0)) + flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD; + else + flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD; + + return flags; +} + +#define IDPF_RX_FLEX_DESC_ADV_HASH1_S 0 +#define IDPF_RX_FLEX_DESC_ADV_HASH2_S 16 +#define IDPF_RX_FLEX_DESC_ADV_HASH3_S 24 + +static inline uint64_t +idpf_splitq_rx_rss_offload(struct rte_mbuf *mb, + volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc) +{ + uint8_t status_err0_qw0; + uint64_t flags = 0; + + status_err0_qw0 = rx_desc->status_err0_qw0; + + if ((status_err0_qw0 & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S)) != 0) { + flags |= RTE_MBUF_F_RX_RSS_HASH; + mb->hash.rss = (rte_le_to_cpu_16(rx_desc->hash1) << + IDPF_RX_FLEX_DESC_ADV_HASH1_S) | + ((uint32_t)(rx_desc->ff2_mirrid_hash2.hash2) << + IDPF_RX_FLEX_DESC_ADV_HASH2_S) | + ((uint32_t)(rx_desc->hash3) << + IDPF_RX_FLEX_DESC_ADV_HASH3_S); + } + + return flags; +} + static void idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq) { @@ -1282,9 +1349,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pktlen_gen_bufq_id; struct idpf_rx_queue *rxq; const uint32_t *ptype_tbl; + uint8_t status_err0_qw1; struct rte_mbuf *rxm; uint16_t rx_id_bufq1; uint16_t rx_id_bufq2; + uint64_t pkt_flags; uint16_t pkt_len; uint16_t bufq_id; uint16_t gen_id; @@ -1349,11 +1418,18 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, rxm->next = NULL; rxm->nb_segs = 1; rxm->port = rxq->port_id; + rxm->ol_flags = 0; rxm->packet_type = ptype_tbl[(rte_le_to_cpu_16(rx_desc->ptype_err_fflags0) & VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >> VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S]; + status_err0_qw1 = rx_desc->status_err0_qw1; + pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1); + pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc); + + rxm->ol_flags |= pkt_flags; + rx_pkts[nb_rx++] = rxm; } @@ -1513,6 +1589,48 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, return nb_tx; } +#define IDPF_RX_FLEX_DESC_STATUS0_XSUM_S \ + (RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S) | \ + RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S) | \ + RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S) | \ + RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S)) + +/* Translate the rx descriptor status and error fields to pkt flags */ +static inline uint64_t +idpf_rxd_to_pkt_flags(uint16_t status_error) +{ + uint64_t flags = 0; + + if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S)) == 0)) + return flags; + + if (likely((status_error & IDPF_RX_FLEX_DESC_STATUS0_XSUM_S) == 0)) { + flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD | + RTE_MBUF_F_RX_L4_CKSUM_GOOD); + return flags; + } + + if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S)) != 0)) + flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD; + else + flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD; + + if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S)) != 0)) + flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD; + else + flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD; + + if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S)) != 0)) + flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD; + + if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S)) != 0)) + flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD; + else + flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD; + + return flags; +} + static inline void idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold, uint16_t rx_id) @@ -1546,6 +1664,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, struct rte_mbuf *rxm; struct rte_mbuf *nmb; uint16_t rx_status0; + uint64_t pkt_flags; uint64_t dma_addr; uint16_t nb_rx; @@ -1611,10 +1730,14 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, rxm->pkt_len = rx_packet_len; rxm->data_len = rx_packet_len; rxm->port = rxq->port_id; + rxm->ol_flags = 0; + pkt_flags = idpf_rxd_to_pkt_flags(rx_status0); rxm->packet_type = ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) & VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)]; + rxm->ol_flags |= pkt_flags; + rx_pkts[nb_rx++] = rxm; } rxq->rx_tail = rx_id; diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c index 4ef354df89..00ac5b2a6b 100644 --- a/drivers/net/idpf/idpf_vchnl.c +++ b/drivers/net/idpf/idpf_vchnl.c @@ -528,6 +528,24 @@ idpf_vc_get_caps(struct idpf_adapter *adapter) memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities)); + caps_msg.csum_caps = + VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 | + VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP | + VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP | + VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP | + VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP | + VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP | + VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP | + VIRTCHNL2_CAP_TX_CSUM_GENERIC | + VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 | + VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP | + VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP | + VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP | + VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP | + VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP | + VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP | + VIRTCHNL2_CAP_RX_CSUM_GENERIC; + caps_msg.rss_caps = VIRTCHNL2_CAP_RSS_IPV4_TCP | VIRTCHNL2_CAP_RSS_IPV4_UDP | From patchwork Mon Oct 31 08:33:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xing, Beilei" X-Patchwork-Id: 119356 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0521CA00C2; Mon, 31 Oct 2022 10:06:34 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0A6CF42B6D; Mon, 31 Oct 2022 10:04:57 +0100 (CET) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id 6191B40DF6 for ; Mon, 31 Oct 2022 10:04:32 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667207072; x=1698743072; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ZHtbm7ocM+oQCXmuwEBmWznzSXP2g6MadfJjp8DCK7Y=; b=BuFAAawWR1bqk7kCU9BXI7OvJNh7Aw9E8VcH+r39tNyumvnnhFGDXfRS 2yGQjx+BcrA0W8sa2gtLiH4AEjqK9ptvRL46agSgIQ1IXjij3SH9dkVXp r6sjtv0xqc6Xs+bJbx2YFVVaoC5sjLu8n9l8RimktrrRxoq5RmoYAQxWQ qkIfB25wLJwkinRK6JyXKI1hgDJC6OL+cXWlQGjnW0QN+W3ztKUuvWNyt LZDLazIgWM3W3xXp+Ftmf83LZL/kNHBCcyp8DY+cAyf5DGzfTJaLi6keC eFtOvN53hbjNmyd1xnLMmrgOGmOy/je6MUCaXQNgi9OVNs4+CM+IazybM g==; X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="335507850" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="335507850" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Oct 2022 02:04:31 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="878664944" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="878664944" Received: from dpdk-beileix-3.sh.intel.com ([10.67.110.253]) by fmsmga006.fm.intel.com with ESMTP; 31 Oct 2022 02:04:28 -0700 From: beilei.xing@intel.com To: jingjing.wu@intel.com, beilei.xing@intel.com Cc: dev@dpdk.org, Junfeng Guo , Xiaoyun Li Subject: [PATCH v18 16/18] net/idpf: add support for Tx offloading Date: Mon, 31 Oct 2022 08:33:44 +0000 Message-Id: <20221031083346.16558-17-beilei.xing@intel.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20221031083346.16558-1-beilei.xing@intel.com> References: <20221031051556.98549-1-beilei.xing@intel.com> <20221031083346.16558-1-beilei.xing@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Junfeng Guo Add Tx offloading support: - support TSO for single queue model and split queue model. Signed-off-by: Beilei Xing Signed-off-by: Xiaoyun Li Signed-off-by: Junfeng Guo --- doc/guides/nics/features/idpf.ini | 1 + drivers/net/idpf/idpf_ethdev.c | 4 +- drivers/net/idpf/idpf_rxtx.c | 128 +++++++++++++++++++++++++++++- drivers/net/idpf/idpf_rxtx.h | 22 +++++ 4 files changed, 152 insertions(+), 3 deletions(-) diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini index 868571654f..d82b4aa0ff 100644 --- a/doc/guides/nics/features/idpf.ini +++ b/doc/guides/nics/features/idpf.ini @@ -8,6 +8,7 @@ ; [Features] MTU update = Y +TSO = P L3 checksum offload = P L4 checksum offload = P Linux = Y diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c index a09f104425..084426260c 100644 --- a/drivers/net/idpf/idpf_ethdev.c +++ b/drivers/net/idpf/idpf_ethdev.c @@ -67,7 +67,9 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) RTE_ETH_RX_OFFLOAD_TCP_CKSUM | RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM; - dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS; + dev_info->tx_offload_capa = + RTE_ETH_TX_OFFLOAD_TCP_TSO | + RTE_ETH_TX_OFFLOAD_MULTI_SEGS; dev_info->default_txconf = (struct rte_eth_txconf) { .tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH, diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c index f15e61a785..cc296d7ab1 100644 --- a/drivers/net/idpf/idpf_rxtx.c +++ b/drivers/net/idpf/idpf_rxtx.c @@ -1506,6 +1506,49 @@ idpf_split_tx_free(struct idpf_tx_queue *cq) cq->tx_tail = next; } +/* Check if the context descriptor is needed for TX offloading */ +static inline uint16_t +idpf_calc_context_desc(uint64_t flags) +{ + if ((flags & RTE_MBUF_F_TX_TCP_SEG) != 0) + return 1; + + return 0; +} + +/* set TSO context descriptor + */ +static inline void +idpf_set_splitq_tso_ctx(struct rte_mbuf *mbuf, + union idpf_tx_offload tx_offload, + volatile union idpf_flex_tx_ctx_desc *ctx_desc) +{ + uint16_t cmd_dtype; + uint32_t tso_len; + uint8_t hdr_len; + + if (tx_offload.l4_len == 0) { + PMD_TX_LOG(DEBUG, "L4 length set to 0"); + return; + } + + hdr_len = tx_offload.l2_len + + tx_offload.l3_len + + tx_offload.l4_len; + cmd_dtype = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX | + IDPF_TX_FLEX_CTX_DESC_CMD_TSO; + tso_len = mbuf->pkt_len - hdr_len; + + ctx_desc->tso.qw1.cmd_dtype = rte_cpu_to_le_16(cmd_dtype); + ctx_desc->tso.qw0.hdr_len = hdr_len; + ctx_desc->tso.qw0.mss_rt = + rte_cpu_to_le_16((uint16_t)mbuf->tso_segsz & + IDPF_TXD_FLEX_CTX_MSS_RT_M); + ctx_desc->tso.qw0.flex_tlen = + rte_cpu_to_le_32(tso_len & + IDPF_TXD_FLEX_CTX_MSS_RT_M); +} + uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) @@ -1514,11 +1557,14 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, volatile struct idpf_flex_tx_sched_desc *txr; volatile struct idpf_flex_tx_sched_desc *txd; struct idpf_tx_entry *sw_ring; + union idpf_tx_offload tx_offload = {0}; struct idpf_tx_entry *txe, *txn; uint16_t nb_used, tx_id, sw_id; struct rte_mbuf *tx_pkt; uint16_t nb_to_clean; uint16_t nb_tx = 0; + uint64_t ol_flags; + uint16_t nb_ctx; if (unlikely(txq == NULL) || unlikely(!txq->q_started)) return nb_tx; @@ -1548,7 +1594,29 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, if (txq->nb_free < tx_pkt->nb_segs) break; - nb_used = tx_pkt->nb_segs; + + ol_flags = tx_pkt->ol_flags; + tx_offload.l2_len = tx_pkt->l2_len; + tx_offload.l3_len = tx_pkt->l3_len; + tx_offload.l4_len = tx_pkt->l4_len; + tx_offload.tso_segsz = tx_pkt->tso_segsz; + /* Calculate the number of context descriptors needed. */ + nb_ctx = idpf_calc_context_desc(ol_flags); + nb_used = tx_pkt->nb_segs + nb_ctx; + + /* context descriptor */ + if (nb_ctx != 0) { + volatile union idpf_flex_tx_ctx_desc *ctx_desc = + (volatile union idpf_flex_tx_ctx_desc *)&txr[tx_id]; + + if ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) != 0) + idpf_set_splitq_tso_ctx(tx_pkt, tx_offload, + ctx_desc); + + tx_id++; + if (tx_id == txq->nb_tx_desc) + tx_id = 0; + } do { txd = &txr[tx_id]; @@ -1799,14 +1867,17 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, { volatile struct idpf_flex_tx_desc *txd; volatile struct idpf_flex_tx_desc *txr; + union idpf_tx_offload tx_offload = {0}; struct idpf_tx_entry *txe, *txn; struct idpf_tx_entry *sw_ring; struct idpf_tx_queue *txq; struct rte_mbuf *tx_pkt; struct rte_mbuf *m_seg; uint64_t buf_dma_addr; + uint64_t ol_flags; uint16_t tx_last; uint16_t nb_used; + uint16_t nb_ctx; uint16_t td_cmd; uint16_t tx_id; uint16_t nb_tx; @@ -1833,11 +1904,19 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, tx_pkt = *tx_pkts++; RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf); + ol_flags = tx_pkt->ol_flags; + tx_offload.l2_len = tx_pkt->l2_len; + tx_offload.l3_len = tx_pkt->l3_len; + tx_offload.l4_len = tx_pkt->l4_len; + tx_offload.tso_segsz = tx_pkt->tso_segsz; + /* Calculate the number of context descriptors needed. */ + nb_ctx = idpf_calc_context_desc(ol_flags); + /* The number of descriptors that must be allocated for * a packet equals to the number of the segments of that * packet plus 1 context descriptor if needed. */ - nb_used = (uint16_t)(tx_pkt->nb_segs); + nb_used = (uint16_t)(tx_pkt->nb_segs + nb_ctx); tx_last = (uint16_t)(tx_id + nb_used - 1); /* Circular ring */ @@ -1865,6 +1944,29 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, } } + if (nb_ctx != 0) { + /* Setup TX context descriptor if required */ + volatile union idpf_flex_tx_ctx_desc *ctx_txd = + (volatile union idpf_flex_tx_ctx_desc *) + &txr[tx_id]; + + txn = &sw_ring[txe->next_id]; + RTE_MBUF_PREFETCH_TO_FREE(txn->mbuf); + if (txe->mbuf != NULL) { + rte_pktmbuf_free_seg(txe->mbuf); + txe->mbuf = NULL; + } + + /* TSO enabled */ + if ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) != 0) + idpf_set_splitq_tso_ctx(tx_pkt, tx_offload, + ctx_txd); + + txe->last_id = tx_last; + tx_id = txe->next_id; + txe = txn; + } + m_seg = tx_pkt; do { txd = &txr[tx_id]; @@ -1924,6 +2026,9 @@ uint16_t idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) { +#ifdef RTE_LIBRTE_ETHDEV_DEBUG + int ret; +#endif int i; uint64_t ol_flags; struct rte_mbuf *m; @@ -1938,12 +2043,31 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts, rte_errno = EINVAL; return i; } + } else if ((m->tso_segsz < IDPF_MIN_TSO_MSS) || + (m->tso_segsz > IDPF_MAX_TSO_MSS) || + (m->pkt_len > IDPF_MAX_TSO_FRAME_SIZE)) { + /* MSS outside the range are considered malicious */ + rte_errno = EINVAL; + return i; + } + + if ((ol_flags & IDPF_TX_OFFLOAD_NOTSUP_MASK) != 0) { + rte_errno = ENOTSUP; + return i; } if (m->pkt_len < IDPF_MIN_FRAME_SIZE) { rte_errno = EINVAL; return i; } + +#ifdef RTE_LIBRTE_ETHDEV_DEBUG + ret = rte_validate_tx_offload(m); + if (ret != 0) { + rte_errno = -ret; + return i; + } +#endif } return i; diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h index 3853ed55c9..54d297aac6 100644 --- a/drivers/net/idpf/idpf_rxtx.h +++ b/drivers/net/idpf/idpf_rxtx.h @@ -23,6 +23,16 @@ #define IDPF_TX_MAX_MTU_SEG 10 +#define IDPF_MIN_TSO_MSS 88 +#define IDPF_MAX_TSO_MSS 9728 +#define IDPF_MAX_TSO_FRAME_SIZE 262143 +#define IDPF_TX_MAX_MTU_SEG 10 + +#define IDPF_TX_OFFLOAD_MASK RTE_MBUF_F_TX_TCP_SEG + +#define IDPF_TX_OFFLOAD_NOTSUP_MASK \ + (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK) + #define IDPF_GET_PTYPE_SIZE(p) \ (sizeof(struct virtchnl2_ptype) + \ (((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0]))) @@ -115,6 +125,18 @@ struct idpf_tx_queue { struct idpf_tx_queue *complq; }; +/* Offload features */ +union idpf_tx_offload { + uint64_t data; + struct { + uint64_t l2_len:7; /* L2 (MAC) Header Length. */ + uint64_t l3_len:9; /* L3 (IP) Header Length. */ + uint64_t l4_len:8; /* L4 Header Length. */ + uint64_t tso_segsz:16; /* TCP TSO segment size */ + /* uint64_t unused : 24; */ + }; +}; + struct idpf_rxq_ops { void (*release_mbufs)(struct idpf_rx_queue *rxq); }; From patchwork Mon Oct 31 08:33:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xing, Beilei" X-Patchwork-Id: 119357 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DB47BA00C2; Mon, 31 Oct 2022 10:06:40 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CF71B40E5A; Mon, 31 Oct 2022 10:04:57 +0100 (CET) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id 2AC8740DF6 for ; Mon, 31 Oct 2022 10:04:33 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667207073; x=1698743073; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=blACAC8uZaoMxJQ22ZcVWJCHn0tZyILQMH0i5CBv+B8=; b=DjQol829AnEJJgxsQXRT59P+SCGpJHXBYgHBv/sgSZmNzx1DsiXJI5fh CkBkf52B6YMgOkdwyoPZipXc+5WR7sh5xm0y2oFugMBrD3ZbACze4eMTu lSJa0uOFdNqUU4VdnkLkGKiX3iLHWNknX1OSXKvq+WvTuxL8qip9PzLTy rQ4RnzWPfVO2Jn+4r9gdH+N5IXtxXRaIHaULgskeqQJfK8yxbqBorX7Xb C+1Iz1IiWWiT4/63HkmKNUUO2xTaIgy+64E3g1JvSOavQzrTQTssdJYY+ vUMfuQ8yl2YgrCffbOn1IGBuxjMLyvO++vii3d66Vr52WMkHgDyD152WX Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="335507860" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="335507860" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Oct 2022 02:04:32 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="878664957" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="878664957" Received: from dpdk-beileix-3.sh.intel.com ([10.67.110.253]) by fmsmga006.fm.intel.com with ESMTP; 31 Oct 2022 02:04:30 -0700 From: beilei.xing@intel.com To: jingjing.wu@intel.com, beilei.xing@intel.com Cc: dev@dpdk.org, Junfeng Guo , Wenjun Wu Subject: [PATCH v18 17/18] net/idpf: add AVX512 data path for single queue model Date: Mon, 31 Oct 2022 08:33:45 +0000 Message-Id: <20221031083346.16558-18-beilei.xing@intel.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20221031083346.16558-1-beilei.xing@intel.com> References: <20221031051556.98549-1-beilei.xing@intel.com> <20221031083346.16558-1-beilei.xing@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Junfeng Guo Add support of AVX512 vector data path for single queue model. Signed-off-by: Wenjun Wu Signed-off-by: Junfeng Guo --- doc/guides/nics/idpf.rst | 19 + drivers/net/idpf/idpf_ethdev.c | 3 +- drivers/net/idpf/idpf_ethdev.h | 5 + drivers/net/idpf/idpf_rxtx.c | 145 ++++ drivers/net/idpf/idpf_rxtx.h | 21 + drivers/net/idpf/idpf_rxtx_vec_avx512.c | 857 ++++++++++++++++++++++++ drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +++ drivers/net/idpf/meson.build | 28 + 8 files changed, 1177 insertions(+), 1 deletion(-) create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst index c1001d5d0c..3039c61748 100644 --- a/doc/guides/nics/idpf.rst +++ b/doc/guides/nics/idpf.rst @@ -64,3 +64,22 @@ Refer to the document :ref:`compiling and testing a PMD for a NIC tx_offload_capa = RTE_ETH_TX_OFFLOAD_TCP_TSO | - RTE_ETH_TX_OFFLOAD_MULTI_SEGS; + RTE_ETH_TX_OFFLOAD_MULTI_SEGS | + RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE; dev_info->default_txconf = (struct rte_eth_txconf) { .tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH, diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h index 8d0804f603..7d54e5db60 100644 --- a/drivers/net/idpf/idpf_ethdev.h +++ b/drivers/net/idpf/idpf_ethdev.h @@ -162,6 +162,11 @@ struct idpf_adapter { uint32_t max_txq_per_msg; uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned; + + bool rx_vec_allowed; + bool tx_vec_allowed; + bool rx_use_avx512; + bool tx_use_avx512; }; TAILQ_HEAD(idpf_adapter_list, idpf_adapter); diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c index cc296d7ab1..9e20f2b9d3 100644 --- a/drivers/net/idpf/idpf_rxtx.c +++ b/drivers/net/idpf/idpf_rxtx.c @@ -4,9 +4,11 @@ #include #include +#include #include "idpf_ethdev.h" #include "idpf_rxtx.h" +#include "idpf_rxtx_vec_common.h" static int check_rx_thresh(uint16_t nb_desc, uint16_t thresh) @@ -252,6 +254,8 @@ reset_single_rx_queue(struct idpf_rx_queue *rxq) rxq->pkt_first_seg = NULL; rxq->pkt_last_seg = NULL; + rxq->rxrearm_start = 0; + rxq->rxrearm_nb = 0; } static void @@ -2073,25 +2077,166 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts, return i; } +static void __rte_cold +release_rxq_mbufs_vec(struct idpf_rx_queue *rxq) +{ + const uint16_t mask = rxq->nb_rx_desc - 1; + uint16_t i; + + if (rxq->sw_ring == NULL || rxq->rxrearm_nb >= rxq->nb_rx_desc) + return; + + /* free all mbufs that are valid in the ring */ + if (rxq->rxrearm_nb == 0) { + for (i = 0; i < rxq->nb_rx_desc; i++) { + if (rxq->sw_ring[i] != NULL) + rte_pktmbuf_free_seg(rxq->sw_ring[i]); + } + } else { + for (i = rxq->rx_tail; i != rxq->rxrearm_start; i = (i + 1) & mask) { + if (rxq->sw_ring[i] != NULL) + rte_pktmbuf_free_seg(rxq->sw_ring[i]); + } + } + + rxq->rxrearm_nb = rxq->nb_rx_desc; + + /* set all entries to NULL */ + memset(rxq->sw_ring, 0, sizeof(rxq->sw_ring[0]) * rxq->nb_rx_desc); +} + +static const struct idpf_rxq_ops def_singleq_rx_ops_vec = { + .release_mbufs = release_rxq_mbufs_vec, +}; + +static inline int +idpf_singleq_rx_vec_setup_default(struct idpf_rx_queue *rxq) +{ + uintptr_t p; + struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */ + + mb_def.nb_segs = 1; + mb_def.data_off = RTE_PKTMBUF_HEADROOM; + mb_def.port = rxq->port_id; + rte_mbuf_refcnt_set(&mb_def, 1); + + /* prevent compiler reordering: rearm_data covers previous fields */ + rte_compiler_barrier(); + p = (uintptr_t)&mb_def.rearm_data; + rxq->mbuf_initializer = *(uint64_t *)p; + return 0; +} + +int __rte_cold +idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq) +{ + rxq->ops = &def_singleq_rx_ops_vec; + return idpf_singleq_rx_vec_setup_default(rxq); +} + void idpf_set_rx_function(struct rte_eth_dev *dev) { struct idpf_vport *vport = dev->data->dev_private; +#ifdef RTE_ARCH_X86 + struct idpf_adapter *ad = vport->adapter; + struct idpf_rx_queue *rxq; + int i; + + if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH && + rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) { + ad->rx_vec_allowed = true; + + if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512) +#ifdef CC_AVX512_SUPPORT + if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 && + rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1) + ad->rx_use_avx512 = true; +#else + PMD_DRV_LOG(NOTICE, + "AVX512 is not supported in build env"); +#endif /* CC_AVX512_SUPPORT */ + } else { + ad->rx_vec_allowed = false; + } +#endif /* RTE_ARCH_X86 */ + +#ifdef RTE_ARCH_X86 + if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) { + dev->rx_pkt_burst = idpf_splitq_recv_pkts; + } else { + if (ad->rx_vec_allowed) { + for (i = 0; i < dev->data->nb_tx_queues; i++) { + rxq = dev->data->rx_queues[i]; + (void)idpf_singleq_rx_vec_setup(rxq); + } +#ifdef CC_AVX512_SUPPORT + if (ad->rx_use_avx512) { + dev->rx_pkt_burst = idpf_singleq_recv_pkts_avx512; + return; + } +#endif /* CC_AVX512_SUPPORT */ + } + dev->rx_pkt_burst = idpf_singleq_recv_pkts; + } +#else if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) dev->rx_pkt_burst = idpf_splitq_recv_pkts; else dev->rx_pkt_burst = idpf_singleq_recv_pkts; +#endif /* RTE_ARCH_X86 */ } void idpf_set_tx_function(struct rte_eth_dev *dev) { struct idpf_vport *vport = dev->data->dev_private; +#ifdef RTE_ARCH_X86 + struct idpf_adapter *ad = vport->adapter; +#ifdef CC_AVX512_SUPPORT + struct idpf_tx_queue *txq; + int i; +#endif /* CC_AVX512_SUPPORT */ + + if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH && + rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) { + ad->tx_vec_allowed = true; + if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512) +#ifdef CC_AVX512_SUPPORT + if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 && + rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1) + ad->tx_use_avx512 = true; +#else + PMD_DRV_LOG(NOTICE, + "AVX512 is not supported in build env"); +#endif /* CC_AVX512_SUPPORT */ + } else { + ad->tx_vec_allowed = false; + } +#endif /* RTE_ARCH_X86 */ + if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) { dev->tx_pkt_burst = idpf_splitq_xmit_pkts; dev->tx_pkt_prepare = idpf_prep_pkts; } else { +#ifdef RTE_ARCH_X86 + if (ad->tx_vec_allowed) { +#ifdef CC_AVX512_SUPPORT + if (ad->tx_use_avx512) { + for (i = 0; i < dev->data->nb_tx_queues; i++) { + txq = dev->data->tx_queues[i]; + if (txq == NULL) + continue; + idpf_singleq_tx_vec_setup_avx512(txq); + } + dev->tx_pkt_burst = idpf_singleq_xmit_pkts_avx512; + dev->tx_pkt_prepare = idpf_prep_pkts; + return; + } +#endif /* CC_AVX512_SUPPORT */ + } +#endif /* RTE_ARCH_X86 */ dev->tx_pkt_burst = idpf_singleq_xmit_pkts; dev->tx_pkt_prepare = idpf_prep_pkts; } diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h index 54d297aac6..506bf8a57d 100644 --- a/drivers/net/idpf/idpf_rxtx.h +++ b/drivers/net/idpf/idpf_rxtx.h @@ -18,6 +18,12 @@ #define IDPF_RX_MAX_BURST 32 #define IDPF_DEFAULT_RX_FREE_THRESH 32 +/* used for Vector PMD */ +#define IDPF_VPMD_RX_MAX_BURST 32 +#define IDPF_VPMD_TX_MAX_BURST 32 +#define IDPF_VPMD_DESCS_PER_LOOP 4 +#define IDPF_RXQ_REARM_THRESH 64 + #define IDPF_DEFAULT_TX_RS_THRESH 32 #define IDPF_DEFAULT_TX_FREE_THRESH 32 @@ -54,6 +60,11 @@ struct idpf_rx_queue { struct rte_mbuf *pkt_last_seg; /* last segment of current packet */ struct rte_mbuf fake_mbuf; /* dummy mbuf */ + /* used for VPMD */ + uint16_t rxrearm_nb; /* number of remaining to be re-armed */ + uint16_t rxrearm_start; /* the idx we start the re-arming from */ + uint64_t mbuf_initializer; /* value to init mbufs */ + uint16_t rx_nb_avail; uint16_t rx_next_avail; @@ -84,6 +95,10 @@ struct idpf_tx_entry { uint16_t last_id; }; +struct idpf_tx_vec_entry { + struct rte_mbuf *mbuf; +}; + /* Structure associated with each TX queue. */ struct idpf_tx_queue { const struct rte_memzone *mz; /* memzone for Tx ring */ @@ -149,6 +164,7 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, uint16_t nb_desc, unsigned int socket_id, const struct rte_eth_rxconf *rx_conf, struct rte_mempool *mp); +int idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq); int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id); int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id); int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id); @@ -157,16 +173,21 @@ void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid); int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, uint16_t nb_desc, unsigned int socket_id, const struct rte_eth_txconf *tx_conf); +int idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq); int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id); int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id); int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id); void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid); uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts); +uint16_t idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t nb_pkts); uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts); uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts); +uint16_t idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t nb_pkts); uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts); uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, diff --git a/drivers/net/idpf/idpf_rxtx_vec_avx512.c b/drivers/net/idpf/idpf_rxtx_vec_avx512.c new file mode 100644 index 0000000000..fb2b6bb53c --- /dev/null +++ b/drivers/net/idpf/idpf_rxtx_vec_avx512.c @@ -0,0 +1,857 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2022 Intel Corporation + */ + +#include "idpf_rxtx_vec_common.h" + +#include + +#ifndef __INTEL_COMPILER +#pragma GCC diagnostic ignored "-Wcast-qual" +#endif + +#define IDPF_DESCS_PER_LOOP_AVX 8 +#define PKTLEN_SHIFT 10 + +static __rte_always_inline void +idpf_singleq_rearm_common(struct idpf_rx_queue *rxq) +{ + struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start]; + volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring; + uint16_t rx_id; + int i; + + rxdp += rxq->rxrearm_start; + + /* Pull 'n' more MBUFs into the software ring */ + if (rte_mempool_get_bulk(rxq->mp, + (void *)rxp, + IDPF_RXQ_REARM_THRESH) < 0) { + if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >= + rxq->nb_rx_desc) { + __m128i dma_addr0; + + dma_addr0 = _mm_setzero_si128(); + for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) { + rxp[i] = &rxq->fake_mbuf; + _mm_store_si128((__m128i *)&rxdp[i].read, + dma_addr0); + } + } + rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed += + IDPF_RXQ_REARM_THRESH; + return; + } + struct rte_mbuf *mb0, *mb1, *mb2, *mb3; + struct rte_mbuf *mb4, *mb5, *mb6, *mb7; + __m512i dma_addr0_3, dma_addr4_7; + __m512i hdr_room = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM); + /* Initialize the mbufs in vector, process 8 mbufs in one loop */ + for (i = 0; i < IDPF_RXQ_REARM_THRESH; + i += 8, rxp += 8, rxdp += 8) { + __m128i vaddr0, vaddr1, vaddr2, vaddr3; + __m128i vaddr4, vaddr5, vaddr6, vaddr7; + __m256i vaddr0_1, vaddr2_3; + __m256i vaddr4_5, vaddr6_7; + __m512i vaddr0_3, vaddr4_7; + + mb0 = rxp[0]; + mb1 = rxp[1]; + mb2 = rxp[2]; + mb3 = rxp[3]; + mb4 = rxp[4]; + mb5 = rxp[5]; + mb6 = rxp[6]; + mb7 = rxp[7]; + + /* load buf_addr(lo 64bit) and buf_iova(hi 64bit) */ + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_iova) != + offsetof(struct rte_mbuf, buf_addr) + 8); + vaddr0 = _mm_loadu_si128((__m128i *)&mb0->buf_addr); + vaddr1 = _mm_loadu_si128((__m128i *)&mb1->buf_addr); + vaddr2 = _mm_loadu_si128((__m128i *)&mb2->buf_addr); + vaddr3 = _mm_loadu_si128((__m128i *)&mb3->buf_addr); + vaddr4 = _mm_loadu_si128((__m128i *)&mb4->buf_addr); + vaddr5 = _mm_loadu_si128((__m128i *)&mb5->buf_addr); + vaddr6 = _mm_loadu_si128((__m128i *)&mb6->buf_addr); + vaddr7 = _mm_loadu_si128((__m128i *)&mb7->buf_addr); + + /** + * merge 0 & 1, by casting 0 to 256-bit and inserting 1 + * into the high lanes. Similarly for 2 & 3, and so on. + */ + vaddr0_1 = + _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr0), + vaddr1, 1); + vaddr2_3 = + _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr2), + vaddr3, 1); + vaddr4_5 = + _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr4), + vaddr5, 1); + vaddr6_7 = + _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr6), + vaddr7, 1); + vaddr0_3 = + _mm512_inserti64x4(_mm512_castsi256_si512(vaddr0_1), + vaddr2_3, 1); + vaddr4_7 = + _mm512_inserti64x4(_mm512_castsi256_si512(vaddr4_5), + vaddr6_7, 1); + + /* convert pa to dma_addr hdr/data */ + dma_addr0_3 = _mm512_unpackhi_epi64(vaddr0_3, vaddr0_3); + dma_addr4_7 = _mm512_unpackhi_epi64(vaddr4_7, vaddr4_7); + + /* add headroom to pa values */ + dma_addr0_3 = _mm512_add_epi64(dma_addr0_3, hdr_room); + dma_addr4_7 = _mm512_add_epi64(dma_addr4_7, hdr_room); + + /* flush desc with pa dma_addr */ + _mm512_store_si512((__m512i *)&rxdp->read, dma_addr0_3); + _mm512_store_si512((__m512i *)&(rxdp + 4)->read, dma_addr4_7); + } + + rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH; + if (rxq->rxrearm_start >= rxq->nb_rx_desc) + rxq->rxrearm_start = 0; + + rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH; + + rx_id = (uint16_t)((rxq->rxrearm_start == 0) ? + (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1)); + + /* Update the tail pointer on the NIC */ + IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id); +} + +static __rte_always_inline void +idpf_singleq_rearm(struct idpf_rx_queue *rxq) +{ + int i; + uint16_t rx_id; + volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring; + struct rte_mempool_cache *cache = + rte_mempool_default_cache(rxq->mp, rte_lcore_id()); + struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start]; + + rxdp += rxq->rxrearm_start; + + if (unlikely(cache == NULL)) + return idpf_singleq_rearm_common(rxq); + + /* We need to pull 'n' more MBUFs into the software ring from mempool + * We inline the mempool function here, so we can vectorize the copy + * from the cache into the shadow ring. + */ + + /* Can this be satisfied from the cache? */ + if (cache->len < IDPF_RXQ_REARM_THRESH) { + /* No. Backfill the cache first, and then fill from it */ + uint32_t req = IDPF_RXQ_REARM_THRESH + (cache->size - + cache->len); + + /* How many do we require i.e. number to fill the cache + the request */ + int ret = rte_mempool_ops_dequeue_bulk + (rxq->mp, &cache->objs[cache->len], req); + if (ret == 0) { + cache->len += req; + } else { + if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >= + rxq->nb_rx_desc) { + __m128i dma_addr0; + + dma_addr0 = _mm_setzero_si128(); + for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) { + rxp[i] = &rxq->fake_mbuf; + _mm_storeu_si128((__m128i *)&rxdp[i].read, + dma_addr0); + } + } + rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed += + IDPF_RXQ_REARM_THRESH; + return; + } + } + + const __m512i iova_offsets = _mm512_set1_epi64(offsetof + (struct rte_mbuf, buf_iova)); + const __m512i headroom = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM); + + /* to shuffle the addresses to correct slots. Values 4-7 will contain + * zeros, so use 7 for a zero-value. + */ + const __m512i permute_idx = _mm512_set_epi64(7, 7, 3, 1, 7, 7, 2, 0); + + /* Initialize the mbufs in vector, process 8 mbufs in one loop, taking + * from mempool cache and populating both shadow and HW rings + */ + for (i = 0; i < IDPF_RXQ_REARM_THRESH / IDPF_DESCS_PER_LOOP_AVX; i++) { + const __m512i mbuf_ptrs = _mm512_loadu_si512 + (&cache->objs[cache->len - IDPF_DESCS_PER_LOOP_AVX]); + _mm512_storeu_si512(rxp, mbuf_ptrs); + + const __m512i iova_base_addrs = _mm512_i64gather_epi64 + (_mm512_add_epi64(mbuf_ptrs, iova_offsets), + 0, /* base */ + 1 /* scale */); + const __m512i iova_addrs = _mm512_add_epi64(iova_base_addrs, + headroom); + const __m512i iovas0 = _mm512_castsi256_si512 + (_mm512_extracti64x4_epi64(iova_addrs, 0)); + const __m512i iovas1 = _mm512_castsi256_si512 + (_mm512_extracti64x4_epi64(iova_addrs, 1)); + + /* permute leaves desc 2-3 addresses in header address slots 0-1 + * but these are ignored by driver since header split not + * enabled. Similarly for desc 6 & 7. + */ + const __m512i desc0_1 = _mm512_permutexvar_epi64 + (permute_idx, + iovas0); + const __m512i desc2_3 = _mm512_bsrli_epi128(desc0_1, 8); + + const __m512i desc4_5 = _mm512_permutexvar_epi64 + (permute_idx, + iovas1); + const __m512i desc6_7 = _mm512_bsrli_epi128(desc4_5, 8); + + _mm512_storeu_si512((void *)rxdp, desc0_1); + _mm512_storeu_si512((void *)(rxdp + 2), desc2_3); + _mm512_storeu_si512((void *)(rxdp + 4), desc4_5); + _mm512_storeu_si512((void *)(rxdp + 6), desc6_7); + + rxp += IDPF_DESCS_PER_LOOP_AVX; + rxdp += IDPF_DESCS_PER_LOOP_AVX; + cache->len -= IDPF_DESCS_PER_LOOP_AVX; + } + + rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH; + if (rxq->rxrearm_start >= rxq->nb_rx_desc) + rxq->rxrearm_start = 0; + + rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH; + + rx_id = (uint16_t)((rxq->rxrearm_start == 0) ? + (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1)); + + /* Update the tail pointer on the NIC */ + IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id); +} + +#define IDPF_RX_LEN_MASK 0x80808080 +static __rte_always_inline uint16_t +_idpf_singleq_recv_raw_pkts_avx512(struct idpf_rx_queue *rxq, + struct rte_mbuf **rx_pkts, + uint16_t nb_pkts) +{ + const uint32_t *type_table = rxq->adapter->ptype_tbl; + + const __m256i mbuf_init = _mm256_set_epi64x(0, 0, 0, + rxq->mbuf_initializer); + struct rte_mbuf **sw_ring = &rxq->sw_ring[rxq->rx_tail]; + volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring; + + rxdp += rxq->rx_tail; + + rte_prefetch0(rxdp); + + /* nb_pkts has to be floor-aligned to IDPF_DESCS_PER_LOOP_AVX */ + nb_pkts = RTE_ALIGN_FLOOR(nb_pkts, IDPF_DESCS_PER_LOOP_AVX); + + /* See if we need to rearm the RX queue - gives the prefetch a bit + * of time to act + */ + if (rxq->rxrearm_nb > IDPF_RXQ_REARM_THRESH) + idpf_singleq_rearm(rxq); + + /* Before we start moving massive data around, check to see if + * there is actually a packet available + */ + if ((rxdp->flex_nic_wb.status_error0 & + rte_cpu_to_le_32(1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)) == 0) + return 0; + + /* 8 packets DD mask, LSB in each 32-bit value */ + const __m256i dd_check = _mm256_set1_epi32(1); + + /* mask to shuffle from desc. to mbuf (4 descriptors)*/ + const __m512i shuf_msk = + _mm512_set_epi32 + (/* 1st descriptor */ + 0xFFFFFFFF, /* rss set as unknown */ + 0xFFFF0504, /* vlan_macip set as unknown */ + /* octet 15~14, 16 bits data_len */ + 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */ + /* octet 15~14, low 16 bits pkt_len */ + 0xFFFFFFFF, /* pkt_type set as unknown */ + /* 2nd descriptor */ + 0xFFFFFFFF, /* rss set as unknown */ + 0xFFFF0504, /* vlan_macip set as unknown */ + /* octet 15~14, 16 bits data_len */ + 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */ + /* octet 15~14, low 16 bits pkt_len */ + 0xFFFFFFFF, /* pkt_type set as unknown */ + /* 3rd descriptor */ + 0xFFFFFFFF, /* rss set as unknown */ + 0xFFFF0504, /* vlan_macip set as unknown */ + /* octet 15~14, 16 bits data_len */ + 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */ + /* octet 15~14, low 16 bits pkt_len */ + 0xFFFFFFFF, /* pkt_type set as unknown */ + /* 4th descriptor */ + 0xFFFFFFFF, /* rss set as unknown */ + 0xFFFF0504, /* vlan_macip set as unknown */ + /* octet 15~14, 16 bits data_len */ + 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */ + /* octet 15~14, low 16 bits pkt_len */ + 0xFFFFFFFF /* pkt_type set as unknown */ + ); + /** + * compile-time check the shuffle layout is correct. + * NOTE: the first field (lowest address) is given last in set_epi + * calls above. + */ + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != + offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != + offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) != + offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) != + offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12); + + uint16_t i, received; + + for (i = 0, received = 0; i < nb_pkts; + i += IDPF_DESCS_PER_LOOP_AVX, + rxdp += IDPF_DESCS_PER_LOOP_AVX) { + /* step 1, copy over 8 mbuf pointers to rx_pkts array */ + _mm256_storeu_si256((void *)&rx_pkts[i], + _mm256_loadu_si256((void *)&sw_ring[i])); +#ifdef RTE_ARCH_X86_64 + _mm256_storeu_si256 + ((void *)&rx_pkts[i + 4], + _mm256_loadu_si256((void *)&sw_ring[i + 4])); +#endif + + __m512i raw_desc0_3, raw_desc4_7; + const __m128i raw_desc7 = + _mm_load_si128((void *)(rxdp + 7)); + rte_compiler_barrier(); + const __m128i raw_desc6 = + _mm_load_si128((void *)(rxdp + 6)); + rte_compiler_barrier(); + const __m128i raw_desc5 = + _mm_load_si128((void *)(rxdp + 5)); + rte_compiler_barrier(); + const __m128i raw_desc4 = + _mm_load_si128((void *)(rxdp + 4)); + rte_compiler_barrier(); + const __m128i raw_desc3 = + _mm_load_si128((void *)(rxdp + 3)); + rte_compiler_barrier(); + const __m128i raw_desc2 = + _mm_load_si128((void *)(rxdp + 2)); + rte_compiler_barrier(); + const __m128i raw_desc1 = + _mm_load_si128((void *)(rxdp + 1)); + rte_compiler_barrier(); + const __m128i raw_desc0 = + _mm_load_si128((void *)(rxdp + 0)); + + raw_desc4_7 = _mm512_broadcast_i32x4(raw_desc4); + raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc5, 1); + raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc6, 2); + raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc7, 3); + raw_desc0_3 = _mm512_broadcast_i32x4(raw_desc0); + raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc1, 1); + raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc2, 2); + raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc3, 3); + + /** + * convert descriptors 4-7 into mbufs, adjusting length and + * re-arranging fields. Then write into the mbuf + */ + const __m512i len4_7 = _mm512_slli_epi32(raw_desc4_7, + PKTLEN_SHIFT); + const __m512i desc4_7 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK, + raw_desc4_7, + len4_7); + __m512i mb4_7 = _mm512_shuffle_epi8(desc4_7, shuf_msk); + + /** + * to get packet types, shift 64-bit values down 30 bits + * and so ptype is in lower 8-bits in each + */ + const __m512i ptypes4_7 = _mm512_srli_epi64(desc4_7, 16); + const __m256i ptypes6_7 = _mm512_extracti64x4_epi64(ptypes4_7, 1); + const __m256i ptypes4_5 = _mm512_extracti64x4_epi64(ptypes4_7, 0); + const uint8_t ptype7 = _mm256_extract_epi8(ptypes6_7, 16); + const uint8_t ptype6 = _mm256_extract_epi8(ptypes6_7, 0); + const uint8_t ptype5 = _mm256_extract_epi8(ptypes4_5, 16); + const uint8_t ptype4 = _mm256_extract_epi8(ptypes4_5, 0); + + const __m512i ptype4_7 = _mm512_set_epi32 + (0, 0, 0, type_table[ptype7], + 0, 0, 0, type_table[ptype6], + 0, 0, 0, type_table[ptype5], + 0, 0, 0, type_table[ptype4]); + mb4_7 = _mm512_mask_blend_epi32(0x1111, mb4_7, ptype4_7); + + /** + * convert descriptors 0-3 into mbufs, adjusting length and + * re-arranging fields. Then write into the mbuf + */ + const __m512i len0_3 = _mm512_slli_epi32(raw_desc0_3, + PKTLEN_SHIFT); + const __m512i desc0_3 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK, + raw_desc0_3, + len0_3); + __m512i mb0_3 = _mm512_shuffle_epi8(desc0_3, shuf_msk); + + /* get the packet types */ + const __m512i ptypes0_3 = _mm512_srli_epi64(desc0_3, 16); + const __m256i ptypes2_3 = _mm512_extracti64x4_epi64(ptypes0_3, 1); + const __m256i ptypes0_1 = _mm512_extracti64x4_epi64(ptypes0_3, 0); + const uint8_t ptype3 = _mm256_extract_epi8(ptypes2_3, 16); + const uint8_t ptype2 = _mm256_extract_epi8(ptypes2_3, 0); + const uint8_t ptype1 = _mm256_extract_epi8(ptypes0_1, 16); + const uint8_t ptype0 = _mm256_extract_epi8(ptypes0_1, 0); + + const __m512i ptype0_3 = _mm512_set_epi32 + (0, 0, 0, type_table[ptype3], + 0, 0, 0, type_table[ptype2], + 0, 0, 0, type_table[ptype1], + 0, 0, 0, type_table[ptype0]); + mb0_3 = _mm512_mask_blend_epi32(0x1111, mb0_3, ptype0_3); + + /** + * use permute/extract to get status content + * After the operations, the packets status flags are in the + * order (hi->lo): [1, 3, 5, 7, 0, 2, 4, 6] + */ + /* merge the status bits into one register */ + const __m512i status_permute_msk = _mm512_set_epi32 + (0, 0, 0, 0, + 0, 0, 0, 0, + 22, 30, 6, 14, + 18, 26, 2, 10); + const __m512i raw_status0_7 = _mm512_permutex2var_epi32 + (raw_desc4_7, status_permute_msk, raw_desc0_3); + __m256i status0_7 = _mm512_extracti64x4_epi64 + (raw_status0_7, 0); + + /* now do flag manipulation */ + + /** + * At this point, we have the 8 sets of flags in the low 16-bits + * of each 32-bit value. + * We want to extract these, and merge them with the mbuf init + * data so we can do a single write to the mbuf to set the flags + * and all the other initialization fields. Extracting the + * appropriate flags means that we have to do a shift and blend + * for each mbuf before we do the write. However, we can also + * add in the previously computed rx_descriptor fields to + * make a single 256-bit write per mbuf + */ + /* check the structure matches expectations */ + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != + offsetof(struct rte_mbuf, rearm_data) + 8); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) != + RTE_ALIGN(offsetof(struct rte_mbuf, + rearm_data), + 16)); + /* build up data and do writes */ + __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5, + rearm6, rearm7; + const __m256i mb4_5 = _mm512_extracti64x4_epi64(mb4_7, 0); + const __m256i mb6_7 = _mm512_extracti64x4_epi64(mb4_7, 1); + const __m256i mb0_1 = _mm512_extracti64x4_epi64(mb0_3, 0); + const __m256i mb2_3 = _mm512_extracti64x4_epi64(mb0_3, 1); + + rearm6 = _mm256_permute2f128_si256(mbuf_init, mb6_7, 0x20); + rearm4 = _mm256_permute2f128_si256(mbuf_init, mb4_5, 0x20); + rearm2 = _mm256_permute2f128_si256(mbuf_init, mb2_3, 0x20); + rearm0 = _mm256_permute2f128_si256(mbuf_init, mb0_1, 0x20); + + /* write to mbuf */ + _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data, + rearm6); + _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data, + rearm4); + _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data, + rearm2); + _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data, + rearm0); + + rearm7 = _mm256_blend_epi32(mbuf_init, mb6_7, 0xF0); + rearm5 = _mm256_blend_epi32(mbuf_init, mb4_5, 0xF0); + rearm3 = _mm256_blend_epi32(mbuf_init, mb2_3, 0xF0); + rearm1 = _mm256_blend_epi32(mbuf_init, mb0_1, 0xF0); + + /* again write to mbufs */ + _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data, + rearm7); + _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data, + rearm5); + _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data, + rearm3); + _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data, + rearm1); + + /* perform dd_check */ + status0_7 = _mm256_and_si256(status0_7, dd_check); + status0_7 = _mm256_packs_epi32(status0_7, + _mm256_setzero_si256()); + + uint64_t burst = __builtin_popcountll + (_mm_cvtsi128_si64 + (_mm256_extracti128_si256 + (status0_7, 1))); + burst += __builtin_popcountll + (_mm_cvtsi128_si64 + (_mm256_castsi256_si128(status0_7))); + received += burst; + if (burst != IDPF_DESCS_PER_LOOP_AVX) + break; + } + + /* update tail pointers */ + rxq->rx_tail += received; + rxq->rx_tail &= (rxq->nb_rx_desc - 1); + if ((rxq->rx_tail & 1) == 1 && received > 1) { /* keep aligned */ + rxq->rx_tail--; + received--; + } + rxq->rxrearm_nb += received; + return received; +} + +/** + * Notice: + * - nb_pkts < IDPF_DESCS_PER_LOOP, just return no packet + */ +uint16_t +idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t nb_pkts) +{ + return _idpf_singleq_recv_raw_pkts_avx512(rx_queue, rx_pkts, nb_pkts); +} + +static __rte_always_inline int +idpf_tx_free_bufs_avx512(struct idpf_tx_queue *txq) +{ + struct idpf_tx_vec_entry *txep; + uint32_t n; + uint32_t i; + int nb_free = 0; + struct rte_mbuf *m, *free[txq->rs_thresh]; + + /* check DD bits on threshold descriptor */ + if ((txq->tx_ring[txq->next_dd].qw1.cmd_dtype & + rte_cpu_to_le_64(IDPF_TXD_QW1_DTYPE_M)) != + rte_cpu_to_le_64(IDPF_TX_DESC_DTYPE_DESC_DONE)) + return 0; + + n = txq->rs_thresh; + + /* first buffer to free from S/W ring is at index + * tx_next_dd - (tx_rs_thresh-1) + */ + txep = (void *)txq->sw_ring; + txep += txq->next_dd - (n - 1); + + if (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE && (n & 31) == 0) { + struct rte_mempool *mp = txep[0].mbuf->pool; + struct rte_mempool_cache *cache = rte_mempool_default_cache(mp, + rte_lcore_id()); + void **cache_objs; + + if (cache == NULL || cache->len == 0) + goto normal; + + cache_objs = &cache->objs[cache->len]; + + if (n > RTE_MEMPOOL_CACHE_MAX_SIZE) { + rte_mempool_ops_enqueue_bulk(mp, (void *)txep, n); + goto done; + } + + /* The cache follows the following algorithm + * 1. Add the objects to the cache + * 2. Anything greater than the cache min value (if it crosses the + * cache flush threshold) is flushed to the ring. + */ + /* Add elements back into the cache */ + uint32_t copied = 0; + /* n is multiple of 32 */ + while (copied < n) { + const __m512i a = _mm512_loadu_si512(&txep[copied]); + const __m512i b = _mm512_loadu_si512(&txep[copied + 8]); + const __m512i c = _mm512_loadu_si512(&txep[copied + 16]); + const __m512i d = _mm512_loadu_si512(&txep[copied + 24]); + + _mm512_storeu_si512(&cache_objs[copied], a); + _mm512_storeu_si512(&cache_objs[copied + 8], b); + _mm512_storeu_si512(&cache_objs[copied + 16], c); + _mm512_storeu_si512(&cache_objs[copied + 24], d); + copied += 32; + } + cache->len += n; + + if (cache->len >= cache->flushthresh) { + rte_mempool_ops_enqueue_bulk(mp, + &cache->objs[cache->size], + cache->len - cache->size); + cache->len = cache->size; + } + goto done; + } + +normal: + m = rte_pktmbuf_prefree_seg(txep[0].mbuf); + if (likely(m != NULL)) { + free[0] = m; + nb_free = 1; + for (i = 1; i < n; i++) { + m = rte_pktmbuf_prefree_seg(txep[i].mbuf); + if (likely(m != NULL)) { + if (likely(m->pool == free[0]->pool)) { + free[nb_free++] = m; + } else { + rte_mempool_put_bulk(free[0]->pool, + (void *)free, + nb_free); + free[0] = m; + nb_free = 1; + } + } + } + rte_mempool_put_bulk(free[0]->pool, (void **)free, nb_free); + } else { + for (i = 1; i < n; i++) { + m = rte_pktmbuf_prefree_seg(txep[i].mbuf); + if (m != NULL) + rte_mempool_put(m->pool, m); + } + } + +done: + /* buffers were freed, update counters */ + txq->nb_free = (uint16_t)(txq->nb_free + txq->rs_thresh); + txq->next_dd = (uint16_t)(txq->next_dd + txq->rs_thresh); + if (txq->next_dd >= txq->nb_tx_desc) + txq->next_dd = (uint16_t)(txq->rs_thresh - 1); + + return txq->rs_thresh; +} + +static __rte_always_inline void +tx_backlog_entry_avx512(struct idpf_tx_vec_entry *txep, + struct rte_mbuf **tx_pkts, uint16_t nb_pkts) +{ + int i; + + for (i = 0; i < (int)nb_pkts; ++i) + txep[i].mbuf = tx_pkts[i]; +} + +#define IDPF_FLEX_TXD_QW1_BUF_SZ_S 48 +static __rte_always_inline void +idpf_vtx1(volatile struct idpf_flex_tx_desc *txdp, + struct rte_mbuf *pkt, uint64_t flags) +{ + uint64_t high_qw = + (IDPF_TX_DESC_DTYPE_FLEX_DATA << IDPF_FLEX_TXD_QW1_DTYPE_S | + ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S) | + ((uint64_t)pkt->data_len << IDPF_FLEX_TXD_QW1_BUF_SZ_S)); + + __m128i descriptor = _mm_set_epi64x(high_qw, + pkt->buf_iova + pkt->data_off); + _mm_storeu_si128((__m128i *)txdp, descriptor); +} + +#define IDPF_TX_LEN_MASK 0xAA +#define IDPF_TX_OFF_MASK 0x55 +static __rte_always_inline void +idpf_vtx(volatile struct idpf_flex_tx_desc *txdp, + struct rte_mbuf **pkt, uint16_t nb_pkts, uint64_t flags) +{ + const uint64_t hi_qw_tmpl = (IDPF_TX_DESC_DTYPE_FLEX_DATA | + ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S)); + + /* if unaligned on 32-bit boundary, do one to align */ + if (((uintptr_t)txdp & 0x1F) != 0 && nb_pkts != 0) { + idpf_vtx1(txdp, *pkt, flags); + nb_pkts--, txdp++, pkt++; + } + + /* do 4 at a time while possible, in bursts */ + for (; nb_pkts > 3; txdp += 4, pkt += 4, nb_pkts -= 4) { + uint64_t hi_qw3 = + hi_qw_tmpl | + ((uint64_t)pkt[3]->data_len << + IDPF_FLEX_TXD_QW1_BUF_SZ_S); + uint64_t hi_qw2 = + hi_qw_tmpl | + ((uint64_t)pkt[2]->data_len << + IDPF_FLEX_TXD_QW1_BUF_SZ_S); + uint64_t hi_qw1 = + hi_qw_tmpl | + ((uint64_t)pkt[1]->data_len << + IDPF_FLEX_TXD_QW1_BUF_SZ_S); + uint64_t hi_qw0 = + hi_qw_tmpl | + ((uint64_t)pkt[0]->data_len << + IDPF_FLEX_TXD_QW1_BUF_SZ_S); + + __m512i desc0_3 = + _mm512_set_epi64 + (hi_qw3, + pkt[3]->buf_iova + pkt[3]->data_off, + hi_qw2, + pkt[2]->buf_iova + pkt[2]->data_off, + hi_qw1, + pkt[1]->buf_iova + pkt[1]->data_off, + hi_qw0, + pkt[0]->buf_iova + pkt[0]->data_off); + _mm512_storeu_si512((void *)txdp, desc0_3); + } + + /* do any last ones */ + while (nb_pkts) { + idpf_vtx1(txdp, *pkt, flags); + txdp++, pkt++, nb_pkts--; + } +} + +static __rte_always_inline uint16_t +idpf_xmit_fixed_burst_vec_avx512(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t nb_pkts) +{ + struct idpf_tx_queue *txq = tx_queue; + volatile struct idpf_flex_tx_desc *txdp; + struct idpf_tx_vec_entry *txep; + uint16_t n, nb_commit, tx_id; + uint64_t flags = IDPF_TX_FLEX_DESC_CMD_EOP; + uint64_t rs = IDPF_TX_FLEX_DESC_CMD_RS | flags; + + /* cross rx_thresh boundary is not allowed */ + nb_pkts = RTE_MIN(nb_pkts, txq->rs_thresh); + + if (txq->nb_free < txq->free_thresh) + idpf_tx_free_bufs_avx512(txq); + + nb_pkts = (uint16_t)RTE_MIN(txq->nb_free, nb_pkts); + nb_commit = nb_pkts; + if (unlikely(nb_pkts == 0)) + return 0; + + tx_id = txq->tx_tail; + txdp = &txq->tx_ring[tx_id]; + txep = (void *)txq->sw_ring; + txep += tx_id; + + txq->nb_free = (uint16_t)(txq->nb_free - nb_pkts); + + n = (uint16_t)(txq->nb_tx_desc - tx_id); + if (nb_commit >= n) { + tx_backlog_entry_avx512(txep, tx_pkts, n); + + idpf_vtx(txdp, tx_pkts, n - 1, flags); + tx_pkts += (n - 1); + txdp += (n - 1); + + idpf_vtx1(txdp, *tx_pkts++, rs); + + nb_commit = (uint16_t)(nb_commit - n); + + tx_id = 0; + txq->next_rs = (uint16_t)(txq->rs_thresh - 1); + + /* avoid reach the end of ring */ + txdp = &txq->tx_ring[tx_id]; + txep = (void *)txq->sw_ring; + txep += tx_id; + } + + tx_backlog_entry_avx512(txep, tx_pkts, nb_commit); + + idpf_vtx(txdp, tx_pkts, nb_commit, flags); + + tx_id = (uint16_t)(tx_id + nb_commit); + if (tx_id > txq->next_rs) { + txq->tx_ring[txq->next_rs].qw1.cmd_dtype |= + rte_cpu_to_le_64(((uint64_t)IDPF_TX_FLEX_DESC_CMD_RS) << + IDPF_FLEX_TXD_QW1_CMD_S); + txq->next_rs = + (uint16_t)(txq->next_rs + txq->rs_thresh); + } + + txq->tx_tail = tx_id; + + IDPF_PCI_REG_WRITE(txq->qtx_tail, txq->tx_tail); + + return nb_pkts; +} + +static __rte_always_inline uint16_t +idpf_xmit_pkts_vec_avx512_cmn(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t nb_pkts) +{ + uint16_t nb_tx = 0; + struct idpf_tx_queue *txq = tx_queue; + + while (nb_pkts) { + uint16_t ret, num; + + num = (uint16_t)RTE_MIN(nb_pkts, txq->rs_thresh); + ret = idpf_xmit_fixed_burst_vec_avx512(tx_queue, &tx_pkts[nb_tx], + num); + nb_tx += ret; + nb_pkts -= ret; + if (ret < num) + break; + } + + return nb_tx; +} + +uint16_t +idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t nb_pkts) +{ + return idpf_xmit_pkts_vec_avx512_cmn(tx_queue, tx_pkts, nb_pkts); +} + +static inline void +idpf_singleq_tx_release_mbufs_avx512(struct idpf_tx_queue *txq) +{ + unsigned int i; + const uint16_t max_desc = (uint16_t)(txq->nb_tx_desc - 1); + struct idpf_tx_vec_entry *swr = (void *)txq->sw_ring; + + if (txq->sw_ring == NULL || txq->nb_free == max_desc) + return; + + i = txq->next_dd - txq->rs_thresh + 1; + if (txq->tx_tail < i) { + for (; i < txq->nb_tx_desc; i++) { + rte_pktmbuf_free_seg(swr[i].mbuf); + swr[i].mbuf = NULL; + } + i = 0; + } +} + +static const struct idpf_txq_ops avx512_singleq_tx_vec_ops = { + .release_mbufs = idpf_singleq_tx_release_mbufs_avx512, +}; + +int __rte_cold +idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq) +{ + txq->ops = &avx512_singleq_tx_vec_ops; + return 0; +} diff --git a/drivers/net/idpf/idpf_rxtx_vec_common.h b/drivers/net/idpf/idpf_rxtx_vec_common.h new file mode 100644 index 0000000000..0f4e10e154 --- /dev/null +++ b/drivers/net/idpf/idpf_rxtx_vec_common.h @@ -0,0 +1,100 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2022 Intel Corporation + */ + +#ifndef _IDPF_RXTX_VEC_COMMON_H_ +#define _IDPF_RXTX_VEC_COMMON_H_ +#include +#include +#include + +#include "idpf_ethdev.h" +#include "idpf_rxtx.h" + +#ifndef __INTEL_COMPILER +#pragma GCC diagnostic ignored "-Wcast-qual" +#endif + +#define IDPF_VECTOR_PATH 0 +#define ICE_RX_NO_VECTOR_FLAGS ( \ + RTE_ETH_RX_OFFLOAD_IPV4_CKSUM | \ + RTE_ETH_RX_OFFLOAD_UDP_CKSUM | \ + RTE_ETH_RX_OFFLOAD_TCP_CKSUM | \ + RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM | \ + RTE_ETH_RX_OFFLOAD_TIMESTAMP) +#define ICE_TX_NO_VECTOR_FLAGS ( \ + RTE_ETH_TX_OFFLOAD_TCP_TSO | \ + RTE_ETH_TX_OFFLOAD_MULTI_SEGS) + +static inline int +idpf_rx_vec_queue_default(struct idpf_rx_queue *rxq) +{ + if (rxq == NULL) + return -1; + + if (rte_is_power_of_2(rxq->nb_rx_desc) == 0) + return -1; + + if (rxq->rx_free_thresh < IDPF_VPMD_RX_MAX_BURST) + return -1; + + if ((rxq->nb_rx_desc % rxq->rx_free_thresh) != 0) + return -1; + + if ((rxq->offloads & ICE_RX_NO_VECTOR_FLAGS) != 0) + return -1; + + return IDPF_VECTOR_PATH; +} + +static inline int +idpf_tx_vec_queue_default(struct idpf_tx_queue *txq) +{ + if (txq == NULL) + return -1; + + if (txq->rs_thresh < IDPF_VPMD_TX_MAX_BURST || + (txq->rs_thresh & 3) != 0) + return -1; + + if ((txq->offloads & ICE_TX_NO_VECTOR_FLAGS) != 0) + return -1; + + return IDPF_VECTOR_PATH; +} + +static inline int +idpf_rx_vec_dev_check_default(struct rte_eth_dev *dev) +{ + int i; + struct idpf_rx_queue *rxq; + int ret = 0; + + for (i = 0; i < dev->data->nb_rx_queues; i++) { + rxq = dev->data->rx_queues[i]; + ret = (idpf_rx_vec_queue_default(rxq)); + if (ret < 0) + return -1; + } + + return IDPF_VECTOR_PATH; +} + +static inline int +idpf_tx_vec_dev_check_default(struct rte_eth_dev *dev) +{ + int i; + struct idpf_tx_queue *txq; + int ret = 0; + + for (i = 0; i < dev->data->nb_tx_queues; i++) { + txq = dev->data->tx_queues[i]; + ret = idpf_tx_vec_queue_default(txq); + if (ret < 0) + return -1; + } + + return IDPF_VECTOR_PATH; +} + +#endif /*_IDPF_RXTX_VEC_COMMON_H_*/ diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build index b632b76656..da99c098ab 100644 --- a/drivers/net/idpf/meson.build +++ b/drivers/net/idpf/meson.build @@ -14,3 +14,31 @@ sources = files( 'idpf_rxtx.c', 'idpf_vchnl.c', ) + +if arch_subdir == 'x86' + idpf_avx512_cpu_support = ( + cc.get_define('__AVX512F__', args: machine_args) != '' and + cc.get_define('__AVX512BW__', args: machine_args) != '' + ) + + idpf_avx512_cc_support = ( + not machine_args.contains('-mno-avx512f') and + cc.has_argument('-mavx512f') and + cc.has_argument('-mavx512bw') + ) + + if idpf_avx512_cpu_support == true or idpf_avx512_cc_support == true + cflags += ['-DCC_AVX512_SUPPORT'] + avx512_args = [cflags, '-mavx512f', '-mavx512bw'] + if cc.has_argument('-march=skylake-avx512') + avx512_args += '-march=skylake-avx512' + endif + idpf_avx512_lib = static_library('idpf_avx512_lib', + 'idpf_rxtx_vec_avx512.c', + dependencies: [static_rte_common_idpf, static_rte_ethdev, static_rte_bus_pci, + static_rte_kvargs, static_rte_hash], + include_directories: includes, + c_args: avx512_args) + objs += idpf_avx512_lib.extract_objects('idpf_rxtx_vec_avx512.c') + endif +endif From patchwork Mon Oct 31 08:33:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xing, Beilei" X-Patchwork-Id: 119358 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 18AD5A00C2; Mon, 31 Oct 2022 10:06:48 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A179C42B83; Mon, 31 Oct 2022 10:04:58 +0100 (CET) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id ECA0340A81 for ; Mon, 31 Oct 2022 10:04:34 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667207075; x=1698743075; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=CxsX7yzVlbWOWDnRvDYVwH3sC4EFQ+fKvfi0jjtRT2s=; b=O9ARewIHcRnneSqd4DXrCCEbW7ug4LBE019EvOcvM/hN1epD4BTFcdjF mlwFwAS0tjODOu8oVM5a5ImpmMLkKFnDtJ2YPqPTr/2M2peIIjVzuDfxX awZNlCGJQPol3J5gpz2xshu8V1LDH9L0/aNl/LDr0Iw/FGwICYl3MvUyA c0aO8V2a/l/Wo9PlGg9qqxHZ7zH6SC/eENZ33j7qo3a+Uz6ISy0Czm+B/ CDIMvhqyRkVuUcYxqzR7apg1gVIt6taW/268SyAu2Wv9F4Kqlu56D5fLB i/7r8zHalvShZ7h5Q08cuxf5HjBBbr8fpQ9fT+oNLviePfppn9VMddkS5 g==; X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="335507868" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="335507868" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Oct 2022 02:04:34 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="878664969" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="878664969" Received: from dpdk-beileix-3.sh.intel.com ([10.67.110.253]) by fmsmga006.fm.intel.com with ESMTP; 31 Oct 2022 02:04:32 -0700 From: beilei.xing@intel.com To: jingjing.wu@intel.com, beilei.xing@intel.com Cc: dev@dpdk.org, Junfeng Guo , Wenjing Qiao Subject: [PATCH v18 18/18] net/idpf: add support for timestamp offload Date: Mon, 31 Oct 2022 08:33:46 +0000 Message-Id: <20221031083346.16558-19-beilei.xing@intel.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20221031083346.16558-1-beilei.xing@intel.com> References: <20221031051556.98549-1-beilei.xing@intel.com> <20221031083346.16558-1-beilei.xing@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Junfeng Guo Add support for timestamp offload. Signed-off-by: Wenjing Qiao Signed-off-by: Junfeng Guo --- doc/guides/nics/features/idpf.ini | 1 + drivers/net/idpf/idpf_ethdev.c | 5 +- drivers/net/idpf/idpf_ethdev.h | 3 ++ drivers/net/idpf/idpf_rxtx.c | 65 ++++++++++++++++++++++ drivers/net/idpf/idpf_rxtx.h | 90 +++++++++++++++++++++++++++++++ 5 files changed, 163 insertions(+), 1 deletion(-) diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini index d82b4aa0ff..099fd7f216 100644 --- a/doc/guides/nics/features/idpf.ini +++ b/doc/guides/nics/features/idpf.ini @@ -11,6 +11,7 @@ MTU update = Y TSO = P L3 checksum offload = P L4 checksum offload = P +Timestamp offload = P Linux = Y x86-32 = Y x86-64 = Y diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c index cd4ebcc2c6..50aac65daf 100644 --- a/drivers/net/idpf/idpf_ethdev.c +++ b/drivers/net/idpf/idpf_ethdev.c @@ -22,6 +22,8 @@ rte_spinlock_t idpf_adapter_lock; struct idpf_adapter_list idpf_adapter_list; bool idpf_adapter_list_init; +uint64_t idpf_timestamp_dynflag; + static const char * const idpf_valid_args[] = { IDPF_TX_SINGLE_Q, IDPF_RX_SINGLE_Q, @@ -65,7 +67,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) RTE_ETH_RX_OFFLOAD_IPV4_CKSUM | RTE_ETH_RX_OFFLOAD_UDP_CKSUM | RTE_ETH_RX_OFFLOAD_TCP_CKSUM | - RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM; + RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM | + RTE_ETH_RX_OFFLOAD_TIMESTAMP; dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_TCP_TSO | diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h index 7d54e5db60..ccdf4abe40 100644 --- a/drivers/net/idpf/idpf_ethdev.h +++ b/drivers/net/idpf/idpf_ethdev.h @@ -167,6 +167,9 @@ struct idpf_adapter { bool tx_vec_allowed; bool rx_use_avx512; bool tx_use_avx512; + + /* For PTP */ + uint64_t time_hw; }; TAILQ_HEAD(idpf_adapter_list, idpf_adapter); diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c index 9e20f2b9d3..bafa007faf 100644 --- a/drivers/net/idpf/idpf_rxtx.c +++ b/drivers/net/idpf/idpf_rxtx.c @@ -10,6 +10,8 @@ #include "idpf_rxtx.h" #include "idpf_rxtx_vec_common.h" +static int idpf_timestamp_dynfield_offset = -1; + static int check_rx_thresh(uint16_t nb_desc, uint16_t thresh) { @@ -900,6 +902,24 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc, socket_id, tx_conf); } + +static int +idpf_register_ts_mbuf(struct idpf_rx_queue *rxq) +{ + int err; + if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0) { + /* Register mbuf field and flag for Rx timestamp */ + err = rte_mbuf_dyn_rx_timestamp_register(&idpf_timestamp_dynfield_offset, + &idpf_timestamp_dynflag); + if (err != 0) { + PMD_DRV_LOG(ERR, + "Cannot register mbuf field/flag for timestamp"); + return -EINVAL; + } + } + return 0; +} + static int idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq) { @@ -993,6 +1013,13 @@ idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id) return -EINVAL; } + err = idpf_register_ts_mbuf(rxq); + if (err != 0) { + PMD_DRV_LOG(ERR, "fail to regidter timestamp mbuf %u", + rx_queue_id); + return -EIO; + } + if (rxq->bufq1 == NULL) { /* Single queue */ err = idpf_alloc_single_rxq_mbufs(rxq); @@ -1354,6 +1381,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, struct idpf_rx_queue *rxq; const uint32_t *ptype_tbl; uint8_t status_err0_qw1; + struct idpf_adapter *ad; struct rte_mbuf *rxm; uint16_t rx_id_bufq1; uint16_t rx_id_bufq2; @@ -1363,9 +1391,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t gen_id; uint16_t rx_id; uint16_t nb_rx; + uint64_t ts_ns; nb_rx = 0; rxq = rx_queue; + ad = rxq->adapter; if (unlikely(rxq == NULL) || unlikely(!rxq->q_started)) return nb_rx; @@ -1376,6 +1406,9 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, rx_desc_ring = rxq->rx_ring; ptype_tbl = rxq->adapter->ptype_tbl; + if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0) + rxq->hw_register_set = 1; + while (nb_rx < nb_pkts) { rx_desc = &rx_desc_ring[rx_id]; @@ -1431,6 +1464,18 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, status_err0_qw1 = rx_desc->status_err0_qw1; pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1); pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc); + if (idpf_timestamp_dynflag > 0 && + (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)) { + /* timestamp */ + ts_ns = idpf_tstamp_convert_32b_64b(ad, + rxq->hw_register_set, + rte_le_to_cpu_32(rx_desc->ts_high)); + rxq->hw_register_set = 0; + *RTE_MBUF_DYNFIELD(rxm, + idpf_timestamp_dynfield_offset, + rte_mbuf_timestamp_t *) = ts_ns; + rxm->ol_flags |= idpf_timestamp_dynflag; + } rxm->ol_flags |= pkt_flags; @@ -1732,18 +1777,22 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, const uint32_t *ptype_tbl; uint16_t rx_id, nb_hold; struct rte_eth_dev *dev; + struct idpf_adapter *ad; uint16_t rx_packet_len; struct rte_mbuf *rxm; struct rte_mbuf *nmb; uint16_t rx_status0; uint64_t pkt_flags; uint64_t dma_addr; + uint64_t ts_ns; uint16_t nb_rx; nb_rx = 0; nb_hold = 0; rxq = rx_queue; + ad = rxq->adapter; + if (unlikely(rxq == NULL) || unlikely(!rxq->q_started)) return nb_rx; @@ -1751,6 +1800,9 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, rx_ring = rxq->rx_ring; ptype_tbl = rxq->adapter->ptype_tbl; + if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0) + rxq->hw_register_set = 1; + while (nb_rx < nb_pkts) { rxdp = &rx_ring[rx_id]; rx_status0 = rte_le_to_cpu_16(rxdp->flex_nic_wb.status_error0); @@ -1810,6 +1862,19 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, rxm->ol_flags |= pkt_flags; + if (idpf_timestamp_dynflag > 0 && + (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0) { + /* timestamp */ + ts_ns = idpf_tstamp_convert_32b_64b(ad, + rxq->hw_register_set, + rte_le_to_cpu_32(rxd.flex_nic_wb.flex_ts.ts_high)); + rxq->hw_register_set = 0; + *RTE_MBUF_DYNFIELD(rxm, + idpf_timestamp_dynfield_offset, + rte_mbuf_timestamp_t *) = ts_ns; + rxm->ol_flags |= idpf_timestamp_dynflag; + } + rx_pkts[nb_rx++] = rxm; } rxq->rx_tail = rx_id; diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h index 506bf8a57d..a98f0285c8 100644 --- a/drivers/net/idpf/idpf_rxtx.h +++ b/drivers/net/idpf/idpf_rxtx.h @@ -7,6 +7,41 @@ #include "idpf_ethdev.h" +/* MTS */ +#define GLTSYN_CMD_SYNC_0_0 (PF_TIMESYNC_BASE + 0x0) +#define PF_GLTSYN_SHTIME_0_0 (PF_TIMESYNC_BASE + 0x4) +#define PF_GLTSYN_SHTIME_L_0 (PF_TIMESYNC_BASE + 0x8) +#define PF_GLTSYN_SHTIME_H_0 (PF_TIMESYNC_BASE + 0xC) +#define GLTSYN_ART_L_0 (PF_TIMESYNC_BASE + 0x10) +#define GLTSYN_ART_H_0 (PF_TIMESYNC_BASE + 0x14) +#define PF_GLTSYN_SHTIME_0_1 (PF_TIMESYNC_BASE + 0x24) +#define PF_GLTSYN_SHTIME_L_1 (PF_TIMESYNC_BASE + 0x28) +#define PF_GLTSYN_SHTIME_H_1 (PF_TIMESYNC_BASE + 0x2C) +#define PF_GLTSYN_SHTIME_0_2 (PF_TIMESYNC_BASE + 0x44) +#define PF_GLTSYN_SHTIME_L_2 (PF_TIMESYNC_BASE + 0x48) +#define PF_GLTSYN_SHTIME_H_2 (PF_TIMESYNC_BASE + 0x4C) +#define PF_GLTSYN_SHTIME_0_3 (PF_TIMESYNC_BASE + 0x64) +#define PF_GLTSYN_SHTIME_L_3 (PF_TIMESYNC_BASE + 0x68) +#define PF_GLTSYN_SHTIME_H_3 (PF_TIMESYNC_BASE + 0x6C) + +#define PF_TIMESYNC_BAR4_BASE 0x0E400000 +#define GLTSYN_ENA (PF_TIMESYNC_BAR4_BASE + 0x90) +#define GLTSYN_CMD (PF_TIMESYNC_BAR4_BASE + 0x94) +#define GLTSYC_TIME_L (PF_TIMESYNC_BAR4_BASE + 0x104) +#define GLTSYC_TIME_H (PF_TIMESYNC_BAR4_BASE + 0x108) + +#define GLTSYN_CMD_SYNC_0_4 (PF_TIMESYNC_BAR4_BASE + 0x110) +#define PF_GLTSYN_SHTIME_L_4 (PF_TIMESYNC_BAR4_BASE + 0x118) +#define PF_GLTSYN_SHTIME_H_4 (PF_TIMESYNC_BAR4_BASE + 0x11C) +#define GLTSYN_INCVAL_L (PF_TIMESYNC_BAR4_BASE + 0x150) +#define GLTSYN_INCVAL_H (PF_TIMESYNC_BAR4_BASE + 0x154) +#define GLTSYN_SHADJ_L (PF_TIMESYNC_BAR4_BASE + 0x158) +#define GLTSYN_SHADJ_H (PF_TIMESYNC_BAR4_BASE + 0x15C) + +#define GLTSYN_CMD_SYNC_0_5 (PF_TIMESYNC_BAR4_BASE + 0x130) +#define PF_GLTSYN_SHTIME_L_5 (PF_TIMESYNC_BAR4_BASE + 0x138) +#define PF_GLTSYN_SHTIME_H_5 (PF_TIMESYNC_BAR4_BASE + 0x13C) + /* In QLEN must be whole number of 32 descriptors. */ #define IDPF_ALIGN_RING_DESC 32 #define IDPF_MIN_RING_DESC 32 @@ -43,6 +78,8 @@ (sizeof(struct virtchnl2_ptype) + \ (((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0]))) +extern uint64_t idpf_timestamp_dynflag; + struct idpf_rx_queue { struct idpf_adapter *adapter; /* the adapter this queue belongs to */ struct rte_mempool *mp; /* mbuf pool to populate Rx ring */ @@ -198,4 +235,57 @@ void idpf_stop_queues(struct rte_eth_dev *dev); void idpf_set_rx_function(struct rte_eth_dev *dev); void idpf_set_tx_function(struct rte_eth_dev *dev); +#define IDPF_TIMESYNC_REG_WRAP_GUARD_BAND 10000 +/* Helper function to convert a 32b nanoseconds timestamp to 64b. */ +static inline uint64_t + +idpf_tstamp_convert_32b_64b(struct idpf_adapter *ad, uint32_t flag, + uint32_t in_timestamp) +{ +#ifdef RTE_ARCH_X86_64 + struct idpf_hw *hw = &ad->hw; + const uint64_t mask = 0xFFFFFFFF; + uint32_t hi, lo, lo2, delta; + uint64_t ns; + + if (flag != 0) { + IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_SHTIME_EN_M); + IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_EXEC_CMD_M | + PF_GLTSYN_CMD_SYNC_SHTIME_EN_M); + lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0); + hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0); + /* + * On typical system, the delta between lo and lo2 is ~1000ns, + * so 10000 seems a large-enough but not overly-big guard band. + */ + if (lo > (UINT32_MAX - IDPF_TIMESYNC_REG_WRAP_GUARD_BAND)) + lo2 = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0); + else + lo2 = lo; + + if (lo2 < lo) { + lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0); + hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0); + } + + ad->time_hw = ((uint64_t)hi << 32) | lo; + } + + delta = (in_timestamp - (uint32_t)(ad->time_hw & mask)); + if (delta > (mask / 2)) { + delta = ((uint32_t)(ad->time_hw & mask) - in_timestamp); + ns = ad->time_hw - delta; + } else { + ns = ad->time_hw + delta; + } + + return ns; +#else /* !RTE_ARCH_X86_64 */ + RTE_SET_USED(ad); + RTE_SET_USED(flag); + RTE_SET_USED(in_timestamp); + return 0; +#endif /* RTE_ARCH_X86_64 */ +} + #endif /* _IDPF_RXTX_H_ */