From patchwork Fri Sep 24 14:33:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Conor Walsh X-Patchwork-Id: 99615 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A3802A0548; Fri, 24 Sep 2021 16:34:26 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 89C3E41349; Fri, 24 Sep 2021 16:33:54 +0200 (CEST) Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by mails.dpdk.org (Postfix) with ESMTP id 70AC041340 for ; Fri, 24 Sep 2021 16:33:51 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10116"; a="309640165" X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="309640165" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Sep 2021 07:33:51 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="703871485" Received: from silpixa00401160.ir.intel.com ([10.55.129.96]) by fmsmga006.fm.intel.com with ESMTP; 24 Sep 2021 07:33:49 -0700 From: Conor Walsh To: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, kevin.laatz@intel.com Cc: dev@dpdk.org, Conor Walsh Date: Fri, 24 Sep 2021 14:33:29 +0000 Message-Id: <20210924143335.1092300-7-conor.walsh@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210924143335.1092300-1-conor.walsh@intel.com> References: <20210827172550.1522362-1-conor.walsh@intel.com> <20210924143335.1092300-1-conor.walsh@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v5 06/12] dma/ioat: add data path job submission functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add data path functions for enqueuing and submitting operations to IOAT devices. Signed-off-by: Conor Walsh Reviewed-by: Kevin Laatz Reviewed-by: Chengwen Feng --- doc/guides/dmadevs/ioat.rst | 9 ++++ drivers/dma/ioat/ioat_dmadev.c | 92 ++++++++++++++++++++++++++++++++++ 2 files changed, 101 insertions(+) diff --git a/doc/guides/dmadevs/ioat.rst b/doc/guides/dmadevs/ioat.rst index d93d28023f..ec8ce5a8e5 100644 --- a/doc/guides/dmadevs/ioat.rst +++ b/doc/guides/dmadevs/ioat.rst @@ -89,3 +89,12 @@ IOAT configuration requirements: Once configured, the device can then be made ready for use by calling the ``rte_dma_start()`` API. + +Performing Data Copies +~~~~~~~~~~~~~~~~~~~~~~~ + +Refer to the :ref:`Enqueue / Dequeue APIs ` section of the dmadev library +documentation for details on operation enqueue and submission API usage. + +It is expected that, for efficiency reasons, a burst of operations will be enqueued to the +device via multiple enqueue calls between calls to the ``rte_dma_submit()`` function. diff --git a/drivers/dma/ioat/ioat_dmadev.c b/drivers/dma/ioat/ioat_dmadev.c index 96bf55135f..0e92c80fb0 100644 --- a/drivers/dma/ioat/ioat_dmadev.c +++ b/drivers/dma/ioat/ioat_dmadev.c @@ -5,6 +5,7 @@ #include #include #include +#include #include "ioat_internal.h" @@ -17,6 +18,12 @@ RTE_LOG_REGISTER_DEFAULT(ioat_pmd_logtype, INFO); #define IOAT_PMD_NAME dmadev_ioat #define IOAT_PMD_NAME_STR RTE_STR(IOAT_PMD_NAME) +/* IOAT operations. */ +enum rte_ioat_ops { + ioat_op_copy = 0, /* Standard DMA Operation */ + ioat_op_fill /* Block Fill */ +}; + /* Configure a device. */ static int ioat_dev_configure(struct rte_dma_dev *dev __rte_unused, const struct rte_dma_conf *dev_conf, @@ -208,6 +215,87 @@ ioat_dev_close(struct rte_dma_dev *dev) return 0; } +/* Trigger hardware to begin performing enqueued operations. */ +static inline void +__submit(struct ioat_dmadev *ioat) +{ + *ioat->doorbell = ioat->next_write - ioat->offset; + + ioat->last_write = ioat->next_write; +} + +/* External submit function wrapper. */ +static int +ioat_submit(struct rte_dma_dev *dev, uint16_t qid __rte_unused) +{ + struct ioat_dmadev *ioat = (struct ioat_dmadev *)dev->dev_private; + + __submit(ioat); + + return 0; +} + +/* Write descriptor for enqueue. */ +static inline int +__write_desc(struct rte_dma_dev *dev, uint32_t op, uint64_t src, phys_addr_t dst, + unsigned int length, uint64_t flags) +{ + struct ioat_dmadev *ioat = dev->dev_private; + uint16_t ret; + const unsigned short mask = ioat->qcfg.nb_desc - 1; + const unsigned short read = ioat->next_read; + unsigned short write = ioat->next_write; + const unsigned short space = mask + read - write; + struct ioat_dma_hw_desc *desc; + + if (space == 0) + return -ENOSPC; + + ioat->next_write = write + 1; + write &= mask; + + desc = &ioat->desc_ring[write]; + desc->size = length; + desc->u.control_raw = (uint32_t)((op << IOAT_CMD_OP_SHIFT) | + (1 << IOAT_COMP_UPDATE_SHIFT)); + + /* In IOAT the fence ensures that all operations including the current one + * are completed before moving on, DMAdev assumes that the fence ensures + * all operations before the current one are completed before starting + * the current one, so in IOAT we set the fence for the previous descriptor. + */ + if (flags & RTE_DMA_OP_FLAG_FENCE) + ioat->desc_ring[(write - 1) & mask].u.control.fence = 1; + + desc->src_addr = src; + desc->dest_addr = dst; + + rte_prefetch0(&ioat->desc_ring[ioat->next_write & mask]); + + ret = (uint16_t)(ioat->next_write - 1); + + if (flags & RTE_DMA_OP_FLAG_SUBMIT) + __submit(ioat); + + return ret; +} + +/* Enqueue a fill operation onto the ioat device. */ +static int +ioat_enqueue_fill(struct rte_dma_dev *dev, uint16_t qid __rte_unused, uint64_t pattern, + rte_iova_t dst, unsigned int length, uint64_t flags) +{ + return __write_desc(dev, ioat_op_fill, pattern, dst, length, flags); +} + +/* Enqueue a copy operation onto the ioat device. */ +static int +ioat_enqueue_copy(struct rte_dma_dev *dev, uint16_t qid __rte_unused, rte_iova_t src, + rte_iova_t dst, unsigned int length, uint64_t flags) +{ + return __write_desc(dev, ioat_op_copy, src, dst, length, flags); +} + /* Dump DMA device info. */ static int ioat_dev_dump(const struct rte_dma_dev *dev, FILE *f) @@ -303,6 +391,10 @@ ioat_dmadev_create(const char *name, struct rte_pci_device *dev) dmadev->dev_ops = &ioat_dmadev_ops; + dmadev->copy = ioat_enqueue_copy; + dmadev->fill = ioat_enqueue_fill; + dmadev->submit = ioat_submit; + ioat = dmadev->data->dev_private; ioat->regs = dev->mem_resource[0].addr; ioat->doorbell = &ioat->regs->dmacount;