Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/52020/?format=api
{ "id": 52020, "url": "http://patchwork.dpdk.org/api/patches/52020/?format=api", "web_url": "http://patchwork.dpdk.org/project/dpdk/patch/20190401211757.26241-4-yskoh@mellanox.com/", "project": { "id": 1, "url": "http://patchwork.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<20190401211757.26241-4-yskoh@mellanox.com>", "list_archive_url": "https://inbox.dpdk.org/dev/20190401211757.26241-4-yskoh@mellanox.com", "date": "2019-04-01T21:17:54", "name": "[v3,3/6] net/mlx5: add control of excessive memory pinning by kernel", "commit_ref": null, "pull_url": null, "state": "accepted", "archived": true, "hash": "5bb3a78603ed03c22b65c8f8f75eff8a242cd70e", "submitter": { "id": 636, "url": "http://patchwork.dpdk.org/api/people/636/?format=api", "name": "Yongseok Koh", "email": "yskoh@mellanox.com" }, "delegate": { "id": 6624, "url": "http://patchwork.dpdk.org/api/users/6624/?format=api", "username": "shahafs", "first_name": "Shahaf", "last_name": "Shuler", "email": "shahafs@mellanox.com" }, "mbox": "http://patchwork.dpdk.org/project/dpdk/patch/20190401211757.26241-4-yskoh@mellanox.com/mbox/", "series": [ { "id": 4036, "url": "http://patchwork.dpdk.org/api/series/4036/?format=api", "web_url": "http://patchwork.dpdk.org/project/dpdk/list/?series=4036", "date": "2019-04-01T21:17:52", "name": "net/mlx: enable secondary process to register DMA memory", "version": 3, "mbox": "http://patchwork.dpdk.org/series/4036/mbox/" } ], "comments": "http://patchwork.dpdk.org/api/patches/52020/comments/", "check": "fail", "checks": "http://patchwork.dpdk.org/api/patches/52020/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@dpdk.org", "Delivered-To": "patchwork@dpdk.org", "Received": [ "from [92.243.14.124] (localhost [127.0.0.1])\n\tby dpdk.org (Postfix) with ESMTP id 107305B40;\n\tMon, 1 Apr 2019 23:18:12 +0200 (CEST)", "from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129])\n\tby dpdk.org (Postfix) with ESMTP id 0D2DA5B16\n\tfor <dev@dpdk.org>; Mon, 1 Apr 2019 23:18:07 +0200 (CEST)", "from Internal Mail-Server by MTLPINE1 (envelope-from\n\tyskoh@mellanox.com)\n\twith ESMTPS (AES256-SHA encrypted); 2 Apr 2019 00:18:06 +0300", "from scfae-sc-2.mti.labs.mlnx (scfae-sc-2.mti.labs.mlnx\n\t[10.101.0.96])\n\tby labmailer.mlnx (8.13.8/8.13.8) with ESMTP id x31LHwji003672;\n\tTue, 2 Apr 2019 00:18:05 +0300" ], "From": "Yongseok Koh <yskoh@mellanox.com>", "To": "shahafs@mellanox.com", "Cc": "dev@dpdk.org", "Date": "Mon, 1 Apr 2019 14:17:54 -0700", "Message-Id": "<20190401211757.26241-4-yskoh@mellanox.com>", "X-Mailer": "git-send-email 2.11.0", "In-Reply-To": "<20190401211757.26241-1-yskoh@mellanox.com>", "References": "<20190307074151.18815-1-yskoh@mellanox.com>\n\t<20190401211757.26241-1-yskoh@mellanox.com>", "Subject": "[dpdk-dev] [PATCH v3 3/6] net/mlx5: add control of excessive memory\n\tpinning by kernel", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n\t<mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n\t<mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "content": "A new PMD parameter (mr_ext_memseg_en) is added to control extension of\nmemseg when creating a MR. It is enabled by default.\n\nIf enabled, mlx5_mr_create() tries to maximize the range of MR\nregistration so that the LKey lookup tables on datapath become smaller and\nget the best performance. However, it may worsen memory utilization\nbecause registered memory is pinned by kernel driver. Even if a page in the\nextended chunk is freed, that doesn't become reusable until the entire\nmemory is freed and the MR is destroyed.\n\nTo make freed pages available immediately, this parameter has to be turned\noff but it could drop performance.\n\nSigned-off-by: Yongseok Koh <yskoh@mellanox.com>\nAcked-by: Shahaf Shuler <shahafs@mellanox.com>\n---\n doc/guides/nics/mlx5.rst | 11 +++++++++++\n drivers/net/mlx5/mlx5.c | 7 +++++++\n drivers/net/mlx5/mlx5.h | 2 ++\n drivers/net/mlx5/mlx5_mr.c | 21 ++++++++++++++++-----\n 4 files changed, 36 insertions(+), 5 deletions(-)", "diff": "diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst\nindex fa9bf73da7..e5e3d9061e 100644\n--- a/doc/guides/nics/mlx5.rst\n+++ b/doc/guides/nics/mlx5.rst\n@@ -485,6 +485,17 @@ Run-time configuration\n \n Disabled by default.\n \n+- ``mr_ext_memseg_en`` parameter [int]\n+\n+ A nonzero value enables extending memseg when registering DMA memory. If\n+ enabled, the number of entries in MR (Memory Region) lookup table on datapath\n+ is minimized and it benefits performance. On the other hand, it worsens memory\n+ utilization because registered memory is pinned by kernel driver. Even if a\n+ page in the extended chunk is freed, that doesn't become reusable until the\n+ entire memory is freed.\n+\n+ Enabled by default.\n+\n - ``representor`` parameter [list]\n \n This parameter can be used to instantiate DPDK Ethernet devices from\ndiff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c\nindex 2b7a6d121f..40445056f5 100644\n--- a/drivers/net/mlx5/mlx5.c\n+++ b/drivers/net/mlx5/mlx5.c\n@@ -107,6 +107,9 @@\n /* Activate Netlink support in VF mode. */\n #define MLX5_VF_NL_EN \"vf_nl_en\"\n \n+/* Enable extending memsegs when creating a MR. */\n+#define MLX5_MR_EXT_MEMSEG_EN \"mr_ext_memseg_en\"\n+\n /* Select port representors to instantiate. */\n #define MLX5_REPRESENTOR \"representor\"\n \n@@ -732,6 +735,8 @@ mlx5_args_check(const char *key, const char *val, void *opaque)\n \t\tconfig->vf_nl_en = !!tmp;\n \t} else if (strcmp(MLX5_DV_FLOW_EN, key) == 0) {\n \t\tconfig->dv_flow_en = !!tmp;\n+\t} else if (strcmp(MLX5_MR_EXT_MEMSEG_EN, key) == 0) {\n+\t\tconfig->mr_ext_memseg_en = !!tmp;\n \t} else {\n \t\tDRV_LOG(WARNING, \"%s: unknown parameter\", key);\n \t\trte_errno = EINVAL;\n@@ -773,6 +778,7 @@ mlx5_args(struct mlx5_dev_config *config, struct rte_devargs *devargs)\n \t\tMLX5_L3_VXLAN_EN,\n \t\tMLX5_VF_NL_EN,\n \t\tMLX5_DV_FLOW_EN,\n+\t\tMLX5_MR_EXT_MEMSEG_EN,\n \t\tMLX5_REPRESENTOR,\n \t\tNULL,\n \t};\n@@ -1853,6 +1859,7 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,\n \t\t.txqs_vec = MLX5_ARG_UNSET,\n \t\t.inline_max_packet_sz = MLX5_ARG_UNSET,\n \t\t.vf_nl_en = 1,\n+\t\t.mr_ext_memseg_en = 1,\n \t\t.mprq = {\n \t\t\t.enabled = 0, /* Disabled by default. */\n \t\t\t.stride_num_n = MLX5_MPRQ_STRIDE_NUM_N,\ndiff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h\nindex 12692505c3..b3445f198f 100644\n--- a/drivers/net/mlx5/mlx5.h\n+++ b/drivers/net/mlx5/mlx5.h\n@@ -167,6 +167,8 @@ struct mlx5_dev_config {\n \tunsigned int tx_vec_en:1; /* Tx vector is enabled. */\n \tunsigned int rx_vec_en:1; /* Rx vector is enabled. */\n \tunsigned int mpw_hdr_dseg:1; /* Enable DSEGs in the title WQEBB. */\n+\tunsigned int mr_ext_memseg_en:1;\n+\t/* Whether memseg should be extended for MR creation. */\n \tunsigned int l3_vxlan_en:1; /* Enable L3 VXLAN flow creation. */\n \tunsigned int vf_nl_en:1; /* Enable Netlink requests in VF mode. */\n \tunsigned int dv_flow_en:1; /* Enable DV flow. */\ndiff --git a/drivers/net/mlx5/mlx5_mr.c b/drivers/net/mlx5/mlx5_mr.c\nindex e7f55be6e1..78d829722e 100644\n--- a/drivers/net/mlx5/mlx5_mr.c\n+++ b/drivers/net/mlx5/mlx5_mr.c\n@@ -535,6 +535,7 @@ mlx5_mr_create(struct rte_eth_dev *dev, struct mlx5_mr_cache *entry,\n \t uintptr_t addr)\n {\n \tstruct mlx5_priv *priv = dev->data->dev_private;\n+\tstruct mlx5_dev_config *config = &priv->config;\n \tstruct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config;\n \tconst struct rte_memseg_list *msl;\n \tconst struct rte_memseg *ms;\n@@ -570,14 +571,24 @@ mlx5_mr_create(struct rte_eth_dev *dev, struct mlx5_mr_cache *entry,\n \t */\n \tmlx5_mr_garbage_collect(dev);\n \t/*\n-\t * Find out a contiguous virtual address chunk in use, to which the\n-\t * given address belongs, in order to register maximum range. In the\n-\t * best case where mempools are not dynamically recreated and\n+\t * If enabled, find out a contiguous virtual address chunk in use, to\n+\t * which the given address belongs, in order to register maximum range.\n+\t * In the best case where mempools are not dynamically recreated and\n \t * '--socket-mem' is specified as an EAL option, it is very likely to\n \t * have only one MR(LKey) per a socket and per a hugepage-size even\n-\t * though the system memory is highly fragmented.\n+\t * though the system memory is highly fragmented. As the whole memory\n+\t * chunk will be pinned by kernel, it can't be reused unless entire\n+\t * chunk is freed from EAL.\n+\t *\n+\t * If disabled, just register one memseg (page). Then, memory\n+\t * consumption will be minimized but it may drop performance if there\n+\t * are many MRs to lookup on the datapath.\n \t */\n-\tif (!rte_memseg_contig_walk(mr_find_contig_memsegs_cb, &data)) {\n+\tif (!config->mr_ext_memseg_en) {\n+\t\tdata.msl = rte_mem_virt2memseg_list((void *)addr);\n+\t\tdata.start = RTE_ALIGN_FLOOR(addr, data.msl->page_sz);\n+\t\tdata.end = data.start + data.msl->page_sz;\n+\t} else if (!rte_memseg_contig_walk(mr_find_contig_memsegs_cb, &data)) {\n \t\tDRV_LOG(WARNING,\n \t\t\t\"port %u unable to find virtually contiguous\"\n \t\t\t\" chunk for address (%p).\"\n", "prefixes": [ "v3", "3/6" ] }