From patchwork Fri Jul 5 13:10:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anatoly Burakov X-Patchwork-Id: 56156 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1DFE21BEB2; Fri, 5 Jul 2019 15:10:46 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id 163341D9E for ; Fri, 5 Jul 2019 15:10:43 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 05 Jul 2019 06:10:43 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.63,455,1557212400"; d="scan'208";a="187862973" Received: from silpixa00399498.ir.intel.com (HELO silpixa00399498.ger.corp.intel.com) ([10.237.223.125]) by fmsmga004.fm.intel.com with ESMTP; 05 Jul 2019 06:10:36 -0700 From: Anatoly Burakov To: dev@dpdk.org Cc: Hemant Agrawal , Shreyansh Jain , Matan Azrad , Shahaf Shuler , Yongseok Koh , Maxime Coquelin , Tiwei Bie , Zhihong Wang , Bruce Richardson , thomas@monjalon.net, david.marchand@redhat.com, stephen@networkplumber.org Date: Fri, 5 Jul 2019 14:10:27 +0100 Message-Id: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: Subject: [dpdk-dev] [PATCH v4 1/8] eal: add API to lock/unlock memory hotplug X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Currently, the memory hotplug is locked automatically by all memory-related _walk() functions, but sometimes locking the memory subsystem outside of them is needed. There is no public API to do that, so it creates a dependency on shared memory config to be public. Fix this by introducing a new API to lock/unlock the memory hotplug subsystem. Create a new common file for all things mem config, and a new API namespace rte_mcfg_*, and search-and-replace all usages of the locks with the new API. Signed-off-by: Anatoly Burakov Acked-by: Stephen Hemminger --- drivers/bus/fslmc/fslmc_vfio.c | 8 ++-- drivers/net/mlx4/mlx4_mr.c | 11 +++-- drivers/net/mlx5/mlx5_mr.c | 11 +++-- .../net/virtio/virtio_user/virtio_user_dev.c | 7 ++- lib/librte_eal/common/eal_common_mcfg.c | 34 +++++++++++++++ lib/librte_eal/common/eal_common_memory.c | 43 ++++++++----------- .../common/include/rte_eal_memconfig.h | 24 +++++++++++ lib/librte_eal/common/malloc_heap.c | 14 +++--- lib/librte_eal/common/meson.build | 1 + lib/librte_eal/common/rte_malloc.c | 32 ++++++-------- lib/librte_eal/freebsd/eal/Makefile | 1 + lib/librte_eal/linux/eal/Makefile | 1 + lib/librte_eal/linux/eal/eal_vfio.c | 16 +++---- lib/librte_eal/rte_eal_version.map | 4 ++ 14 files changed, 125 insertions(+), 82 deletions(-) create mode 100644 lib/librte_eal/common/eal_common_mcfg.c diff --git a/drivers/bus/fslmc/fslmc_vfio.c b/drivers/bus/fslmc/fslmc_vfio.c index 1aae56fa9..44e4fa6e2 100644 --- a/drivers/bus/fslmc/fslmc_vfio.c +++ b/drivers/bus/fslmc/fslmc_vfio.c @@ -347,14 +347,12 @@ fslmc_dmamap_seg(const struct rte_memseg_list *msl __rte_unused, int rte_fslmc_vfio_dmamap(void) { int i = 0, ret; - struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; - rte_rwlock_t *mem_lock = &mcfg->memory_hotplug_lock; /* Lock before parsing and registering callback to memory subsystem */ - rte_rwlock_read_lock(mem_lock); + rte_mcfg_mem_read_lock(); if (rte_memseg_walk(fslmc_dmamap_seg, &i) < 0) { - rte_rwlock_read_unlock(mem_lock); + rte_mcfg_mem_read_unlock(); return -1; } @@ -378,7 +376,7 @@ int rte_fslmc_vfio_dmamap(void) /* Existing segments have been mapped and memory callback for hotplug * has been installed. */ - rte_rwlock_read_unlock(mem_lock); + rte_mcfg_mem_read_unlock(); return 0; } diff --git a/drivers/net/mlx4/mlx4_mr.c b/drivers/net/mlx4/mlx4_mr.c index 48d458ad4..80827ce75 100644 --- a/drivers/net/mlx4/mlx4_mr.c +++ b/drivers/net/mlx4/mlx4_mr.c @@ -593,7 +593,6 @@ mlx4_mr_create_primary(struct rte_eth_dev *dev, struct mlx4_mr_cache *entry, uintptr_t addr) { struct mlx4_priv *priv = dev->data->dev_private; - struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; const struct rte_memseg_list *msl; const struct rte_memseg *ms; struct mlx4_mr *mr = NULL; @@ -696,7 +695,7 @@ mlx4_mr_create_primary(struct rte_eth_dev *dev, struct mlx4_mr_cache *entry, * just single page. If not, go on with the big chunk atomically from * here. */ - rte_rwlock_read_lock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_lock(); data_re = data; if (len > msl->page_sz && !rte_memseg_contig_walk(mr_find_contig_memsegs_cb, &data_re)) { @@ -714,7 +713,7 @@ mlx4_mr_create_primary(struct rte_eth_dev *dev, struct mlx4_mr_cache *entry, */ data.start = RTE_ALIGN_FLOOR(addr, msl->page_sz); data.end = data.start + msl->page_sz; - rte_rwlock_read_unlock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_unlock(); mr_free(mr); goto alloc_resources; } @@ -734,7 +733,7 @@ mlx4_mr_create_primary(struct rte_eth_dev *dev, struct mlx4_mr_cache *entry, DEBUG("port %u found MR for %p on final lookup, abort", dev->data->port_id, (void *)addr); rte_rwlock_write_unlock(&priv->mr.rwlock); - rte_rwlock_read_unlock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_unlock(); /* * Must be unlocked before calling rte_free() because * mlx4_mr_mem_event_free_cb() can be called inside. @@ -802,12 +801,12 @@ mlx4_mr_create_primary(struct rte_eth_dev *dev, struct mlx4_mr_cache *entry, /* Lookup can't fail. */ assert(entry->lkey != UINT32_MAX); rte_rwlock_write_unlock(&priv->mr.rwlock); - rte_rwlock_read_unlock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_unlock(); return entry->lkey; err_mrlock: rte_rwlock_write_unlock(&priv->mr.rwlock); err_memlock: - rte_rwlock_read_unlock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_unlock(); err_nolock: /* * In case of error, as this can be called in a datapath, a warning diff --git a/drivers/net/mlx5/mlx5_mr.c b/drivers/net/mlx5/mlx5_mr.c index 66e8e874e..872d0591e 100644 --- a/drivers/net/mlx5/mlx5_mr.c +++ b/drivers/net/mlx5/mlx5_mr.c @@ -580,7 +580,6 @@ mlx5_mr_create_primary(struct rte_eth_dev *dev, struct mlx5_mr_cache *entry, struct mlx5_priv *priv = dev->data->dev_private; struct mlx5_ibv_shared *sh = priv->sh; struct mlx5_dev_config *config = &priv->config; - struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; const struct rte_memseg_list *msl; const struct rte_memseg *ms; struct mlx5_mr *mr = NULL; @@ -684,7 +683,7 @@ mlx5_mr_create_primary(struct rte_eth_dev *dev, struct mlx5_mr_cache *entry, * just single page. If not, go on with the big chunk atomically from * here. */ - rte_rwlock_read_lock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_lock(); data_re = data; if (len > msl->page_sz && !rte_memseg_contig_walk(mr_find_contig_memsegs_cb, &data_re)) { @@ -702,7 +701,7 @@ mlx5_mr_create_primary(struct rte_eth_dev *dev, struct mlx5_mr_cache *entry, */ data.start = RTE_ALIGN_FLOOR(addr, msl->page_sz); data.end = data.start + msl->page_sz; - rte_rwlock_read_unlock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_unlock(); mr_free(mr); goto alloc_resources; } @@ -722,7 +721,7 @@ mlx5_mr_create_primary(struct rte_eth_dev *dev, struct mlx5_mr_cache *entry, DEBUG("port %u found MR for %p on final lookup, abort", dev->data->port_id, (void *)addr); rte_rwlock_write_unlock(&sh->mr.rwlock); - rte_rwlock_read_unlock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_unlock(); /* * Must be unlocked before calling rte_free() because * mlx5_mr_mem_event_free_cb() can be called inside. @@ -790,12 +789,12 @@ mlx5_mr_create_primary(struct rte_eth_dev *dev, struct mlx5_mr_cache *entry, /* Lookup can't fail. */ assert(entry->lkey != UINT32_MAX); rte_rwlock_write_unlock(&sh->mr.rwlock); - rte_rwlock_read_unlock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_unlock(); return entry->lkey; err_mrlock: rte_rwlock_write_unlock(&sh->mr.rwlock); err_memlock: - rte_rwlock_read_unlock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_unlock(); err_nolock: /* * In case of error, as this can be called in a datapath, a warning diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c index e743695e4..c3ab9a21d 100644 --- a/drivers/net/virtio/virtio_user/virtio_user_dev.c +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c @@ -125,7 +125,6 @@ is_vhost_user_by_type(const char *path) int virtio_user_start_device(struct virtio_user_dev *dev) { - struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; uint64_t features; int ret; @@ -142,7 +141,7 @@ virtio_user_start_device(struct virtio_user_dev *dev) * replaced when we get proper supports from the * memory subsystem in the future. */ - rte_rwlock_read_lock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_lock(); pthread_mutex_lock(&dev->mutex); if (is_vhost_user_by_type(dev->path) && dev->vhostfd < 0) @@ -180,12 +179,12 @@ virtio_user_start_device(struct virtio_user_dev *dev) dev->started = true; pthread_mutex_unlock(&dev->mutex); - rte_rwlock_read_unlock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_unlock(); return 0; error: pthread_mutex_unlock(&dev->mutex); - rte_rwlock_read_unlock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_unlock(); /* TODO: free resource here or caller to check */ return -1; } diff --git a/lib/librte_eal/common/eal_common_mcfg.c b/lib/librte_eal/common/eal_common_mcfg.c new file mode 100644 index 000000000..985d36cc2 --- /dev/null +++ b/lib/librte_eal/common/eal_common_mcfg.c @@ -0,0 +1,34 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2019 Intel Corporation + */ + +#include +#include + +void +rte_mcfg_mem_read_lock(void) +{ + struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; + rte_rwlock_read_lock(&mcfg->memory_hotplug_lock); +} + +void +rte_mcfg_mem_read_unlock(void) +{ + struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; + rte_rwlock_read_unlock(&mcfg->memory_hotplug_lock); +} + +void +rte_mcfg_mem_write_lock(void) +{ + struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; + rte_rwlock_write_lock(&mcfg->memory_hotplug_lock); +} + +void +rte_mcfg_mem_write_unlock(void) +{ + struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; + rte_rwlock_write_unlock(&mcfg->memory_hotplug_lock); +} diff --git a/lib/librte_eal/common/eal_common_memory.c b/lib/librte_eal/common/eal_common_memory.c index 858d56382..fe22b139b 100644 --- a/lib/librte_eal/common/eal_common_memory.c +++ b/lib/librte_eal/common/eal_common_memory.c @@ -596,13 +596,12 @@ rte_memseg_contig_walk_thread_unsafe(rte_memseg_contig_walk_t func, void *arg) int rte_memseg_contig_walk(rte_memseg_contig_walk_t func, void *arg) { - struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; int ret = 0; /* do not allow allocations/frees/init while we iterate */ - rte_rwlock_read_lock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_lock(); ret = rte_memseg_contig_walk_thread_unsafe(func, arg); - rte_rwlock_read_unlock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_unlock(); return ret; } @@ -638,13 +637,12 @@ rte_memseg_walk_thread_unsafe(rte_memseg_walk_t func, void *arg) int rte_memseg_walk(rte_memseg_walk_t func, void *arg) { - struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; int ret = 0; /* do not allow allocations/frees/init while we iterate */ - rte_rwlock_read_lock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_lock(); ret = rte_memseg_walk_thread_unsafe(func, arg); - rte_rwlock_read_unlock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_unlock(); return ret; } @@ -671,13 +669,12 @@ rte_memseg_list_walk_thread_unsafe(rte_memseg_list_walk_t func, void *arg) int rte_memseg_list_walk(rte_memseg_list_walk_t func, void *arg) { - struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; int ret = 0; /* do not allow allocations/frees/init while we iterate */ - rte_rwlock_read_lock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_lock(); ret = rte_memseg_list_walk_thread_unsafe(func, arg); - rte_rwlock_read_unlock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_unlock(); return ret; } @@ -727,12 +724,11 @@ rte_memseg_get_fd_thread_unsafe(const struct rte_memseg *ms) int rte_memseg_get_fd(const struct rte_memseg *ms) { - struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; int ret; - rte_rwlock_read_lock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_lock(); ret = rte_memseg_get_fd_thread_unsafe(ms); - rte_rwlock_read_unlock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_unlock(); return ret; } @@ -783,12 +779,11 @@ rte_memseg_get_fd_offset_thread_unsafe(const struct rte_memseg *ms, int rte_memseg_get_fd_offset(const struct rte_memseg *ms, size_t *offset) { - struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; int ret; - rte_rwlock_read_lock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_lock(); ret = rte_memseg_get_fd_offset_thread_unsafe(ms, offset); - rte_rwlock_read_unlock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_unlock(); return ret; } @@ -809,7 +804,7 @@ rte_extmem_register(void *va_addr, size_t len, rte_iova_t iova_addrs[], rte_errno = EINVAL; return -1; } - rte_rwlock_write_lock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_write_lock(); /* make sure the segment doesn't already exist */ if (malloc_heap_find_external_seg(va_addr, len) != NULL) { @@ -838,14 +833,13 @@ rte_extmem_register(void *va_addr, size_t len, rte_iova_t iova_addrs[], /* memseg list successfully created - increment next socket ID */ mcfg->next_socket_id++; unlock: - rte_rwlock_write_unlock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_write_unlock(); return ret; } int rte_extmem_unregister(void *va_addr, size_t len) { - struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; struct rte_memseg_list *msl; int ret = 0; @@ -853,7 +847,7 @@ rte_extmem_unregister(void *va_addr, size_t len) rte_errno = EINVAL; return -1; } - rte_rwlock_write_lock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_write_lock(); /* find our segment */ msl = malloc_heap_find_external_seg(va_addr, len); @@ -865,14 +859,13 @@ rte_extmem_unregister(void *va_addr, size_t len) ret = malloc_heap_destroy_external_seg(msl); unlock: - rte_rwlock_write_unlock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_write_unlock(); return ret; } static int sync_memory(void *va_addr, size_t len, bool attach) { - struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; struct rte_memseg_list *msl; int ret = 0; @@ -880,7 +873,7 @@ sync_memory(void *va_addr, size_t len, bool attach) rte_errno = EINVAL; return -1; } - rte_rwlock_write_lock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_write_lock(); /* find our segment */ msl = malloc_heap_find_external_seg(va_addr, len); @@ -895,7 +888,7 @@ sync_memory(void *va_addr, size_t len, bool attach) ret = rte_fbarray_detach(&msl->memseg_arr); unlock: - rte_rwlock_write_unlock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_write_unlock(); return ret; } @@ -923,7 +916,7 @@ rte_eal_memory_init(void) return -1; /* lock mem hotplug here, to prevent races while we init */ - rte_rwlock_read_lock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_lock(); if (rte_eal_memseg_init() < 0) goto fail; @@ -942,6 +935,6 @@ rte_eal_memory_init(void) return 0; fail: - rte_rwlock_read_unlock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_unlock(); return -1; } diff --git a/lib/librte_eal/common/include/rte_eal_memconfig.h b/lib/librte_eal/common/include/rte_eal_memconfig.h index 84aabe36c..a554518ef 100644 --- a/lib/librte_eal/common/include/rte_eal_memconfig.h +++ b/lib/librte_eal/common/include/rte_eal_memconfig.h @@ -100,6 +100,30 @@ rte_eal_mcfg_wait_complete(struct rte_mem_config* mcfg) rte_pause(); } +/** + * Lock the internal EAL shared memory configuration for shared access. + */ +void +rte_mcfg_mem_read_lock(void); + +/** + * Unlock the internal EAL shared memory configuration for shared access. + */ +void +rte_mcfg_mem_read_unlock(void); + +/** + * Lock the internal EAL shared memory configuration for exclusive access. + */ +void +rte_mcfg_mem_write_lock(void); + +/** + * Unlock the internal EAL shared memory configuration for exclusive access. + */ +void +rte_mcfg_mem_write_unlock(void); + #ifdef __cplusplus } #endif diff --git a/lib/librte_eal/common/malloc_heap.c b/lib/librte_eal/common/malloc_heap.c index f9235932e..f1d31de0d 100644 --- a/lib/librte_eal/common/malloc_heap.c +++ b/lib/librte_eal/common/malloc_heap.c @@ -485,10 +485,9 @@ try_expand_heap(struct malloc_heap *heap, uint64_t pg_sz, size_t elt_size, int socket, unsigned int flags, size_t align, size_t bound, bool contig) { - struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; int ret; - rte_rwlock_write_lock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_write_lock(); if (rte_eal_process_type() == RTE_PROC_PRIMARY) { ret = try_expand_heap_primary(heap, pg_sz, elt_size, socket, @@ -498,7 +497,7 @@ try_expand_heap(struct malloc_heap *heap, uint64_t pg_sz, size_t elt_size, flags, align, bound, contig); } - rte_rwlock_write_unlock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_write_unlock(); return ret; } @@ -821,7 +820,6 @@ malloc_heap_free_pages(void *aligned_start, size_t aligned_len) int malloc_heap_free(struct malloc_elem *elem) { - struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; struct malloc_heap *heap; void *start, *aligned_start, *end, *aligned_end; size_t len, aligned_len, page_sz; @@ -935,7 +933,7 @@ malloc_heap_free(struct malloc_elem *elem) /* now we can finally free us some pages */ - rte_rwlock_write_lock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_write_lock(); /* * we allow secondary processes to clear the heap of this allocated @@ -990,7 +988,7 @@ malloc_heap_free(struct malloc_elem *elem) RTE_LOG(DEBUG, EAL, "Heap on socket %d was shrunk by %zdMB\n", msl->socket_id, aligned_len >> 20ULL); - rte_rwlock_write_unlock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_write_unlock(); free_unlock: rte_spinlock_unlock(&(heap->lock)); return ret; @@ -1344,7 +1342,7 @@ rte_eal_malloc_heap_init(void) if (register_mp_requests()) { RTE_LOG(ERR, EAL, "Couldn't register malloc multiprocess actions\n"); - rte_rwlock_read_unlock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_unlock(); return -1; } @@ -1352,7 +1350,7 @@ rte_eal_malloc_heap_init(void) * even come before primary itself is fully initialized, and secondaries * do not need to initialize the heap. */ - rte_rwlock_read_unlock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_unlock(); /* secondary process does not need to initialize anything */ if (rte_eal_process_type() != RTE_PROC_PRIMARY) diff --git a/lib/librte_eal/common/meson.build b/lib/librte_eal/common/meson.build index bafd23207..58b433bc2 100644 --- a/lib/librte_eal/common/meson.build +++ b/lib/librte_eal/common/meson.build @@ -18,6 +18,7 @@ common_sources = files( 'eal_common_launch.c', 'eal_common_lcore.c', 'eal_common_log.c', + 'eal_common_mcfg.c', 'eal_common_memalloc.c', 'eal_common_memory.c', 'eal_common_memzone.c', diff --git a/lib/librte_eal/common/rte_malloc.c b/lib/librte_eal/common/rte_malloc.c index b119ebae3..2cad7beaa 100644 --- a/lib/librte_eal/common/rte_malloc.c +++ b/lib/librte_eal/common/rte_malloc.c @@ -223,7 +223,7 @@ rte_malloc_heap_get_socket(const char *name) rte_errno = EINVAL; return -1; } - rte_rwlock_read_lock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_lock(); for (idx = 0; idx < RTE_MAX_HEAPS; idx++) { struct malloc_heap *tmp = &mcfg->malloc_heaps[idx]; @@ -239,7 +239,7 @@ rte_malloc_heap_get_socket(const char *name) rte_errno = ENOENT; ret = -1; } - rte_rwlock_read_unlock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_unlock(); return ret; } @@ -254,7 +254,7 @@ rte_malloc_heap_socket_is_external(int socket_id) if (socket_id == SOCKET_ID_ANY) return 0; - rte_rwlock_read_lock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_lock(); for (idx = 0; idx < RTE_MAX_HEAPS; idx++) { struct malloc_heap *tmp = &mcfg->malloc_heaps[idx]; @@ -264,7 +264,7 @@ rte_malloc_heap_socket_is_external(int socket_id) break; } } - rte_rwlock_read_unlock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_unlock(); return ret; } @@ -352,7 +352,6 @@ int rte_malloc_heap_memory_add(const char *heap_name, void *va_addr, size_t len, rte_iova_t iova_addrs[], unsigned int n_pages, size_t page_sz) { - struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; struct malloc_heap *heap = NULL; struct rte_memseg_list *msl; unsigned int n; @@ -369,7 +368,7 @@ rte_malloc_heap_memory_add(const char *heap_name, void *va_addr, size_t len, rte_errno = EINVAL; return -1; } - rte_rwlock_write_lock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_write_lock(); /* find our heap */ heap = find_named_heap(heap_name); @@ -398,7 +397,7 @@ rte_malloc_heap_memory_add(const char *heap_name, void *va_addr, size_t len, rte_spinlock_unlock(&heap->lock); unlock: - rte_rwlock_write_unlock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_write_unlock(); return ret; } @@ -406,7 +405,6 @@ rte_malloc_heap_memory_add(const char *heap_name, void *va_addr, size_t len, int rte_malloc_heap_memory_remove(const char *heap_name, void *va_addr, size_t len) { - struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; struct malloc_heap *heap = NULL; struct rte_memseg_list *msl; int ret; @@ -418,7 +416,7 @@ rte_malloc_heap_memory_remove(const char *heap_name, void *va_addr, size_t len) rte_errno = EINVAL; return -1; } - rte_rwlock_write_lock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_write_lock(); /* find our heap */ heap = find_named_heap(heap_name); if (heap == NULL) { @@ -448,7 +446,7 @@ rte_malloc_heap_memory_remove(const char *heap_name, void *va_addr, size_t len) ret = malloc_heap_destroy_external_seg(msl); unlock: - rte_rwlock_write_unlock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_write_unlock(); return ret; } @@ -456,7 +454,6 @@ rte_malloc_heap_memory_remove(const char *heap_name, void *va_addr, size_t len) static int sync_memory(const char *heap_name, void *va_addr, size_t len, bool attach) { - struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; struct malloc_heap *heap = NULL; struct rte_memseg_list *msl; int ret; @@ -468,7 +465,7 @@ sync_memory(const char *heap_name, void *va_addr, size_t len, bool attach) rte_errno = EINVAL; return -1; } - rte_rwlock_read_lock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_lock(); /* find our heap */ heap = find_named_heap(heap_name); @@ -516,7 +513,7 @@ sync_memory(const char *heap_name, void *va_addr, size_t len, bool attach) } } unlock: - rte_rwlock_read_unlock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_read_unlock(); return ret; } @@ -549,7 +546,7 @@ rte_malloc_heap_create(const char *heap_name) /* check if there is space in the heap list, or if heap with this name * already exists. */ - rte_rwlock_write_lock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_write_lock(); for (i = 0; i < RTE_MAX_HEAPS; i++) { struct malloc_heap *tmp = &mcfg->malloc_heaps[i]; @@ -578,7 +575,7 @@ rte_malloc_heap_create(const char *heap_name) /* we're sure that we can create a new heap, so do it */ ret = malloc_heap_create(heap, heap_name); unlock: - rte_rwlock_write_unlock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_write_unlock(); return ret; } @@ -586,7 +583,6 @@ rte_malloc_heap_create(const char *heap_name) int rte_malloc_heap_destroy(const char *heap_name) { - struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; struct malloc_heap *heap = NULL; int ret; @@ -597,7 +593,7 @@ rte_malloc_heap_destroy(const char *heap_name) rte_errno = EINVAL; return -1; } - rte_rwlock_write_lock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_write_lock(); /* start from non-socket heaps */ heap = find_named_heap(heap_name); @@ -621,7 +617,7 @@ rte_malloc_heap_destroy(const char *heap_name) if (ret < 0) rte_spinlock_unlock(&heap->lock); unlock: - rte_rwlock_write_unlock(&mcfg->memory_hotplug_lock); + rte_mcfg_mem_write_unlock(); return ret; } diff --git a/lib/librte_eal/freebsd/eal/Makefile b/lib/librte_eal/freebsd/eal/Makefile index ca616c480..eb921275e 100644 --- a/lib/librte_eal/freebsd/eal/Makefile +++ b/lib/librte_eal/freebsd/eal/Makefile @@ -44,6 +44,7 @@ SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += eal_common_timer.c SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += eal_common_memzone.c SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += eal_common_log.c SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += eal_common_launch.c +SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += eal_common_mcfg.c SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += eal_common_memalloc.c SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += eal_common_memory.c SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += eal_common_tailqs.c diff --git a/lib/librte_eal/linux/eal/Makefile b/lib/librte_eal/linux/eal/Makefile index 729795a10..dfe8e9a49 100644 --- a/lib/librte_eal/linux/eal/Makefile +++ b/lib/librte_eal/linux/eal/Makefile @@ -52,6 +52,7 @@ SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += eal_common_timer.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += eal_common_memzone.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += eal_common_log.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += eal_common_launch.c +SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += eal_common_mcfg.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += eal_common_memalloc.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += eal_common_memory.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += eal_common_tailqs.c diff --git a/lib/librte_eal/linux/eal/eal_vfio.c b/lib/librte_eal/linux/eal/eal_vfio.c index feada64c0..96a03a657 100644 --- a/lib/librte_eal/linux/eal/eal_vfio.c +++ b/lib/librte_eal/linux/eal/eal_vfio.c @@ -635,8 +635,6 @@ int rte_vfio_setup_device(const char *sysfs_base, const char *dev_addr, int *vfio_dev_fd, struct vfio_device_info *device_info) { - struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; - rte_rwlock_t *mem_lock = &mcfg->memory_hotplug_lock; struct vfio_group_status group_status = { .argsz = sizeof(group_status) }; @@ -739,7 +737,7 @@ rte_vfio_setup_device(const char *sysfs_base, const char *dev_addr, /* lock memory hotplug before mapping and release it * after registering callback, to prevent races */ - rte_rwlock_read_lock(mem_lock); + rte_mcfg_mem_read_lock(); if (vfio_cfg == default_vfio_cfg) ret = t->dma_map_func(vfio_container_fd); else @@ -750,7 +748,7 @@ rte_vfio_setup_device(const char *sysfs_base, const char *dev_addr, dev_addr, errno, strerror(errno)); close(vfio_group_fd); rte_vfio_clear_group(vfio_group_fd); - rte_rwlock_read_unlock(mem_lock); + rte_mcfg_mem_read_unlock(); return -1; } @@ -781,7 +779,7 @@ rte_vfio_setup_device(const char *sysfs_base, const char *dev_addr, map->len); rte_spinlock_recursive_unlock( &user_mem_maps->lock); - rte_rwlock_read_unlock(mem_lock); + rte_mcfg_mem_read_unlock(); return -1; } } @@ -795,7 +793,7 @@ rte_vfio_setup_device(const char *sysfs_base, const char *dev_addr, else ret = 0; /* unlock memory hotplug */ - rte_rwlock_read_unlock(mem_lock); + rte_mcfg_mem_read_unlock(); if (ret && rte_errno != ENOTSUP) { RTE_LOG(ERR, EAL, "Could not install memory event callback for VFIO\n"); @@ -862,8 +860,6 @@ int rte_vfio_release_device(const char *sysfs_base, const char *dev_addr, int vfio_dev_fd) { - struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; - rte_rwlock_t *mem_lock = &mcfg->memory_hotplug_lock; struct vfio_group_status group_status = { .argsz = sizeof(group_status) }; @@ -876,7 +872,7 @@ rte_vfio_release_device(const char *sysfs_base, const char *dev_addr, * VFIO device, because this might be the last device and we might need * to unregister the callback. */ - rte_rwlock_read_lock(mem_lock); + rte_mcfg_mem_read_lock(); /* get group number */ ret = rte_vfio_get_group_num(sysfs_base, dev_addr, &iommu_group_num); @@ -947,7 +943,7 @@ rte_vfio_release_device(const char *sysfs_base, const char *dev_addr, ret = 0; out: - rte_rwlock_read_unlock(mem_lock); + rte_mcfg_mem_read_unlock(); return ret; } diff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map index a53a29a35..754060dc9 100644 --- a/lib/librte_eal/rte_eal_version.map +++ b/lib/librte_eal/rte_eal_version.map @@ -292,6 +292,10 @@ DPDK_19.08 { rte_lcore_index; rte_lcore_to_socket_id; + rte_mcfg_mem_read_lock; + rte_mcfg_mem_read_unlock; + rte_mcfg_mem_write_lock; + rte_mcfg_mem_write_unlock; rte_rand; rte_srand; From patchwork Fri Jul 5 13:10:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anatoly Burakov X-Patchwork-Id: 56157 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 79C2A1BEC1; Fri, 5 Jul 2019 15:10:49 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id E20951BEB8 for ; Fri, 5 Jul 2019 15:10:47 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 05 Jul 2019 06:10:47 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.63,455,1557212400"; d="scan'208";a="187862988" Received: from silpixa00399498.ir.intel.com (HELO silpixa00399498.ger.corp.intel.com) ([10.237.223.125]) by fmsmga004.fm.intel.com with ESMTP; 05 Jul 2019 06:10:43 -0700 From: Anatoly Burakov To: dev@dpdk.org Cc: Konstantin Ananyev , David Hunt , Byron Marohn , Pablo de Lara Guarch , Jerin Jacob , Yipeng Wang , Sameh Gobriel , Bruce Richardson , Ferruh Yigit , Vladimir Medvedkin , Olivier Matz , Andrew Rybchenko , Reshma Pattan , Gage Eads , thomas@monjalon.net, david.marchand@redhat.com, stephen@networkplumber.org Date: Fri, 5 Jul 2019 14:10:28 +0100 Message-Id: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: Subject: [dpdk-dev] [PATCH v4 2/8] eal: add EAL tailq list lock/unlock API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Currently, locking/unlocking the TAILQ list requires direct access to the shared memory config. Add an API to do the same, and search-and-replace all usages. Signed-off-by: Anatoly Burakov Acked-by: Stephen Hemminger --- lib/librte_acl/rte_acl.c | 18 ++++++------ lib/librte_distributor/rte_distributor.c | 4 +-- lib/librte_distributor/rte_distributor_v20.c | 4 +-- lib/librte_eal/common/eal_common_mcfg.c | 28 +++++++++++++++++++ lib/librte_eal/common/eal_common_tailqs.c | 4 +-- lib/librte_eal/common/include/rte_eal.h | 5 ---- .../common/include/rte_eal_memconfig.h | 24 ++++++++++++++++ lib/librte_eal/rte_eal_version.map | 4 +++ lib/librte_efd/rte_efd.c | 14 +++++----- lib/librte_eventdev/rte_event_ring.c | 16 +++++------ lib/librte_hash/rte_cuckoo_hash.c | 16 +++++------ lib/librte_hash/rte_fbk_hash.c | 14 +++++----- lib/librte_kni/rte_kni.c | 16 +++++------ lib/librte_lpm/rte_lpm.c | 24 ++++++++-------- lib/librte_lpm/rte_lpm6.c | 14 +++++----- lib/librte_member/rte_member.c | 16 +++++------ lib/librte_mempool/rte_mempool.c | 8 +++--- lib/librte_reorder/rte_reorder.c | 14 +++++----- lib/librte_ring/rte_ring.c | 18 ++++++------ lib/librte_stack/rte_stack.c | 18 ++++++------ 20 files changed, 165 insertions(+), 114 deletions(-) diff --git a/lib/librte_acl/rte_acl.c b/lib/librte_acl/rte_acl.c index fd5bd5e4e..7ff11d25f 100644 --- a/lib/librte_acl/rte_acl.c +++ b/lib/librte_acl/rte_acl.c @@ -156,13 +156,13 @@ rte_acl_find_existing(const char *name) acl_list = RTE_TAILQ_CAST(rte_acl_tailq.head, rte_acl_list); - rte_rwlock_read_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_read_lock(); TAILQ_FOREACH(te, acl_list, next) { ctx = (struct rte_acl_ctx *) te->data; if (strncmp(name, ctx->name, sizeof(ctx->name)) == 0) break; } - rte_rwlock_read_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_read_unlock(); if (te == NULL) { rte_errno = ENOENT; @@ -182,7 +182,7 @@ rte_acl_free(struct rte_acl_ctx *ctx) acl_list = RTE_TAILQ_CAST(rte_acl_tailq.head, rte_acl_list); - rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_lock(); /* find our tailq entry */ TAILQ_FOREACH(te, acl_list, next) { @@ -190,13 +190,13 @@ rte_acl_free(struct rte_acl_ctx *ctx) break; } if (te == NULL) { - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); return; } TAILQ_REMOVE(acl_list, te, next); - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); rte_free(ctx->mem); rte_free(ctx); @@ -226,7 +226,7 @@ rte_acl_create(const struct rte_acl_param *param) sz = sizeof(*ctx) + param->max_rule_num * param->rule_size; /* get EAL TAILQ lock. */ - rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_lock(); /* if we already have one with that name */ TAILQ_FOREACH(te, acl_list, next) { @@ -268,7 +268,7 @@ rte_acl_create(const struct rte_acl_param *param) } exit: - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); return ctx; } @@ -377,10 +377,10 @@ rte_acl_list_dump(void) acl_list = RTE_TAILQ_CAST(rte_acl_tailq.head, rte_acl_list); - rte_rwlock_read_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_read_lock(); TAILQ_FOREACH(te, acl_list, next) { ctx = (struct rte_acl_ctx *) te->data; rte_acl_dump(ctx); } - rte_rwlock_read_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_read_unlock(); } diff --git a/lib/librte_distributor/rte_distributor.c b/lib/librte_distributor/rte_distributor.c index 208abfb1d..9eb78b330 100644 --- a/lib/librte_distributor/rte_distributor.c +++ b/lib/librte_distributor/rte_distributor.c @@ -645,9 +645,9 @@ rte_distributor_create_v1705(const char *name, rte_dist_burst_list); - rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_lock(); TAILQ_INSERT_TAIL(dist_burst_list, d, next); - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); return d; } diff --git a/lib/librte_distributor/rte_distributor_v20.c b/lib/librte_distributor/rte_distributor_v20.c index cd5940713..1fc03b971 100644 --- a/lib/librte_distributor/rte_distributor_v20.c +++ b/lib/librte_distributor/rte_distributor_v20.c @@ -392,9 +392,9 @@ rte_distributor_create_v20(const char *name, distributor_list = RTE_TAILQ_CAST(rte_distributor_tailq.head, rte_distributor_list); - rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_lock(); TAILQ_INSERT_TAIL(distributor_list, d, next); - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); return d; } diff --git a/lib/librte_eal/common/eal_common_mcfg.c b/lib/librte_eal/common/eal_common_mcfg.c index 985d36cc2..05167e4dc 100644 --- a/lib/librte_eal/common/eal_common_mcfg.c +++ b/lib/librte_eal/common/eal_common_mcfg.c @@ -32,3 +32,31 @@ rte_mcfg_mem_write_unlock(void) struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; rte_rwlock_write_unlock(&mcfg->memory_hotplug_lock); } + +void +rte_mcfg_tailq_read_lock(void) +{ + struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; + rte_rwlock_read_lock(&mcfg->qlock); +} + +void +rte_mcfg_tailq_read_unlock(void) +{ + struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; + rte_rwlock_read_unlock(&mcfg->qlock); +} + +void +rte_mcfg_tailq_write_lock(void) +{ + struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; + rte_rwlock_write_lock(&mcfg->qlock); +} + +void +rte_mcfg_tailq_write_unlock(void) +{ + struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; + rte_rwlock_write_unlock(&mcfg->qlock); +} diff --git a/lib/librte_eal/common/eal_common_tailqs.c b/lib/librte_eal/common/eal_common_tailqs.c index ca2a7d32a..dc2c13caa 100644 --- a/lib/librte_eal/common/eal_common_tailqs.c +++ b/lib/librte_eal/common/eal_common_tailqs.c @@ -58,7 +58,7 @@ rte_dump_tailq(FILE *f) mcfg = rte_eal_get_configuration()->mem_config; - rte_rwlock_read_lock(&mcfg->qlock); + rte_mcfg_tailq_read_lock(); for (i = 0; i < RTE_MAX_TAILQ; i++) { const struct rte_tailq_head *tailq = &mcfg->tailq_head[i]; const struct rte_tailq_entry_head *head = &tailq->tailq_head; @@ -66,7 +66,7 @@ rte_dump_tailq(FILE *f) fprintf(f, "Tailq %u: qname:<%s>, tqh_first:%p, tqh_last:%p\n", i, tailq->name, head->tqh_first, head->tqh_last); } - rte_rwlock_read_unlock(&mcfg->qlock); + rte_mcfg_tailq_read_unlock(); } static struct rte_tailq_head * diff --git a/lib/librte_eal/common/include/rte_eal.h b/lib/librte_eal/common/include/rte_eal.h index bf0c74e07..28cbf2dde 100644 --- a/lib/librte_eal/common/include/rte_eal.h +++ b/lib/librte_eal/common/include/rte_eal.h @@ -446,11 +446,6 @@ typedef void (*rte_usage_hook_t)(const char * prgname); rte_usage_hook_t rte_set_application_usage_hook(rte_usage_hook_t usage_func); -/** - * macro to get the lock of tailq in mem_config - */ -#define RTE_EAL_TAILQ_RWLOCK (&rte_eal_get_configuration()->mem_config->qlock) - /** * macro to get the multiple lock of mempool shared by multiple-instance */ diff --git a/lib/librte_eal/common/include/rte_eal_memconfig.h b/lib/librte_eal/common/include/rte_eal_memconfig.h index a554518ef..240fa150b 100644 --- a/lib/librte_eal/common/include/rte_eal_memconfig.h +++ b/lib/librte_eal/common/include/rte_eal_memconfig.h @@ -124,6 +124,30 @@ rte_mcfg_mem_write_lock(void); void rte_mcfg_mem_write_unlock(void); +/** + * Lock the internal EAL TAILQ list for shared access. + */ +void +rte_mcfg_tailq_read_lock(void); + +/** + * Unlock the internal EAL TAILQ list for shared access. + */ +void +rte_mcfg_tailq_read_unlock(void); + +/** + * Lock the internal EAL TAILQ list for exclusive access. + */ +void +rte_mcfg_tailq_write_lock(void); + +/** + * Unlock the internal EAL TAILQ list for exclusive access. + */ +void +rte_mcfg_tailq_write_unlock(void); + #ifdef __cplusplus } #endif diff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map index 754060dc9..d78a3a8b9 100644 --- a/lib/librte_eal/rte_eal_version.map +++ b/lib/librte_eal/rte_eal_version.map @@ -296,6 +296,10 @@ DPDK_19.08 { rte_mcfg_mem_read_unlock; rte_mcfg_mem_write_lock; rte_mcfg_mem_write_unlock; + rte_mcfg_tailq_read_lock; + rte_mcfg_tailq_read_unlock; + rte_mcfg_tailq_write_lock; + rte_mcfg_tailq_write_unlock; rte_rand; rte_srand; diff --git a/lib/librte_efd/rte_efd.c b/lib/librte_efd/rte_efd.c index 14e493bc3..b808ce99f 100644 --- a/lib/librte_efd/rte_efd.c +++ b/lib/librte_efd/rte_efd.c @@ -532,7 +532,7 @@ rte_efd_create(const char *name, uint32_t max_num_rules, uint32_t key_len, num_chunks_shift = rte_bsf32(num_chunks); - rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_lock(); /* * Guarantee there's no existing: this is normally already checked @@ -685,7 +685,7 @@ rte_efd_create(const char *name, uint32_t max_num_rules, uint32_t key_len, te->data = (void *) table; TAILQ_INSERT_TAIL(efd_list, te, next); - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); snprintf(ring_name, sizeof(ring_name), "HT_%s", table->name); /* Create ring (Dummy slot index is not enqueued) */ @@ -705,7 +705,7 @@ rte_efd_create(const char *name, uint32_t max_num_rules, uint32_t key_len, return table; error_unlock_exit: - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); rte_efd_free(table); return NULL; @@ -720,7 +720,7 @@ rte_efd_find_existing(const char *name) efd_list = RTE_TAILQ_CAST(rte_efd_tailq.head, rte_efd_list); - rte_rwlock_read_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_read_lock(); TAILQ_FOREACH(te, efd_list, next) { @@ -728,7 +728,7 @@ rte_efd_find_existing(const char *name) if (strncmp(name, table->name, RTE_EFD_NAMESIZE) == 0) break; } - rte_rwlock_read_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_read_unlock(); if (te == NULL) { rte_errno = ENOENT; @@ -751,7 +751,7 @@ rte_efd_free(struct rte_efd_table *table) rte_free(table->chunks[socket_id]); efd_list = RTE_TAILQ_CAST(rte_efd_tailq.head, rte_efd_list); - rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_lock(); TAILQ_FOREACH_SAFE(te, efd_list, next, temp) { if (te->data == (void *) table) { @@ -761,7 +761,7 @@ rte_efd_free(struct rte_efd_table *table) } } - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); rte_ring_free(table->free_slots); rte_free(table->offline_chunks); rte_free(table->keys); diff --git a/lib/librte_eventdev/rte_event_ring.c b/lib/librte_eventdev/rte_event_ring.c index 16d02a953..50190de01 100644 --- a/lib/librte_eventdev/rte_event_ring.c +++ b/lib/librte_eventdev/rte_event_ring.c @@ -72,7 +72,7 @@ rte_event_ring_create(const char *name, unsigned int count, int socket_id, return NULL; } - rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_lock(); /* * reserve a memory zone for this ring. If we can't get rte_config or @@ -89,7 +89,7 @@ rte_event_ring_create(const char *name, unsigned int count, int socket_id, if (rte_memzone_free(mz) != 0) RTE_LOG(ERR, RING, "Cannot free memzone\n"); rte_free(te); - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); return NULL; } @@ -102,7 +102,7 @@ rte_event_ring_create(const char *name, unsigned int count, int socket_id, RTE_LOG(ERR, RING, "Cannot reserve memory\n"); rte_free(te); } - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); return r; } @@ -118,7 +118,7 @@ rte_event_ring_lookup(const char *name) ring_list = RTE_TAILQ_CAST(rte_event_ring_tailq.head, rte_event_ring_list); - rte_rwlock_read_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_read_lock(); TAILQ_FOREACH(te, ring_list, next) { r = (struct rte_event_ring *) te->data; @@ -126,7 +126,7 @@ rte_event_ring_lookup(const char *name) break; } - rte_rwlock_read_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_read_unlock(); if (te == NULL) { rte_errno = ENOENT; @@ -163,7 +163,7 @@ rte_event_ring_free(struct rte_event_ring *r) ring_list = RTE_TAILQ_CAST(rte_event_ring_tailq.head, rte_event_ring_list); - rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_lock(); /* find out tailq entry */ TAILQ_FOREACH(te, ring_list, next) { @@ -172,13 +172,13 @@ rte_event_ring_free(struct rte_event_ring *r) } if (te == NULL) { - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); return; } TAILQ_REMOVE(ring_list, te, next); - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); rte_free(te); } diff --git a/lib/librte_hash/rte_cuckoo_hash.c b/lib/librte_hash/rte_cuckoo_hash.c index 51198b477..47cd0da5b 100644 --- a/lib/librte_hash/rte_cuckoo_hash.c +++ b/lib/librte_hash/rte_cuckoo_hash.c @@ -52,13 +52,13 @@ rte_hash_find_existing(const char *name) hash_list = RTE_TAILQ_CAST(rte_hash_tailq.head, rte_hash_list); - rte_rwlock_read_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_read_lock(); TAILQ_FOREACH(te, hash_list, next) { h = (struct rte_hash *) te->data; if (strncmp(name, h->name, RTE_HASH_NAMESIZE) == 0) break; } - rte_rwlock_read_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_read_unlock(); if (te == NULL) { rte_errno = ENOENT; @@ -239,7 +239,7 @@ rte_hash_create(const struct rte_hash_parameters *params) snprintf(hash_name, sizeof(hash_name), "HT_%s", params->name); - rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_lock(); /* guarantee there's no existing: this is normally already checked * by ring creation above */ @@ -437,11 +437,11 @@ rte_hash_create(const struct rte_hash_parameters *params) te->data = (void *) h; TAILQ_INSERT_TAIL(hash_list, te, next); - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); return h; err_unlock: - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); err: rte_ring_free(r); rte_ring_free(r_ext); @@ -466,7 +466,7 @@ rte_hash_free(struct rte_hash *h) hash_list = RTE_TAILQ_CAST(rte_hash_tailq.head, rte_hash_list); - rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_lock(); /* find out tailq entry */ TAILQ_FOREACH(te, hash_list, next) { @@ -475,13 +475,13 @@ rte_hash_free(struct rte_hash *h) } if (te == NULL) { - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); return; } TAILQ_REMOVE(hash_list, te, next); - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); if (h->use_local_cache) rte_free(h->local_free_slots); diff --git a/lib/librte_hash/rte_fbk_hash.c b/lib/librte_hash/rte_fbk_hash.c index 9360f7981..db118c930 100644 --- a/lib/librte_hash/rte_fbk_hash.c +++ b/lib/librte_hash/rte_fbk_hash.c @@ -50,13 +50,13 @@ rte_fbk_hash_find_existing(const char *name) fbk_hash_list = RTE_TAILQ_CAST(rte_fbk_hash_tailq.head, rte_fbk_hash_list); - rte_rwlock_read_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_read_lock(); TAILQ_FOREACH(te, fbk_hash_list, next) { h = (struct rte_fbk_hash_table *) te->data; if (strncmp(name, h->name, RTE_FBK_HASH_NAMESIZE) == 0) break; } - rte_rwlock_read_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_read_unlock(); if (te == NULL) { rte_errno = ENOENT; return NULL; @@ -103,7 +103,7 @@ rte_fbk_hash_create(const struct rte_fbk_hash_params *params) snprintf(hash_name, sizeof(hash_name), "FBK_%s", params->name); - rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_lock(); /* guarantee there's no existing */ TAILQ_FOREACH(te, fbk_hash_list, next) { @@ -165,7 +165,7 @@ rte_fbk_hash_create(const struct rte_fbk_hash_params *params) TAILQ_INSERT_TAIL(fbk_hash_list, te, next); exit: - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); return ht; } @@ -188,7 +188,7 @@ rte_fbk_hash_free(struct rte_fbk_hash_table *ht) fbk_hash_list = RTE_TAILQ_CAST(rte_fbk_hash_tailq.head, rte_fbk_hash_list); - rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_lock(); /* find out tailq entry */ TAILQ_FOREACH(te, fbk_hash_list, next) { @@ -197,13 +197,13 @@ rte_fbk_hash_free(struct rte_fbk_hash_table *ht) } if (te == NULL) { - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); return; } TAILQ_REMOVE(fbk_hash_list, te, next); - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); rte_free(ht); rte_free(te); diff --git a/lib/librte_kni/rte_kni.c b/lib/librte_kni/rte_kni.c index 00104a35d..309e45918 100644 --- a/lib/librte_kni/rte_kni.c +++ b/lib/librte_kni/rte_kni.c @@ -214,7 +214,7 @@ rte_kni_alloc(struct rte_mempool *pktmbuf_pool, return NULL; } - rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_lock(); kni = __rte_kni_get(conf->name); if (kni != NULL) { @@ -304,7 +304,7 @@ rte_kni_alloc(struct rte_mempool *pktmbuf_pool, kni_list = RTE_TAILQ_CAST(rte_kni_tailq.head, rte_kni_list); TAILQ_INSERT_TAIL(kni_list, te, next); - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); /* Allocate mbufs and then put them into alloc_q */ kni_allocate_mbufs(kni); @@ -318,7 +318,7 @@ rte_kni_alloc(struct rte_mempool *pktmbuf_pool, kni_fail: rte_free(te); unlock: - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); return NULL; } @@ -381,7 +381,7 @@ rte_kni_release(struct rte_kni *kni) kni_list = RTE_TAILQ_CAST(rte_kni_tailq.head, rte_kni_list); - rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_lock(); TAILQ_FOREACH(te, kni_list, next) { if (te->data == kni) @@ -399,7 +399,7 @@ rte_kni_release(struct rte_kni *kni) TAILQ_REMOVE(kni_list, te, next); - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); /* mbufs in all fifo should be released, except request/response */ @@ -423,7 +423,7 @@ rte_kni_release(struct rte_kni *kni) return 0; unlock: - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); return -1; } @@ -640,11 +640,11 @@ rte_kni_get(const char *name) if (name == NULL || name[0] == '\0') return NULL; - rte_rwlock_read_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_read_lock(); kni = __rte_kni_get(name); - rte_rwlock_read_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_read_unlock(); return kni; } diff --git a/lib/librte_lpm/rte_lpm.c b/lib/librte_lpm/rte_lpm.c index 6b7b28a2e..b91f74216 100644 --- a/lib/librte_lpm/rte_lpm.c +++ b/lib/librte_lpm/rte_lpm.c @@ -97,13 +97,13 @@ rte_lpm_find_existing_v20(const char *name) lpm_list = RTE_TAILQ_CAST(rte_lpm_tailq.head, rte_lpm_list); - rte_rwlock_read_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_read_lock(); TAILQ_FOREACH(te, lpm_list, next) { l = te->data; if (strncmp(name, l->name, RTE_LPM_NAMESIZE) == 0) break; } - rte_rwlock_read_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_read_unlock(); if (te == NULL) { rte_errno = ENOENT; @@ -123,13 +123,13 @@ rte_lpm_find_existing_v1604(const char *name) lpm_list = RTE_TAILQ_CAST(rte_lpm_tailq.head, rte_lpm_list); - rte_rwlock_read_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_read_lock(); TAILQ_FOREACH(te, lpm_list, next) { l = te->data; if (strncmp(name, l->name, RTE_LPM_NAMESIZE) == 0) break; } - rte_rwlock_read_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_read_unlock(); if (te == NULL) { rte_errno = ENOENT; @@ -170,7 +170,7 @@ rte_lpm_create_v20(const char *name, int socket_id, int max_rules, /* Determine the amount of memory to allocate. */ mem_size = sizeof(*lpm) + (sizeof(lpm->rules_tbl[0]) * max_rules); - rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_lock(); /* guarantee there's no existing */ TAILQ_FOREACH(te, lpm_list, next) { @@ -212,7 +212,7 @@ rte_lpm_create_v20(const char *name, int socket_id, int max_rules, TAILQ_INSERT_TAIL(lpm_list, te, next); exit: - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); return lpm; } @@ -247,7 +247,7 @@ rte_lpm_create_v1604(const char *name, int socket_id, tbl8s_size = (sizeof(struct rte_lpm_tbl_entry) * RTE_LPM_TBL8_GROUP_NUM_ENTRIES * config->number_tbl8s); - rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_lock(); /* guarantee there's no existing */ TAILQ_FOREACH(te, lpm_list, next) { @@ -315,7 +315,7 @@ rte_lpm_create_v1604(const char *name, int socket_id, TAILQ_INSERT_TAIL(lpm_list, te, next); exit: - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); return lpm; } @@ -339,7 +339,7 @@ rte_lpm_free_v20(struct rte_lpm_v20 *lpm) lpm_list = RTE_TAILQ_CAST(rte_lpm_tailq.head, rte_lpm_list); - rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_lock(); /* find our tailq entry */ TAILQ_FOREACH(te, lpm_list, next) { @@ -349,7 +349,7 @@ rte_lpm_free_v20(struct rte_lpm_v20 *lpm) if (te != NULL) TAILQ_REMOVE(lpm_list, te, next); - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); rte_free(lpm); rte_free(te); @@ -368,7 +368,7 @@ rte_lpm_free_v1604(struct rte_lpm *lpm) lpm_list = RTE_TAILQ_CAST(rte_lpm_tailq.head, rte_lpm_list); - rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_lock(); /* find our tailq entry */ TAILQ_FOREACH(te, lpm_list, next) { @@ -378,7 +378,7 @@ rte_lpm_free_v1604(struct rte_lpm *lpm) if (te != NULL) TAILQ_REMOVE(lpm_list, te, next); - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); rte_free(lpm->tbl8); rte_free(lpm->rules_tbl); diff --git a/lib/librte_lpm/rte_lpm6.c b/lib/librte_lpm/rte_lpm6.c index a91803113..5af74a539 100644 --- a/lib/librte_lpm/rte_lpm6.c +++ b/lib/librte_lpm/rte_lpm6.c @@ -316,7 +316,7 @@ rte_lpm6_create(const char *name, int socket_id, mem_size = sizeof(*lpm) + (sizeof(lpm->tbl8[0]) * RTE_LPM6_TBL8_GROUP_NUM_ENTRIES * config->number_tbl8s); - rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_lock(); /* Guarantee there's no existing */ TAILQ_FOREACH(te, lpm_list, next) { @@ -363,11 +363,11 @@ rte_lpm6_create(const char *name, int socket_id, te->data = (void *) lpm; TAILQ_INSERT_TAIL(lpm_list, te, next); - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); return lpm; fail: - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); fail_wo_unlock: rte_free(tbl8_hdrs); @@ -389,13 +389,13 @@ rte_lpm6_find_existing(const char *name) lpm_list = RTE_TAILQ_CAST(rte_lpm6_tailq.head, rte_lpm6_list); - rte_rwlock_read_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_read_lock(); TAILQ_FOREACH(te, lpm_list, next) { l = (struct rte_lpm6 *) te->data; if (strncmp(name, l->name, RTE_LPM6_NAMESIZE) == 0) break; } - rte_rwlock_read_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_read_unlock(); if (te == NULL) { rte_errno = ENOENT; @@ -420,7 +420,7 @@ rte_lpm6_free(struct rte_lpm6 *lpm) lpm_list = RTE_TAILQ_CAST(rte_lpm6_tailq.head, rte_lpm6_list); - rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_lock(); /* find our tailq entry */ TAILQ_FOREACH(te, lpm_list, next) { @@ -431,7 +431,7 @@ rte_lpm6_free(struct rte_lpm6 *lpm) if (te != NULL) TAILQ_REMOVE(lpm_list, te, next); - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); rte_free(lpm->tbl8_hdrs); rte_free(lpm->tbl8_pool); diff --git a/lib/librte_member/rte_member.c b/lib/librte_member/rte_member.c index fd228f4ba..efed28dd9 100644 --- a/lib/librte_member/rte_member.c +++ b/lib/librte_member/rte_member.c @@ -32,13 +32,13 @@ rte_member_find_existing(const char *name) member_list = RTE_TAILQ_CAST(rte_member_tailq.head, rte_member_list); - rte_rwlock_read_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_read_lock(); TAILQ_FOREACH(te, member_list, next) { setsum = (struct rte_member_setsum *) te->data; if (strncmp(name, setsum->name, RTE_MEMBER_NAMESIZE) == 0) break; } - rte_rwlock_read_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_read_unlock(); if (te == NULL) { rte_errno = ENOENT; @@ -56,17 +56,17 @@ rte_member_free(struct rte_member_setsum *setsum) if (setsum == NULL) return; member_list = RTE_TAILQ_CAST(rte_member_tailq.head, rte_member_list); - rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_lock(); TAILQ_FOREACH(te, member_list, next) { if (te->data == (void *)setsum) break; } if (te == NULL) { - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); return; } TAILQ_REMOVE(member_list, te, next); - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); switch (setsum->type) { case RTE_MEMBER_TYPE_HT: @@ -105,7 +105,7 @@ rte_member_create(const struct rte_member_parameters *params) member_list = RTE_TAILQ_CAST(rte_member_tailq.head, rte_member_list); - rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_lock(); TAILQ_FOREACH(te, member_list, next) { setsum = te->data; @@ -159,13 +159,13 @@ rte_member_create(const struct rte_member_parameters *params) te->data = (void *)setsum; TAILQ_INSERT_TAIL(member_list, te, next); - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); return setsum; error_unlock_exit: rte_free(te); rte_free(setsum); - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); return NULL; } diff --git a/lib/librte_mempool/rte_mempool.c b/lib/librte_mempool/rte_mempool.c index 69bd2a65c..238287a01 100644 --- a/lib/librte_mempool/rte_mempool.c +++ b/lib/librte_mempool/rte_mempool.c @@ -711,7 +711,7 @@ rte_mempool_free(struct rte_mempool *mp) return; mempool_list = RTE_TAILQ_CAST(rte_mempool_tailq.head, rte_mempool_list); - rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_lock(); /* find out tailq entry */ TAILQ_FOREACH(te, mempool_list, next) { if (te->data == (void *)mp) @@ -722,7 +722,7 @@ rte_mempool_free(struct rte_mempool *mp) TAILQ_REMOVE(mempool_list, te, next); rte_free(te); } - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); rte_mempool_free_memchunks(mp); rte_mempool_ops_free(mp); @@ -898,9 +898,9 @@ rte_mempool_create_empty(const char *name, unsigned n, unsigned elt_size, te->data = mp; - rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_lock(); TAILQ_INSERT_TAIL(mempool_list, te, next); - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); rte_rwlock_write_unlock(RTE_EAL_MEMPOOL_RWLOCK); return mp; diff --git a/lib/librte_reorder/rte_reorder.c b/lib/librte_reorder/rte_reorder.c index 3a4a1b0a0..ae6e3f578 100644 --- a/lib/librte_reorder/rte_reorder.c +++ b/lib/librte_reorder/rte_reorder.c @@ -119,7 +119,7 @@ rte_reorder_create(const char *name, unsigned socket_id, unsigned int size) return NULL; } - rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_lock(); /* guarantee there's no existing */ TAILQ_FOREACH(te, reorder_list, next) { @@ -152,7 +152,7 @@ rte_reorder_create(const char *name, unsigned socket_id, unsigned int size) } exit: - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); return b; } @@ -193,7 +193,7 @@ rte_reorder_free(struct rte_reorder_buffer *b) reorder_list = RTE_TAILQ_CAST(rte_reorder_tailq.head, rte_reorder_list); - rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_lock(); /* find our tailq entry */ TAILQ_FOREACH(te, reorder_list, next) { @@ -201,13 +201,13 @@ rte_reorder_free(struct rte_reorder_buffer *b) break; } if (te == NULL) { - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); return; } TAILQ_REMOVE(reorder_list, te, next); - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); rte_reorder_free_mbufs(b); @@ -229,13 +229,13 @@ rte_reorder_find_existing(const char *name) reorder_list = RTE_TAILQ_CAST(rte_reorder_tailq.head, rte_reorder_list); - rte_rwlock_read_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_read_lock(); TAILQ_FOREACH(te, reorder_list, next) { b = (struct rte_reorder_buffer *) te->data; if (strncmp(name, b->name, RTE_REORDER_NAMESIZE) == 0) break; } - rte_rwlock_read_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_read_unlock(); if (te == NULL) { rte_errno = ENOENT; diff --git a/lib/librte_ring/rte_ring.c b/lib/librte_ring/rte_ring.c index b89ecf999..9ea26a631 100644 --- a/lib/librte_ring/rte_ring.c +++ b/lib/librte_ring/rte_ring.c @@ -147,7 +147,7 @@ rte_ring_create(const char *name, unsigned count, int socket_id, return NULL; } - rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_lock(); /* reserve a memory zone for this ring. If we can't get rte_config or * we are secondary process, the memzone_reserve function will set @@ -169,7 +169,7 @@ rte_ring_create(const char *name, unsigned count, int socket_id, RTE_LOG(ERR, RING, "Cannot reserve memory\n"); rte_free(te); } - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); return r; } @@ -200,7 +200,7 @@ rte_ring_free(struct rte_ring *r) } ring_list = RTE_TAILQ_CAST(rte_ring_tailq.head, rte_ring_list); - rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_lock(); /* find out tailq entry */ TAILQ_FOREACH(te, ring_list, next) { @@ -209,13 +209,13 @@ rte_ring_free(struct rte_ring *r) } if (te == NULL) { - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); return; } TAILQ_REMOVE(ring_list, te, next); - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); rte_free(te); } @@ -245,13 +245,13 @@ rte_ring_list_dump(FILE *f) ring_list = RTE_TAILQ_CAST(rte_ring_tailq.head, rte_ring_list); - rte_rwlock_read_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_read_lock(); TAILQ_FOREACH(te, ring_list, next) { rte_ring_dump(f, (struct rte_ring *) te->data); } - rte_rwlock_read_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_read_unlock(); } /* search a ring from its name */ @@ -264,7 +264,7 @@ rte_ring_lookup(const char *name) ring_list = RTE_TAILQ_CAST(rte_ring_tailq.head, rte_ring_list); - rte_rwlock_read_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_read_lock(); TAILQ_FOREACH(te, ring_list, next) { r = (struct rte_ring *) te->data; @@ -272,7 +272,7 @@ rte_ring_lookup(const char *name) break; } - rte_rwlock_read_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_read_unlock(); if (te == NULL) { rte_errno = ENOENT; diff --git a/lib/librte_stack/rte_stack.c b/lib/librte_stack/rte_stack.c index 60f63a65b..2cc7e8e16 100644 --- a/lib/librte_stack/rte_stack.c +++ b/lib/librte_stack/rte_stack.c @@ -84,13 +84,13 @@ rte_stack_create(const char *name, unsigned int count, int socket_id, return NULL; } - rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_lock(); mz = rte_memzone_reserve_aligned(mz_name, sz, socket_id, 0, __alignof__(*s)); if (mz == NULL) { STACK_LOG_ERR("Cannot reserve stack memzone!\n"); - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); rte_free(te); return NULL; } @@ -102,7 +102,7 @@ rte_stack_create(const char *name, unsigned int count, int socket_id, /* Store the name for later lookups */ ret = strlcpy(s->name, name, sizeof(s->name)); if (ret < 0 || ret >= (int)sizeof(s->name)) { - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); rte_errno = ENAMETOOLONG; rte_free(te); @@ -120,7 +120,7 @@ rte_stack_create(const char *name, unsigned int count, int socket_id, TAILQ_INSERT_TAIL(stack_list, te, next); - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); return s; } @@ -135,7 +135,7 @@ rte_stack_free(struct rte_stack *s) return; stack_list = RTE_TAILQ_CAST(rte_stack_tailq.head, rte_stack_list); - rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_lock(); /* find out tailq entry */ TAILQ_FOREACH(te, stack_list, next) { @@ -144,13 +144,13 @@ rte_stack_free(struct rte_stack *s) } if (te == NULL) { - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); return; } TAILQ_REMOVE(stack_list, te, next); - rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_write_unlock(); rte_free(te); @@ -171,7 +171,7 @@ rte_stack_lookup(const char *name) stack_list = RTE_TAILQ_CAST(rte_stack_tailq.head, rte_stack_list); - rte_rwlock_read_lock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_read_lock(); TAILQ_FOREACH(te, stack_list, next) { r = (struct rte_stack *) te->data; @@ -179,7 +179,7 @@ rte_stack_lookup(const char *name) break; } - rte_rwlock_read_unlock(RTE_EAL_TAILQ_RWLOCK); + rte_mcfg_tailq_read_unlock(); if (te == NULL) { rte_errno = ENOENT; From patchwork Fri Jul 5 13:10:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anatoly Burakov X-Patchwork-Id: 56158 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 3F66A1BECE; Fri, 5 Jul 2019 15:10:53 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id B404A1BEC3 for ; Fri, 5 Jul 2019 15:10:49 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 05 Jul 2019 06:10:49 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.63,455,1557212400"; d="scan'208";a="187862989" Received: from silpixa00399498.ir.intel.com (HELO silpixa00399498.ger.corp.intel.com) ([10.237.223.125]) by fmsmga004.fm.intel.com with ESMTP; 05 Jul 2019 06:10:47 -0700 From: Anatoly Burakov To: dev@dpdk.org Cc: Olivier Matz , Andrew Rybchenko , thomas@monjalon.net, david.marchand@redhat.com, stephen@networkplumber.org Date: Fri, 5 Jul 2019 14:10:29 +0100 Message-Id: <0c24b8e90a79ec20daca5dfac0130c962d8b1734.1562332112.git.anatoly.burakov@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: Subject: [dpdk-dev] [PATCH v4 3/8] eal: add new API to lock/unlock mempool list X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Currently, in order to lock access to the mempool list, a direct access to the shared memory structure is needed. Add an API to do the same, and search-and-replace all usages. Signed-off-by: Anatoly Burakov Acked-by: Stephen Hemminger --- lib/librte_eal/common/eal_common_mcfg.c | 28 +++++++++++++++++++ lib/librte_eal/common/include/rte_eal.h | 5 ---- .../common/include/rte_eal_memconfig.h | 24 ++++++++++++++++ lib/librte_eal/rte_eal_version.map | 4 +++ lib/librte_mempool/rte_mempool.c | 18 ++++++------ 5 files changed, 65 insertions(+), 14 deletions(-) diff --git a/lib/librte_eal/common/eal_common_mcfg.c b/lib/librte_eal/common/eal_common_mcfg.c index 05167e4dc..ba2bc37b7 100644 --- a/lib/librte_eal/common/eal_common_mcfg.c +++ b/lib/librte_eal/common/eal_common_mcfg.c @@ -60,3 +60,31 @@ rte_mcfg_tailq_write_unlock(void) struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; rte_rwlock_write_unlock(&mcfg->qlock); } + +void +rte_mcfg_mempool_read_lock(void) +{ + struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; + rte_rwlock_read_lock(&mcfg->mplock); +} + +void +rte_mcfg_mempool_read_unlock(void) +{ + struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; + rte_rwlock_read_unlock(&mcfg->mplock); +} + +void +rte_mcfg_mempool_write_lock(void) +{ + struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; + rte_rwlock_write_lock(&mcfg->mplock); +} + +void +rte_mcfg_mempool_write_unlock(void) +{ + struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; + rte_rwlock_write_unlock(&mcfg->mplock); +} diff --git a/lib/librte_eal/common/include/rte_eal.h b/lib/librte_eal/common/include/rte_eal.h index 28cbf2dde..34245b053 100644 --- a/lib/librte_eal/common/include/rte_eal.h +++ b/lib/librte_eal/common/include/rte_eal.h @@ -446,11 +446,6 @@ typedef void (*rte_usage_hook_t)(const char * prgname); rte_usage_hook_t rte_set_application_usage_hook(rte_usage_hook_t usage_func); -/** - * macro to get the multiple lock of mempool shared by multiple-instance - */ -#define RTE_EAL_MEMPOOL_RWLOCK (&rte_eal_get_configuration()->mem_config->mplock) - /** * Whether EAL is using huge pages (disabled by --no-huge option). * The no-huge mode cannot be used with UIO poll-mode drivers like igb/ixgbe. diff --git a/lib/librte_eal/common/include/rte_eal_memconfig.h b/lib/librte_eal/common/include/rte_eal_memconfig.h index 240fa150b..58dcbb96d 100644 --- a/lib/librte_eal/common/include/rte_eal_memconfig.h +++ b/lib/librte_eal/common/include/rte_eal_memconfig.h @@ -148,6 +148,30 @@ rte_mcfg_tailq_write_lock(void); void rte_mcfg_tailq_write_unlock(void); +/** + * Lock the internal EAL Mempool list for shared access. + */ +void +rte_mcfg_mempool_read_lock(void); + +/** + * Unlock the internal EAL Mempool list for shared access. + */ +void +rte_mcfg_mempool_read_unlock(void); + +/** + * Lock the internal EAL Mempool list for exclusive access. + */ +void +rte_mcfg_mempool_write_lock(void); + +/** + * Unlock the internal EAL Mempool list for exclusive access. + */ +void +rte_mcfg_mempool_write_unlock(void); + #ifdef __cplusplus } #endif diff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map index d78a3a8b9..cc4ef04a0 100644 --- a/lib/librte_eal/rte_eal_version.map +++ b/lib/librte_eal/rte_eal_version.map @@ -296,6 +296,10 @@ DPDK_19.08 { rte_mcfg_mem_read_unlock; rte_mcfg_mem_write_lock; rte_mcfg_mem_write_unlock; + rte_mcfg_mempool_read_lock; + rte_mcfg_mempool_read_unlock; + rte_mcfg_mempool_write_lock; + rte_mcfg_mempool_write_unlock; rte_mcfg_tailq_read_lock; rte_mcfg_tailq_read_unlock; rte_mcfg_tailq_write_lock; diff --git a/lib/librte_mempool/rte_mempool.c b/lib/librte_mempool/rte_mempool.c index 238287a01..5c688d456 100644 --- a/lib/librte_mempool/rte_mempool.c +++ b/lib/librte_mempool/rte_mempool.c @@ -830,7 +830,7 @@ rte_mempool_create_empty(const char *name, unsigned n, unsigned elt_size, return NULL; } - rte_rwlock_write_lock(RTE_EAL_MEMPOOL_RWLOCK); + rte_mcfg_mempool_write_lock(); /* * reserve a memory zone for this mempool: private data is @@ -901,12 +901,12 @@ rte_mempool_create_empty(const char *name, unsigned n, unsigned elt_size, rte_mcfg_tailq_write_lock(); TAILQ_INSERT_TAIL(mempool_list, te, next); rte_mcfg_tailq_write_unlock(); - rte_rwlock_write_unlock(RTE_EAL_MEMPOOL_RWLOCK); + rte_mcfg_mempool_write_unlock(); return mp; exit_unlock: - rte_rwlock_write_unlock(RTE_EAL_MEMPOOL_RWLOCK); + rte_mcfg_mempool_write_unlock(); rte_free(te); rte_mempool_free(mp); return NULL; @@ -1268,14 +1268,14 @@ rte_mempool_list_dump(FILE *f) mempool_list = RTE_TAILQ_CAST(rte_mempool_tailq.head, rte_mempool_list); - rte_rwlock_read_lock(RTE_EAL_MEMPOOL_RWLOCK); + rte_mcfg_mempool_read_lock(); TAILQ_FOREACH(te, mempool_list, next) { mp = (struct rte_mempool *) te->data; rte_mempool_dump(f, mp); } - rte_rwlock_read_unlock(RTE_EAL_MEMPOOL_RWLOCK); + rte_mcfg_mempool_read_unlock(); } /* search a mempool from its name */ @@ -1288,7 +1288,7 @@ rte_mempool_lookup(const char *name) mempool_list = RTE_TAILQ_CAST(rte_mempool_tailq.head, rte_mempool_list); - rte_rwlock_read_lock(RTE_EAL_MEMPOOL_RWLOCK); + rte_mcfg_mempool_read_lock(); TAILQ_FOREACH(te, mempool_list, next) { mp = (struct rte_mempool *) te->data; @@ -1296,7 +1296,7 @@ rte_mempool_lookup(const char *name) break; } - rte_rwlock_read_unlock(RTE_EAL_MEMPOOL_RWLOCK); + rte_mcfg_mempool_read_unlock(); if (te == NULL) { rte_errno = ENOENT; @@ -1315,11 +1315,11 @@ void rte_mempool_walk(void (*func)(struct rte_mempool *, void *), mempool_list = RTE_TAILQ_CAST(rte_mempool_tailq.head, rte_mempool_list); - rte_rwlock_read_lock(RTE_EAL_MEMPOOL_RWLOCK); + rte_mcfg_mempool_read_lock(); TAILQ_FOREACH_SAFE(te, mempool_list, next, tmp_te) { (*func)((struct rte_mempool *) te->data, arg); } - rte_rwlock_read_unlock(RTE_EAL_MEMPOOL_RWLOCK); + rte_mcfg_mempool_read_unlock(); } From patchwork Fri Jul 5 13:10:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anatoly Burakov X-Patchwork-Id: 56159 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1D3C51BED5; Fri, 5 Jul 2019 15:10:57 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id 470141BED0 for ; Fri, 5 Jul 2019 15:10:54 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 05 Jul 2019 06:10:53 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.63,455,1557212400"; d="scan'208";a="187863020" Received: from silpixa00399498.ir.intel.com (HELO silpixa00399498.ger.corp.intel.com) ([10.237.223.125]) by fmsmga004.fm.intel.com with ESMTP; 05 Jul 2019 06:10:49 -0700 From: Anatoly Burakov To: dev@dpdk.org Cc: Neil Horman , John McNamara , Marko Kovacevic , Konstantin Ananyev , David Hunt , Bruce Richardson , Byron Marohn , Pablo de Lara Guarch , Yipeng Wang , Sameh Gobriel , Vladimir Medvedkin , Olivier Matz , Andrew Rybchenko , Honnappa Nagarahalli , Reshma Pattan , thomas@monjalon.net, david.marchand@redhat.com, stephen@networkplumber.org Date: Fri, 5 Jul 2019 14:10:30 +0100 Message-Id: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: Subject: [dpdk-dev] [PATCH v4 4/8] eal: hide shared memory config X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Now that everything that has ever accessed the shared memory config is doing so through the public API's, we can make it internal. Since we're removing quite a few headers from rte_eal_memconfig.h, we need to add them back in places where this header is used. This bumps the ABI, so also change all build files and make update documentation. Signed-off-by: Anatoly Burakov Acked-by: Stephen Hemminger --- app/test/test_fbarray.c | 1 + app/test/test_memzone.c | 1 + app/test/test_tailq.c | 1 + doc/guides/rel_notes/deprecation.rst | 3 - doc/guides/rel_notes/release_19_08.rst | 8 +- drivers/bus/pci/linux/pci_vfio.c | 1 + lib/librte_acl/rte_acl.c | 2 + lib/librte_distributor/rte_distributor.c | 1 + lib/librte_distributor/rte_distributor_v20.c | 1 + lib/librte_eal/common/eal_common_mcfg.c | 2 + lib/librte_eal/common/eal_common_memory.c | 1 + lib/librte_eal/common/eal_common_memzone.c | 1 + lib/librte_eal/common/eal_common_tailqs.c | 1 + lib/librte_eal/common/eal_memcfg.h | 80 ++++++++++++++++ .../common/include/rte_eal_memconfig.h | 95 +------------------ lib/librte_eal/common/include/rte_fbarray.h | 1 - lib/librte_eal/common/include/rte_memory.h | 24 ++++- lib/librte_eal/common/malloc_heap.c | 2 + lib/librte_eal/common/malloc_mp.c | 1 + lib/librte_eal/common/rte_malloc.c | 1 + lib/librte_eal/freebsd/eal/Makefile | 2 +- lib/librte_eal/freebsd/eal/eal_memory.c | 1 + lib/librte_eal/linux/eal/Makefile | 2 +- lib/librte_eal/linux/eal/eal.c | 1 + lib/librte_eal/linux/eal/eal_memalloc.c | 1 + lib/librte_eal/linux/eal/eal_memory.c | 1 + lib/librte_eal/linux/eal/eal_vfio.c | 1 + lib/librte_eal/meson.build | 2 +- lib/librte_efd/rte_efd.c | 1 + lib/librte_hash/rte_cuckoo_hash.c | 1 + lib/librte_hash/rte_fbk_hash.c | 1 + lib/librte_lpm/rte_lpm.c | 1 + lib/librte_lpm/rte_lpm6.c | 1 + lib/librte_member/rte_member.c | 1 + lib/librte_mempool/rte_mempool.c | 1 + lib/librte_rcu/rte_rcu_qsbr.h | 1 + lib/librte_reorder/rte_reorder.c | 1 + lib/librte_ring/rte_ring.c | 1 + 38 files changed, 148 insertions(+), 101 deletions(-) create mode 100644 lib/librte_eal/common/eal_memcfg.h diff --git a/app/test/test_fbarray.c b/app/test/test_fbarray.c index d2b041887..a691bf445 100644 --- a/app/test/test_fbarray.c +++ b/app/test/test_fbarray.c @@ -2,6 +2,7 @@ * Copyright(c) 2010-2014 Intel Corporation */ +#include #include #include #include diff --git a/app/test/test_memzone.c b/app/test/test_memzone.c index 9fe465e62..7501b63c5 100644 --- a/app/test/test_memzone.c +++ b/app/test/test_memzone.c @@ -19,6 +19,7 @@ #include #include #include "../../lib/librte_eal/common/malloc_elem.h" +#include "../../lib/librte_eal/common/eal_memcfg.h" #include "test.h" diff --git a/app/test/test_tailq.c b/app/test/test_tailq.c index a4ecea2d8..7c9b69fdb 100644 --- a/app/test/test_tailq.c +++ b/app/test/test_tailq.c @@ -12,6 +12,7 @@ #include #include #include +#include #include "test.h" diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst index e2721fad6..583217da8 100644 --- a/doc/guides/rel_notes/deprecation.rst +++ b/doc/guides/rel_notes/deprecation.rst @@ -23,9 +23,6 @@ Deprecation Notices * eal: The function ``rte_eal_remote_launch`` will return new error codes after read or write error on the pipe, instead of calling ``rte_panic``. -* eal: the ``rte_mem_config`` struct will be made private to remove it from the - externally visible ABI and allow it to be updated in the future. - * eal: both declaring and identifying devices will be streamlined in v18.11. New functions will appear to query a specific port from buses, classes of device and device drivers. Device declaration will be made coherent with the diff --git a/doc/guides/rel_notes/release_19_08.rst b/doc/guides/rel_notes/release_19_08.rst index 21934bf01..cc6dfa33f 100644 --- a/doc/guides/rel_notes/release_19_08.rst +++ b/doc/guides/rel_notes/release_19_08.rst @@ -172,6 +172,10 @@ API Changes Also, make sure to start the actual text at the margin. ========================================================= +* The ``rte_mem_config`` structure has been made private. The new accessor + ``rte_mcfg_*`` functions were introduced to provide replacement for direct + access to the shared mem config. + * The network structures, definitions and functions have been prefixed by ``rte_`` to resolve conflicts with libc headers. @@ -201,6 +205,8 @@ ABI Changes Also, make sure to start the actual text at the margin. ========================================================= +* The ``rte_mem_config`` structure has been made private. + * eventdev: Event based Rx adapter callback The mbuf pointer array in the event eth Rx adapter callback @@ -246,7 +252,7 @@ The libraries prepended with a plus sign were incremented in this version. librte_compressdev.so.1 librte_cryptodev.so.7 librte_distributor.so.1 - librte_eal.so.10 + + librte_eal.so.11 librte_efd.so.1 librte_ethdev.so.12 + librte_eventdev.so.7 diff --git a/drivers/bus/pci/linux/pci_vfio.c b/drivers/bus/pci/linux/pci_vfio.c index ebf6ccd3c..1ceb1c07b 100644 --- a/drivers/bus/pci/linux/pci_vfio.c +++ b/drivers/bus/pci/linux/pci_vfio.c @@ -20,6 +20,7 @@ #include #include #include +#include #include "eal_filesystem.h" diff --git a/lib/librte_acl/rte_acl.c b/lib/librte_acl/rte_acl.c index 7ff11d25f..bd7247cc3 100644 --- a/lib/librte_acl/rte_acl.c +++ b/lib/librte_acl/rte_acl.c @@ -4,6 +4,8 @@ #include #include +#include + #include "acl.h" TAILQ_HEAD(rte_acl_list, rte_tailq_entry); diff --git a/lib/librte_distributor/rte_distributor.c b/lib/librte_distributor/rte_distributor.c index 9eb78b330..0a3213bbf 100644 --- a/lib/librte_distributor/rte_distributor.c +++ b/lib/librte_distributor/rte_distributor.c @@ -14,6 +14,7 @@ #include #include #include +#include #include "rte_distributor_private.h" #include "rte_distributor.h" diff --git a/lib/librte_distributor/rte_distributor_v20.c b/lib/librte_distributor/rte_distributor_v20.c index 1fc03b971..cdc0969a8 100644 --- a/lib/librte_distributor/rte_distributor_v20.c +++ b/lib/librte_distributor/rte_distributor_v20.c @@ -13,6 +13,7 @@ #include #include #include +#include #include "rte_distributor_v20.h" #include "rte_distributor_private.h" diff --git a/lib/librte_eal/common/eal_common_mcfg.c b/lib/librte_eal/common/eal_common_mcfg.c index ba2bc37b7..337890a61 100644 --- a/lib/librte_eal/common/eal_common_mcfg.c +++ b/lib/librte_eal/common/eal_common_mcfg.c @@ -5,6 +5,8 @@ #include #include +#include "eal_memcfg.h" + void rte_mcfg_mem_read_lock(void) { diff --git a/lib/librte_eal/common/eal_common_memory.c b/lib/librte_eal/common/eal_common_memory.c index fe22b139b..19ea47570 100644 --- a/lib/librte_eal/common/eal_common_memory.c +++ b/lib/librte_eal/common/eal_common_memory.c @@ -24,6 +24,7 @@ #include "eal_memalloc.h" #include "eal_private.h" #include "eal_internal_cfg.h" +#include "eal_memcfg.h" #include "malloc_heap.h" /* diff --git a/lib/librte_eal/common/eal_common_memzone.c b/lib/librte_eal/common/eal_common_memzone.c index 521ad7ca1..ef6c909cb 100644 --- a/lib/librte_eal/common/eal_common_memzone.c +++ b/lib/librte_eal/common/eal_common_memzone.c @@ -24,6 +24,7 @@ #include "malloc_heap.h" #include "malloc_elem.h" #include "eal_private.h" +#include "eal_memcfg.h" static inline const struct rte_memzone * memzone_lookup_thread_unsafe(const char *name) diff --git a/lib/librte_eal/common/eal_common_tailqs.c b/lib/librte_eal/common/eal_common_tailqs.c index dc2c13caa..ead06897b 100644 --- a/lib/librte_eal/common/eal_common_tailqs.c +++ b/lib/librte_eal/common/eal_common_tailqs.c @@ -23,6 +23,7 @@ #include #include "eal_private.h" +#include "eal_memcfg.h" TAILQ_HEAD(rte_tailq_elem_head, rte_tailq_elem); /* local tailq list */ diff --git a/lib/librte_eal/common/eal_memcfg.h b/lib/librte_eal/common/eal_memcfg.h new file mode 100644 index 000000000..74f6159c6 --- /dev/null +++ b/lib/librte_eal/common/eal_memcfg.h @@ -0,0 +1,80 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2019 Intel Corporation + */ + +#ifndef EAL_MEMCFG_H +#define EAL_MEMCFG_H + +#include +#include +#include +#include +#include +#include +#include +#include + +/** + * Memory configuration shared across multiple processes. + */ +struct rte_mem_config { + volatile uint32_t magic; /**< Magic number - sanity check. */ + + /* memory topology */ + uint32_t nchannel; /**< Number of channels (0 if unknown). */ + uint32_t nrank; /**< Number of ranks (0 if unknown). */ + + /** + * current lock nest order + * - qlock->mlock (ring/hash/lpm) + * - mplock->qlock->mlock (mempool) + * Notice: + * *ALWAYS* obtain qlock first if having to obtain both qlock and mlock + */ + rte_rwlock_t mlock; /**< used by memzones for thread safety. */ + rte_rwlock_t qlock; /**< used by tailqs for thread safety. */ + rte_rwlock_t mplock; /**< used by mempool library for thread safety. */ + + rte_rwlock_t memory_hotplug_lock; + /**< Indicates whether memory hotplug request is in progress. */ + + /* memory segments and zones */ + struct rte_fbarray memzones; /**< Memzone descriptors. */ + + struct rte_memseg_list memsegs[RTE_MAX_MEMSEG_LISTS]; + /**< List of dynamic arrays holding memsegs */ + + struct rte_tailq_head tailq_head[RTE_MAX_TAILQ]; + /**< Tailqs for objects */ + + struct malloc_heap malloc_heaps[RTE_MAX_HEAPS]; + /**< DPDK malloc heaps */ + + int next_socket_id; /**< Next socket ID for external malloc heap */ + + /* rte_mem_config has to be mapped at the exact same address in all + * processes, so we need to store it. + */ + uint64_t mem_cfg_addr; /**< Address of this structure in memory. */ + + /* Primary and secondary processes cannot run with different legacy or + * single file segments options, so to avoid having to specify these + * options to all processes, store them in shared config and update the + * internal config at init time. + */ + uint32_t legacy_mem; /**< stored legacy mem parameter. */ + uint32_t single_file_segments; + /**< stored single file segments parameter. */ + + uint8_t dma_maskbits; /**< Keeps the more restricted dma mask. */ +} __attribute__((packed)); + +static inline void +rte_eal_mcfg_wait_complete(struct rte_mem_config *mcfg) +{ + /* wait until shared mem_config finish initialising */ + while (mcfg->magic != RTE_MAGIC) + rte_pause(); +} + +#endif /* EAL_MEMCFG_H */ diff --git a/lib/librte_eal/common/include/rte_eal_memconfig.h b/lib/librte_eal/common/include/rte_eal_memconfig.h index 58dcbb96d..dc61a6fed 100644 --- a/lib/librte_eal/common/include/rte_eal_memconfig.h +++ b/lib/librte_eal/common/include/rte_eal_memconfig.h @@ -5,101 +5,16 @@ #ifndef _RTE_EAL_MEMCONFIG_H_ #define _RTE_EAL_MEMCONFIG_H_ -#include -#include -#include -#include -#include -#include -#include -#include +/** + * @file + * + * This API allows access to EAL shared memory configuration through an API. + */ #ifdef __cplusplus extern "C" { #endif -/** - * memseg list is a special case as we need to store a bunch of other data - * together with the array itself. - */ -struct rte_memseg_list { - RTE_STD_C11 - union { - void *base_va; - /**< Base virtual address for this memseg list. */ - uint64_t addr_64; - /**< Makes sure addr is always 64-bits */ - }; - uint64_t page_sz; /**< Page size for all memsegs in this list. */ - int socket_id; /**< Socket ID for all memsegs in this list. */ - volatile uint32_t version; /**< version number for multiprocess sync. */ - size_t len; /**< Length of memory area covered by this memseg list. */ - unsigned int external; /**< 1 if this list points to external memory */ - struct rte_fbarray memseg_arr; -}; - -/** - * the structure for the memory configuration for the RTE. - * Used by the rte_config structure. It is separated out, as for multi-process - * support, the memory details should be shared across instances - */ -struct rte_mem_config { - volatile uint32_t magic; /**< Magic number - Sanity check. */ - - /* memory topology */ - uint32_t nchannel; /**< Number of channels (0 if unknown). */ - uint32_t nrank; /**< Number of ranks (0 if unknown). */ - - /** - * current lock nest order - * - qlock->mlock (ring/hash/lpm) - * - mplock->qlock->mlock (mempool) - * Notice: - * *ALWAYS* obtain qlock first if having to obtain both qlock and mlock - */ - rte_rwlock_t mlock; /**< only used by memzone LIB for thread-safe. */ - rte_rwlock_t qlock; /**< used for tailq operation for thread safe. */ - rte_rwlock_t mplock; /**< only used by mempool LIB for thread-safe. */ - - rte_rwlock_t memory_hotplug_lock; - /**< indicates whether memory hotplug request is in progress. */ - - /* memory segments and zones */ - struct rte_fbarray memzones; /**< Memzone descriptors. */ - - struct rte_memseg_list memsegs[RTE_MAX_MEMSEG_LISTS]; - /**< list of dynamic arrays holding memsegs */ - - struct rte_tailq_head tailq_head[RTE_MAX_TAILQ]; /**< Tailqs for objects */ - - /* Heaps of Malloc */ - struct malloc_heap malloc_heaps[RTE_MAX_HEAPS]; - - /* next socket ID for external malloc heap */ - int next_socket_id; - - /* address of mem_config in primary process. used to map shared config into - * exact same address the primary process maps it. - */ - uint64_t mem_cfg_addr; - - /* legacy mem and single file segments options are shared */ - uint32_t legacy_mem; - uint32_t single_file_segments; - - /* keeps the more restricted dma mask */ - uint8_t dma_maskbits; -} __attribute__((__packed__)); - - -inline static void -rte_eal_mcfg_wait_complete(struct rte_mem_config* mcfg) -{ - /* wait until shared mem_config finish initialising */ - while(mcfg->magic != RTE_MAGIC) - rte_pause(); -} - /** * Lock the internal EAL shared memory configuration for shared access. */ diff --git a/lib/librte_eal/common/include/rte_fbarray.h b/lib/librte_eal/common/include/rte_fbarray.h index d0af2d8c7..6dccdbec9 100644 --- a/lib/librte_eal/common/include/rte_fbarray.h +++ b/lib/librte_eal/common/include/rte_fbarray.h @@ -34,7 +34,6 @@ extern "C" { #endif -#include #include #include diff --git a/lib/librte_eal/common/include/rte_memory.h b/lib/librte_eal/common/include/rte_memory.h index 44cbe6fd2..ca34a10c3 100644 --- a/lib/librte_eal/common/include/rte_memory.h +++ b/lib/librte_eal/common/include/rte_memory.h @@ -22,9 +22,7 @@ extern "C" { #include #include #include - -/* forward declaration for pointers */ -struct rte_memseg_list; +#include __extension__ enum rte_page_sizes { @@ -104,6 +102,26 @@ struct rte_memseg { uint32_t flags; /**< Memseg-specific flags */ } __rte_packed; +/** + * memseg list is a special case as we need to store a bunch of other data + * together with the array itself. + */ +struct rte_memseg_list { + RTE_STD_C11 + union { + void *base_va; + /**< Base virtual address for this memseg list. */ + uint64_t addr_64; + /**< Makes sure addr is always 64-bits */ + }; + uint64_t page_sz; /**< Page size for all memsegs in this list. */ + int socket_id; /**< Socket ID for all memsegs in this list. */ + volatile uint32_t version; /**< version number for multiprocess sync. */ + size_t len; /**< Length of memory area covered by this memseg list. */ + unsigned int external; /**< 1 if this list points to external memory */ + struct rte_fbarray memseg_arr; +}; + /** * Lock page in physical memory and prevent from swapping. * diff --git a/lib/librte_eal/common/malloc_heap.c b/lib/librte_eal/common/malloc_heap.c index f1d31de0d..634ca212f 100644 --- a/lib/librte_eal/common/malloc_heap.c +++ b/lib/librte_eal/common/malloc_heap.c @@ -20,11 +20,13 @@ #include #include #include +#include #include #include #include "eal_internal_cfg.h" #include "eal_memalloc.h" +#include "eal_memcfg.h" #include "malloc_elem.h" #include "malloc_heap.h" #include "malloc_mp.h" diff --git a/lib/librte_eal/common/malloc_mp.c b/lib/librte_eal/common/malloc_mp.c index 7c6112c4e..1f212f834 100644 --- a/lib/librte_eal/common/malloc_mp.c +++ b/lib/librte_eal/common/malloc_mp.c @@ -10,6 +10,7 @@ #include #include "eal_memalloc.h" +#include "eal_memcfg.h" #include "malloc_elem.h" #include "malloc_mp.h" diff --git a/lib/librte_eal/common/rte_malloc.c b/lib/librte_eal/common/rte_malloc.c index 2cad7beaa..fecd9a964 100644 --- a/lib/librte_eal/common/rte_malloc.c +++ b/lib/librte_eal/common/rte_malloc.c @@ -25,6 +25,7 @@ #include "malloc_elem.h" #include "malloc_heap.h" #include "eal_memalloc.h" +#include "eal_memcfg.h" /* Free the memory space back to heap */ diff --git a/lib/librte_eal/freebsd/eal/Makefile b/lib/librte_eal/freebsd/eal/Makefile index eb921275e..89131ea89 100644 --- a/lib/librte_eal/freebsd/eal/Makefile +++ b/lib/librte_eal/freebsd/eal/Makefile @@ -22,7 +22,7 @@ LDLIBS += -lrte_kvargs EXPORT_MAP := ../../rte_eal_version.map -LIBABIVER := 10 +LIBABIVER := 11 # specific to freebsd exec-env SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) := eal.c diff --git a/lib/librte_eal/freebsd/eal/eal_memory.c b/lib/librte_eal/freebsd/eal/eal_memory.c index 4b092e1f2..9b9a0577a 100644 --- a/lib/librte_eal/freebsd/eal/eal_memory.c +++ b/lib/librte_eal/freebsd/eal/eal_memory.c @@ -18,6 +18,7 @@ #include "eal_private.h" #include "eal_internal_cfg.h" #include "eal_filesystem.h" +#include "eal_memcfg.h" #define EAL_PAGE_SIZE (sysconf(_SC_PAGESIZE)) diff --git a/lib/librte_eal/linux/eal/Makefile b/lib/librte_eal/linux/eal/Makefile index dfe8e9a49..0f5725e64 100644 --- a/lib/librte_eal/linux/eal/Makefile +++ b/lib/librte_eal/linux/eal/Makefile @@ -10,7 +10,7 @@ ARCH_DIR ?= $(RTE_ARCH) EXPORT_MAP := ../../rte_eal_version.map VPATH += $(RTE_SDK)/lib/librte_eal/common/arch/$(ARCH_DIR) -LIBABIVER := 10 +LIBABIVER := 11 VPATH += $(RTE_SDK)/lib/librte_eal/common diff --git a/lib/librte_eal/linux/eal/eal.c b/lib/librte_eal/linux/eal/eal.c index 8a0b387ce..f974cec1c 100644 --- a/lib/librte_eal/linux/eal/eal.c +++ b/lib/librte_eal/linux/eal/eal.c @@ -57,6 +57,7 @@ #include "eal_internal_cfg.h" #include "eal_filesystem.h" #include "eal_hugepages.h" +#include "eal_memcfg.h" #include "eal_options.h" #include "eal_vfio.h" #include "hotplug_mp.h" diff --git a/lib/librte_eal/linux/eal/eal_memalloc.c b/lib/librte_eal/linux/eal/eal_memalloc.c index 2019636fb..1f6a7c18f 100644 --- a/lib/librte_eal/linux/eal/eal_memalloc.c +++ b/lib/librte_eal/linux/eal/eal_memalloc.c @@ -44,6 +44,7 @@ #include "eal_filesystem.h" #include "eal_internal_cfg.h" #include "eal_memalloc.h" +#include "eal_memcfg.h" #include "eal_private.h" const int anonymous_hugepages_supported = diff --git a/lib/librte_eal/linux/eal/eal_memory.c b/lib/librte_eal/linux/eal/eal_memory.c index 1853acea5..9c948a374 100644 --- a/lib/librte_eal/linux/eal/eal_memory.c +++ b/lib/librte_eal/linux/eal/eal_memory.c @@ -46,6 +46,7 @@ #include "eal_private.h" #include "eal_memalloc.h" +#include "eal_memcfg.h" #include "eal_internal_cfg.h" #include "eal_filesystem.h" #include "eal_hugepages.h" diff --git a/lib/librte_eal/linux/eal/eal_vfio.c b/lib/librte_eal/linux/eal/eal_vfio.c index 96a03a657..fdc884f70 100644 --- a/lib/librte_eal/linux/eal/eal_vfio.c +++ b/lib/librte_eal/linux/eal/eal_vfio.c @@ -15,6 +15,7 @@ #include #include "eal_filesystem.h" +#include "eal_memcfg.h" #include "eal_vfio.h" #include "eal_private.h" diff --git a/lib/librte_eal/meson.build b/lib/librte_eal/meson.build index ccd5b85b8..2751023a9 100644 --- a/lib/librte_eal/meson.build +++ b/lib/librte_eal/meson.build @@ -12,7 +12,7 @@ subdir('common') # defines common_sources, common_objs, etc. dpdk_conf.set('RTE_EXEC_ENV_' + exec_env.to_upper(), 1) subdir(exec_env + '/eal') -version = 10 # the version of the EAL API +version = 11 # the version of the EAL API allow_experimental_apis = true deps += 'kvargs' if dpdk_conf.has('RTE_USE_LIBBSD') diff --git a/lib/librte_efd/rte_efd.c b/lib/librte_efd/rte_efd.c index b808ce99f..d3d019578 100644 --- a/lib/librte_efd/rte_efd.c +++ b/lib/librte_efd/rte_efd.c @@ -20,6 +20,7 @@ #include #include #include +#include #include "rte_efd.h" #if defined(RTE_ARCH_X86) diff --git a/lib/librte_hash/rte_cuckoo_hash.c b/lib/librte_hash/rte_cuckoo_hash.c index 47cd0da5b..74d5bbd99 100644 --- a/lib/librte_hash/rte_cuckoo_hash.c +++ b/lib/librte_hash/rte_cuckoo_hash.c @@ -27,6 +27,7 @@ #include #include #include +#include #include "rte_hash.h" #include "rte_cuckoo_hash.h" diff --git a/lib/librte_hash/rte_fbk_hash.c b/lib/librte_hash/rte_fbk_hash.c index db118c930..576e8e666 100644 --- a/lib/librte_hash/rte_fbk_hash.c +++ b/lib/librte_hash/rte_fbk_hash.c @@ -20,6 +20,7 @@ #include #include #include +#include #include "rte_fbk_hash.h" diff --git a/lib/librte_lpm/rte_lpm.c b/lib/librte_lpm/rte_lpm.c index b91f74216..70c24ac1f 100644 --- a/lib/librte_lpm/rte_lpm.c +++ b/lib/librte_lpm/rte_lpm.c @@ -21,6 +21,7 @@ #include #include #include +#include #include "rte_lpm.h" diff --git a/lib/librte_lpm/rte_lpm6.c b/lib/librte_lpm/rte_lpm6.c index 5af74a539..9b8aeb972 100644 --- a/lib/librte_lpm/rte_lpm6.c +++ b/lib/librte_lpm/rte_lpm6.c @@ -24,6 +24,7 @@ #include #include #include +#include #include "rte_lpm6.h" diff --git a/lib/librte_member/rte_member.c b/lib/librte_member/rte_member.c index efed28dd9..e0e7f127e 100644 --- a/lib/librte_member/rte_member.c +++ b/lib/librte_member/rte_member.c @@ -10,6 +10,7 @@ #include #include #include +#include #include "rte_member.h" #include "rte_member_ht.h" diff --git a/lib/librte_mempool/rte_mempool.c b/lib/librte_mempool/rte_mempool.c index 5c688d456..7260ce0be 100644 --- a/lib/librte_mempool/rte_mempool.c +++ b/lib/librte_mempool/rte_mempool.c @@ -30,6 +30,7 @@ #include #include #include +#include #include "rte_mempool.h" diff --git a/lib/librte_rcu/rte_rcu_qsbr.h b/lib/librte_rcu/rte_rcu_qsbr.h index 53a5a7165..c80f15c00 100644 --- a/lib/librte_rcu/rte_rcu_qsbr.h +++ b/lib/librte_rcu/rte_rcu_qsbr.h @@ -24,6 +24,7 @@ extern "C" { #endif +#include #include #include #include diff --git a/lib/librte_reorder/rte_reorder.c b/lib/librte_reorder/rte_reorder.c index ae6e3f578..3c9f0e2d0 100644 --- a/lib/librte_reorder/rte_reorder.c +++ b/lib/librte_reorder/rte_reorder.c @@ -11,6 +11,7 @@ #include #include #include +#include #include "rte_reorder.h" diff --git a/lib/librte_ring/rte_ring.c b/lib/librte_ring/rte_ring.c index 9ea26a631..b30b2aa7b 100644 --- a/lib/librte_ring/rte_ring.c +++ b/lib/librte_ring/rte_ring.c @@ -30,6 +30,7 @@ #include #include #include +#include #include "rte_ring.h" From patchwork Fri Jul 5 13:10:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anatoly Burakov X-Patchwork-Id: 56160 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id F40F91BEE0; Fri, 5 Jul 2019 15:10:59 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id 827011BEC5 for ; Fri, 5 Jul 2019 15:10:55 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 05 Jul 2019 06:10:55 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.63,455,1557212400"; d="scan'208";a="187863023" Received: from silpixa00399498.ir.intel.com (HELO silpixa00399498.ger.corp.intel.com) ([10.237.223.125]) by fmsmga004.fm.intel.com with ESMTP; 05 Jul 2019 06:10:54 -0700 From: Anatoly Burakov To: dev@dpdk.org Cc: thomas@monjalon.net, david.marchand@redhat.com, stephen@networkplumber.org Date: Fri, 5 Jul 2019 14:10:31 +0100 Message-Id: <194b654d2c27693033d6b750f66c2ba22dcece58.1562332112.git.anatoly.burakov@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: Subject: [dpdk-dev] [PATCH v4 5/8] eal: remove packed attribute from mcfg structure X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" There is no reason to pack the memconfig structure, and doing so gives out warnings in some static analyzers. Fix it by removing the packed attributed. Signed-off-by: Anatoly Burakov Acked-by: Stephen Hemminger --- lib/librte_eal/common/eal_memcfg.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/librte_eal/common/eal_memcfg.h b/lib/librte_eal/common/eal_memcfg.h index 74f6159c6..e5b90e31d 100644 --- a/lib/librte_eal/common/eal_memcfg.h +++ b/lib/librte_eal/common/eal_memcfg.h @@ -67,7 +67,7 @@ struct rte_mem_config { /**< stored single file segments parameter. */ uint8_t dma_maskbits; /**< Keeps the more restricted dma mask. */ -} __attribute__((packed)); +}; static inline void rte_eal_mcfg_wait_complete(struct rte_mem_config *mcfg) From patchwork Fri Jul 5 13:10:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anatoly Burakov X-Patchwork-Id: 56161 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B2D661BEED; Fri, 5 Jul 2019 15:11:02 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id 15EEF1BED4 for ; Fri, 5 Jul 2019 15:10:56 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 05 Jul 2019 06:10:56 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.63,455,1557212400"; d="scan'208";a="187863043" Received: from silpixa00399498.ir.intel.com (HELO silpixa00399498.ger.corp.intel.com) ([10.237.223.125]) by fmsmga004.fm.intel.com with ESMTP; 05 Jul 2019 06:10:55 -0700 From: Anatoly Burakov To: dev@dpdk.org Cc: Bruce Richardson , thomas@monjalon.net, david.marchand@redhat.com, stephen@networkplumber.org Date: Fri, 5 Jul 2019 14:10:32 +0100 Message-Id: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: Subject: [dpdk-dev] [PATCH v4 6/8] eal: uninline wait for mcfg complete function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Currently, the function to wait until config completion is static inline for no reason. Move its implementation to an EAL common file. Signed-off-by: Anatoly Burakov Acked-by: Stephen Hemminger --- lib/librte_eal/common/eal_common_mcfg.c | 10 ++++++++++ lib/librte_eal/common/eal_memcfg.h | 10 +++------- lib/librte_eal/freebsd/eal/eal.c | 3 ++- lib/librte_eal/linux/eal/eal.c | 2 +- 4 files changed, 16 insertions(+), 9 deletions(-) diff --git a/lib/librte_eal/common/eal_common_mcfg.c b/lib/librte_eal/common/eal_common_mcfg.c index 337890a61..30969c6bf 100644 --- a/lib/librte_eal/common/eal_common_mcfg.c +++ b/lib/librte_eal/common/eal_common_mcfg.c @@ -7,6 +7,16 @@ #include "eal_memcfg.h" +void +eal_mcfg_wait_complete(void) +{ + struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; + + /* wait until shared mem_config finish initialising */ + while (mcfg->magic != RTE_MAGIC) + rte_pause(); +} + void rte_mcfg_mem_read_lock(void) { diff --git a/lib/librte_eal/common/eal_memcfg.h b/lib/librte_eal/common/eal_memcfg.h index e5b90e31d..6c08652fe 100644 --- a/lib/librte_eal/common/eal_memcfg.h +++ b/lib/librte_eal/common/eal_memcfg.h @@ -69,12 +69,8 @@ struct rte_mem_config { uint8_t dma_maskbits; /**< Keeps the more restricted dma mask. */ }; -static inline void -rte_eal_mcfg_wait_complete(struct rte_mem_config *mcfg) -{ - /* wait until shared mem_config finish initialising */ - while (mcfg->magic != RTE_MAGIC) - rte_pause(); -} +/* wait until primary process initialization is complete */ +void +eal_mcfg_wait_complete(void); #endif /* EAL_MEMCFG_H */ diff --git a/lib/librte_eal/freebsd/eal/eal.c b/lib/librte_eal/freebsd/eal/eal.c index 71ba380e0..78022f68a 100644 --- a/lib/librte_eal/freebsd/eal/eal.c +++ b/lib/librte_eal/freebsd/eal/eal.c @@ -52,6 +52,7 @@ #include "eal_filesystem.h" #include "eal_hugepages.h" #include "eal_options.h" +#include "eal_memcfg.h" #define MEMSIZE_IF_NO_HUGE_PAGE (64ULL * 1024ULL * 1024ULL) @@ -382,7 +383,7 @@ rte_config_init(void) case RTE_PROC_SECONDARY: if (rte_eal_config_attach() < 0) return -1; - rte_eal_mcfg_wait_complete(rte_config.mem_config); + eal_mcfg_wait_complete(); if (rte_eal_config_reattach() < 0) return -1; break; diff --git a/lib/librte_eal/linux/eal/eal.c b/lib/librte_eal/linux/eal/eal.c index f974cec1c..561a4f3f3 100644 --- a/lib/librte_eal/linux/eal/eal.c +++ b/lib/librte_eal/linux/eal/eal.c @@ -506,7 +506,7 @@ rte_config_init(void) case RTE_PROC_SECONDARY: if (rte_eal_config_attach() < 0) return -1; - rte_eal_mcfg_wait_complete(rte_config.mem_config); + eal_mcfg_wait_complete(); if (rte_eal_config_reattach() < 0) return -1; eal_update_internal_config(); From patchwork Fri Jul 5 13:10:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anatoly Burakov X-Patchwork-Id: 56162 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A7BC01BEF2; Fri, 5 Jul 2019 15:11:04 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id 9A0AA1BED9 for ; Fri, 5 Jul 2019 15:10:58 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 05 Jul 2019 06:10:58 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.63,455,1557212400"; d="scan'208";a="187863051" Received: from silpixa00399498.ir.intel.com (HELO silpixa00399498.ger.corp.intel.com) ([10.237.223.125]) by fmsmga004.fm.intel.com with ESMTP; 05 Jul 2019 06:10:56 -0700 From: Anatoly Burakov To: dev@dpdk.org Cc: Bruce Richardson , thomas@monjalon.net, david.marchand@redhat.com, stephen@networkplumber.org Date: Fri, 5 Jul 2019 14:10:33 +0100 Message-Id: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: Subject: [dpdk-dev] [PATCH v4 7/8] eal: unify and move mcfg complete function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Currently, mcfg completion function exists in two independent implementations doing the same thing, which is bug prone. Unify the two functions and move them into one place. Signed-off-by: Anatoly Burakov Acked-by: Stephen Hemminger --- lib/librte_eal/common/eal_common_mcfg.c | 14 ++++++++++++++ lib/librte_eal/common/eal_memcfg.h | 4 ++++ lib/librte_eal/freebsd/eal/eal.c | 12 +----------- lib/librte_eal/linux/eal/eal.c | 12 +----------- 4 files changed, 20 insertions(+), 22 deletions(-) diff --git a/lib/librte_eal/common/eal_common_mcfg.c b/lib/librte_eal/common/eal_common_mcfg.c index 30969c6bf..dc6665d6a 100644 --- a/lib/librte_eal/common/eal_common_mcfg.c +++ b/lib/librte_eal/common/eal_common_mcfg.c @@ -5,8 +5,22 @@ #include #include +#include "eal_internal_cfg.h" #include "eal_memcfg.h" +void +eal_mcfg_complete(void) +{ + struct rte_config *cfg = rte_eal_get_configuration(); + struct rte_mem_config *mcfg = cfg->mem_config; + + /* ALL shared mem_config related INIT DONE */ + if (cfg->process_type == RTE_PROC_PRIMARY) + mcfg->magic = RTE_MAGIC; + + internal_config.init_complete = 1; +} + void eal_mcfg_wait_complete(void) { diff --git a/lib/librte_eal/common/eal_memcfg.h b/lib/librte_eal/common/eal_memcfg.h index 6c08652fe..417324aab 100644 --- a/lib/librte_eal/common/eal_memcfg.h +++ b/lib/librte_eal/common/eal_memcfg.h @@ -73,4 +73,8 @@ struct rte_mem_config { void eal_mcfg_wait_complete(void); +/* set mem config as complete */ +void +eal_mcfg_complete(void); + #endif /* EAL_MEMCFG_H */ diff --git a/lib/librte_eal/freebsd/eal/eal.c b/lib/librte_eal/freebsd/eal/eal.c index 78022f68a..340f7dd6c 100644 --- a/lib/librte_eal/freebsd/eal/eal.c +++ b/lib/librte_eal/freebsd/eal/eal.c @@ -632,16 +632,6 @@ sync_func(__attribute__((unused)) void *arg) return 0; } -inline static void -rte_eal_mcfg_complete(void) -{ - /* ALL shared mem_config related INIT DONE */ - if (rte_config.process_type == RTE_PROC_PRIMARY) - rte_config.mem_config->magic = RTE_MAGIC; - - internal_config.init_complete = 1; -} - /* return non-zero if hugepages are enabled. */ int rte_eal_has_hugepages(void) { @@ -919,7 +909,7 @@ rte_eal_init(int argc, char **argv) return -1; } - rte_eal_mcfg_complete(); + eal_mcfg_complete(); /* Call each registered callback, if enabled */ rte_option_init(); diff --git a/lib/librte_eal/linux/eal/eal.c b/lib/librte_eal/linux/eal/eal.c index 561a4f3f3..e15eacffc 100644 --- a/lib/librte_eal/linux/eal/eal.c +++ b/lib/librte_eal/linux/eal/eal.c @@ -934,16 +934,6 @@ sync_func(__attribute__((unused)) void *arg) return 0; } -inline static void -rte_eal_mcfg_complete(void) -{ - /* ALL shared mem_config related INIT DONE */ - if (rte_config.process_type == RTE_PROC_PRIMARY) - rte_config.mem_config->magic = RTE_MAGIC; - - internal_config.init_complete = 1; -} - /* * Request iopl privilege for all RPL, returns 0 on success * iopl() call is mostly for the i386 architecture. For other architectures, @@ -1267,7 +1257,7 @@ rte_eal_init(int argc, char **argv) return -1; } - rte_eal_mcfg_complete(); + eal_mcfg_complete(); /* Call each registered callback, if enabled */ rte_option_init(); From patchwork Fri Jul 5 13:10:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anatoly Burakov X-Patchwork-Id: 56163 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1F37E1BEFA; Fri, 5 Jul 2019 15:11:06 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id 271C41BEE3 for ; Fri, 5 Jul 2019 15:10:59 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 05 Jul 2019 06:10:59 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.63,455,1557212400"; d="scan'208";a="187863059" Received: from silpixa00399498.ir.intel.com (HELO silpixa00399498.ger.corp.intel.com) ([10.237.223.125]) by fmsmga004.fm.intel.com with ESMTP; 05 Jul 2019 06:10:58 -0700 From: Anatoly Burakov To: dev@dpdk.org Cc: Bruce Richardson , thomas@monjalon.net, david.marchand@redhat.com, stephen@networkplumber.org Date: Fri, 5 Jul 2019 14:10:34 +0100 Message-Id: <4d42544cf722666e493a236d5a1065c57726e8fa.1562332112.git.anatoly.burakov@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: Subject: [dpdk-dev] [PATCH v4 8/8] eal: unify internal config initialization X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Currently, each EAL will update internal/shared config in their own way at init, resulting in needless duplication of code and OS-dependent behavior. Move the functions to a common file and add missing FreeBSD steps. Signed-off-by: Anatoly Burakov Acked-by: Stephen Hemminger --- lib/librte_eal/common/eal_common_mcfg.c | 18 ++++++++++++++++++ lib/librte_eal/common/eal_memcfg.h | 8 ++++++++ lib/librte_eal/freebsd/eal/eal.c | 2 ++ lib/librte_eal/linux/eal/eal.c | 22 ++-------------------- 4 files changed, 30 insertions(+), 20 deletions(-) diff --git a/lib/librte_eal/common/eal_common_mcfg.c b/lib/librte_eal/common/eal_common_mcfg.c index dc6665d6a..fe8d2b726 100644 --- a/lib/librte_eal/common/eal_common_mcfg.c +++ b/lib/librte_eal/common/eal_common_mcfg.c @@ -31,6 +31,24 @@ eal_mcfg_wait_complete(void) rte_pause(); } +void +eal_mcfg_update_internal(void) +{ + struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; + + internal_config.legacy_mem = mcfg->legacy_mem; + internal_config.single_file_segments = mcfg->single_file_segments; +} + +void +eal_mcfg_update_from_internal(void) +{ + struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; + + mcfg->legacy_mem = internal_config.legacy_mem; + mcfg->single_file_segments = internal_config.single_file_segments; +} + void rte_mcfg_mem_read_lock(void) { diff --git a/lib/librte_eal/common/eal_memcfg.h b/lib/librte_eal/common/eal_memcfg.h index 417324aab..573233896 100644 --- a/lib/librte_eal/common/eal_memcfg.h +++ b/lib/librte_eal/common/eal_memcfg.h @@ -69,6 +69,14 @@ struct rte_mem_config { uint8_t dma_maskbits; /**< Keeps the more restricted dma mask. */ }; +/* update internal config from shared mem config */ +void +eal_mcfg_update_internal(void); + +/* update shared mem config from internal config */ +void +eal_mcfg_update_from_internal(void); + /* wait until primary process initialization is complete */ void eal_mcfg_wait_complete(void); diff --git a/lib/librte_eal/freebsd/eal/eal.c b/lib/librte_eal/freebsd/eal/eal.c index 340f7dd6c..ec1650c43 100644 --- a/lib/librte_eal/freebsd/eal/eal.c +++ b/lib/librte_eal/freebsd/eal/eal.c @@ -379,6 +379,7 @@ rte_config_init(void) case RTE_PROC_PRIMARY: if (rte_eal_config_create() < 0) return -1; + eal_mcfg_update_from_internal(); break; case RTE_PROC_SECONDARY: if (rte_eal_config_attach() < 0) @@ -386,6 +387,7 @@ rte_config_init(void) eal_mcfg_wait_complete(); if (rte_eal_config_reattach() < 0) return -1; + eal_mcfg_update_internal(); break; case RTE_PROC_AUTO: case RTE_PROC_INVALID: diff --git a/lib/librte_eal/linux/eal/eal.c b/lib/librte_eal/linux/eal/eal.c index e15eacffc..445d72f0c 100644 --- a/lib/librte_eal/linux/eal/eal.c +++ b/lib/librte_eal/linux/eal/eal.c @@ -473,24 +473,6 @@ eal_proc_type_detect(void) return ptype; } -/* copies data from internal config to shared config */ -static void -eal_update_mem_config(void) -{ - struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; - mcfg->legacy_mem = internal_config.legacy_mem; - mcfg->single_file_segments = internal_config.single_file_segments; -} - -/* copies data from shared config to internal config */ -static void -eal_update_internal_config(void) -{ - struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; - internal_config.legacy_mem = mcfg->legacy_mem; - internal_config.single_file_segments = mcfg->single_file_segments; -} - /* Sets up rte_config structure with the pointer to shared memory config.*/ static int rte_config_init(void) @@ -501,7 +483,7 @@ rte_config_init(void) case RTE_PROC_PRIMARY: if (rte_eal_config_create() < 0) return -1; - eal_update_mem_config(); + eal_mcfg_update_from_internal(); break; case RTE_PROC_SECONDARY: if (rte_eal_config_attach() < 0) @@ -509,7 +491,7 @@ rte_config_init(void) eal_mcfg_wait_complete(); if (rte_eal_config_reattach() < 0) return -1; - eal_update_internal_config(); + eal_mcfg_update_internal(); break; case RTE_PROC_AUTO: case RTE_PROC_INVALID: