Cover Detail
Show a cover letter.
GET /api/covers/48238/?format=api
{ "id": 48238, "url": "http://patchwork.dpdk.org/api/covers/48238/?format=api", "web_url": "http://patchwork.dpdk.org/project/dpdk/cover/20181122033055.3431-1-honnappa.nagarahalli@arm.com/", "project": { "id": 1, "url": "http://patchwork.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<20181122033055.3431-1-honnappa.nagarahalli@arm.com>", "list_archive_url": "https://inbox.dpdk.org/dev/20181122033055.3431-1-honnappa.nagarahalli@arm.com", "date": "2018-11-22T03:30:52", "name": "[RFC,0/3] tqs: add thread quiescent state library", "submitter": { "id": 1045, "url": "http://patchwork.dpdk.org/api/people/1045/?format=api", "name": "Honnappa Nagarahalli", "email": "honnappa.nagarahalli@arm.com" }, "mbox": "http://patchwork.dpdk.org/project/dpdk/cover/20181122033055.3431-1-honnappa.nagarahalli@arm.com/mbox/", "series": [ { "id": 2530, "url": "http://patchwork.dpdk.org/api/series/2530/?format=api", "web_url": "http://patchwork.dpdk.org/project/dpdk/list/?series=2530", "date": "2018-11-22T03:30:52", "name": "tqs: add thread quiescent state library", "version": 1, "mbox": "http://patchwork.dpdk.org/series/2530/mbox/" } ], "comments": "http://patchwork.dpdk.org/api/covers/48238/comments/", "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@dpdk.org", "Delivered-To": "patchwork@dpdk.org", "Received": [ "from [92.243.14.124] (localhost [127.0.0.1])\n\tby dpdk.org (Postfix) with ESMTP id 211EB1B1F0;\n\tThu, 22 Nov 2018 04:31:45 +0100 (CET)", "from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com [217.140.101.70])\n\tby dpdk.org (Postfix) with ESMTP id A89961B1EE\n\tfor <dev@dpdk.org>; Thu, 22 Nov 2018 04:31:43 +0100 (CET)", "from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D121E3617;\n\tWed, 21 Nov 2018 19:31:42 -0800 (PST)", "from qc2400f-1.austin.arm.com (qc2400f-1.austin.arm.com\n\t[10.118.14.34])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\t798FA3F5CF; Wed, 21 Nov 2018 19:31:42 -0800 (PST)" ], "From": "Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>", "To": "dev@dpdk.org", "Cc": "nd@arm.com, honnappa.nagarahalli@arm.com, dharmik.thakkar@arm.com,\n\tmalvika.gupta@arm.com, gavin.hu@arm.com", "Date": "Wed, 21 Nov 2018 21:30:52 -0600", "Message-Id": "<20181122033055.3431-1-honnappa.nagarahalli@arm.com>", "X-Mailer": "git-send-email 2.17.1", "Subject": "[dpdk-dev] [RFC 0/3] tqs: add thread quiescent state library", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n\t<mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n\t<mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "content": "Lock-less data structures provide scalability and determinism.\nThey enable use cases where locking may not be allowed\n(for ex: real-time applications).\n\nIn the following paras, the term 'memory' refers to memory allocated\nby typical APIs like malloc or anything that is representative of\nmemory, for ex: an index of a free element array.\n\nSince these data structures are lock less, the writers and readers\nare accessing the data structures simultaneously. Hence, while removing\nan element from a data structure, the writers cannot return the memory\nto the allocator, without knowing that the readers are not\nreferencing that element/memory anymore. Hence, it is required to\nseparate the operation of removing an element into 2 steps:\n\nDelete: in this step, the writer removes the element from the\ndata structure but does not return the associated memory to the allocator.\nThis will ensure that new readers will not get a reference to the removed\nelement. Removing the reference is an atomic operation.\n\nFree: in this step, the writer returns the memory to the\nmemory allocator, only after knowing that all the readers have stopped\nreferencing the removed element.\n\nThis library helps the writer determine when it is safe to free the\nmemory.\n\nThis library makes use of Thread Quiescent State (TQS). TQS can be\ndefined as 'any point in the thread execution where the thread does\nnot hold a reference to shared memory'. It is upto the application to\ndetermine its quiescent state. Let us consider the following diagram:\n\n Time -------------------------------------------------->\n\n | |\n RT1 $++++****D1****+++***D2*|**+++|+++**D3*****++++$\n | |\n RT2 $++++****D1****++|+**D2|***++++++**D3*****++++$\n | |\n RT3 $++++****D1****+++***|D2***|++++++**D2*****++++$\n | |\n |<--->|\n Del | Free\n |\n Cannot free memory\n during this period\n\nRTx - Reader thread\n< and > - Start and end of while(1) loop\n***Dx*** - Reader thread is accessing the shared data structure Dx.\n i.e. critical section.\n+++ - Reader thread is not accessing any shared data structure.\n i.e. non critical section or quiescent state.\nDel - Point in time when the reference to the entry is removed using\n atomic operation.\nFree - Point in time when the writer can free the entry.\n\nAs shown thread RT1 acesses data structures D1, D2 and D3. When it is\naccessing D2, if the writer has to remove an element from D2, the\nwriter cannot return the memory associated with that element to the\nallocator. The writer can return the memory to the allocator only after\nthe reader stops referencng D2. In other words, reader thread RT1\nhas to enter a quiescent state.\n\nSimilarly, since thread RT3 is also accessing D2, writer has to wait till\nRT3 enters quiescent state as well.\n\nHowever, the writer does not need to wait for RT2 to enter quiescent state.\nThread RT2 was not accessing D2 when the delete operation happened.\nSo, RT2 will not get a reference to the deleted entry.\n\nIt can be noted that, the critical sections for D2 and D3 are quiescent states\nfor D1. i.e. for a given data structure Dx, any point in the thread execution\nthat does not reference Dx is a quiescent state.\n\nFor DPDK applications, the start and end of while(1) loop (where no shared\ndata structures are getting accessed) act as perfect quiescent states. This\nwill combine all the shared data structure accesses into a single critical\nsection and keeps the over head introduced by this library to the minimum.\n\nHowever, the length of the critical section and the number of reader threads\nis proportional to the time taken to identify the end of critical section.\nSo, if the application desires, it should be possible to identify the end\nof critical section for each data structure.\n\nTo provide the required flexibility, this library has a concept of TQS\nvariable. The application can create one or more TQS variables to help it\ntrack the end of one or more critical sections.\n\nThe application can create a TQS variable using the API rte_tqs_alloc.\nIt takes a mask of lcore IDs that will report their quiescent states\nusing this variable. This mask can be empty to start with.\n\nrte_tqs_register_lcore API will register a reader thread to report its\nquiescent state. This can be called from any control plane thread or from\nthe reader thread. The application can create a TQS variable with no reader\nthreads and add the threads dynamically using this API.\n\nThe application can trigger the reader threads to report their quiescent\nstate status by calling the API rte_tqs_start. It is possible for multiple\nwriter threads to query the quiescent state status simultaneously. Hence,\nrte_tqs_start returns a token to each caller.\n\nThe application has to call rte_tqs_check API with the token to get the\ncurrent status. Option to block till all the threads enter the quiescent\nstate is provided. If this API indicates that all the threads have entered\nthe quiescent state, the application can free the deleted entry.\n\nThe separation of triggering the reporting from querying the status provides\nthe writer threads flexibility to do useful work instead of waiting for the\nreader threads to enter the quiescent state.\n\nrte_tqs_unregister_lcore API will remove a reader thread from reporting its\nquiescent state using a TQS variable. The rte_tqs_check API will not wait\nfor this reader thread to report the quiescent state status anymore.\n\nFinally, a TQS variable can be deleted by calling rte_tqs_free API.\nApplication must make sure that the reader threads are not referencing the\nTQS variable anymore before deleting it.\n\nThe reader threads should call rte_tqs_update API to indicate that they\nentered a quiescent state. This API checks if a writer has triggered a\nquiescent state query and update the state accordingly.\n\nNext Steps:\n1) Add more test cases\n2) Convert to patch\n3) Incorporate feedback from community\n4) Add documentation\n\nDharmik Thakkar (1):\n test/tqs: Add API and functional tests\n\nHonnappa Nagarahalli (2):\n log: add TQS log type\n tqs: add thread quiescent state library\n\n config/common_base | 6 +\n lib/Makefile | 2 +\n lib/librte_eal/common/include/rte_log.h | 1 +\n lib/librte_tqs/Makefile | 23 +\n lib/librte_tqs/meson.build | 5 +\n lib/librte_tqs/rte_tqs.c | 249 +++++++++++\n lib/librte_tqs/rte_tqs.h | 352 +++++++++++++++\n lib/librte_tqs/rte_tqs_version.map | 16 +\n lib/meson.build | 2 +-\n mk/rte.app.mk | 1 +\n test/test/Makefile | 2 +\n test/test/autotest_data.py | 6 +\n test/test/meson.build | 5 +-\n test/test/test_tqs.c | 540 ++++++++++++++++++++++++\n 14 files changed, 1208 insertions(+), 2 deletions(-)\n create mode 100644 lib/librte_tqs/Makefile\n create mode 100644 lib/librte_tqs/meson.build\n create mode 100644 lib/librte_tqs/rte_tqs.c\n create mode 100644 lib/librte_tqs/rte_tqs.h\n create mode 100644 lib/librte_tqs/rte_tqs_version.map\n create mode 100644 test/test/test_tqs.c" }