Message ID | 20201023080058.13335-1-david.marchand@redhat.com (mailing list archive) |
---|---|
Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id DC4BBA04DE; Fri, 23 Oct 2020 10:01:19 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0D0F28D9A; Fri, 23 Oct 2020 10:01:17 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by dpdk.org (Postfix) with ESMTP id 441186CAA for <dev@dpdk.org>; Fri, 23 Oct 2020 10:01:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603440072; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Ida0lpSfUTG6+nttRkXPnX8vYMs8tC6UC2rKsR3ux7k=; b=e4vD6PNa9rktoqU9om0uPlvwZaFzsHGTn8zB+6iO8p3xkQBEeBHhPH9j280wJShFgSKwkS Rfeot/XzMH8Z1m69EiLApf+XeJT+jBnN/eimTXxSyCpimIAeqXCWrX7Snqb90FcUoEZhVk /At9iW1jLoynFeWqjD41a2rVEIp0ygM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-591-1SznDt3xPOSsGvsEZuhXCg-1; Fri, 23 Oct 2020 04:01:10 -0400 X-MC-Unique: 1SznDt3xPOSsGvsEZuhXCg-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 48A1F64096; Fri, 23 Oct 2020 08:01:09 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.33]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3C91C26330; Fri, 23 Oct 2020 08:01:08 +0000 (UTC) From: David Marchand <david.marchand@redhat.com> To: dev@dpdk.org Cc: jerinj@marvell.com, skori@marvell.com Date: Fri, 23 Oct 2020 10:00:55 +0200 Message-Id: <20201023080058.13335-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="US-ASCII" Subject: [dpdk-dev] [PATCH 0/3] Rework CTF event description storage X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> |
Series |
Rework CTF event description storage
|
|
Message
David Marchand
Oct. 23, 2020, 8 a.m. UTC
Following recent increase of an internal array that was limiting CTF event descriptions, I had a second look at the code. All of this is slow path, so I see no reason in keeping this limitation and we can go with dynamic allocations. While at it, I tweaked the metadata file output. I consider this -rc2 material.
Comments
On Fri, Oct 23, 2020 at 10:01 AM David Marchand <david.marchand@redhat.com> wrote: > > Following recent increase of an internal array that was limiting CTF event > descriptions, I had a second look at the code. > All of this is slow path, so I see no reason in keeping this limitation > and we can go with dynamic allocations. > > While at it, I tweaked the metadata file output. > > I consider this -rc2 material. Comments? Thanks.
On Wed, Oct 28, 2020 at 1:13 AM David Marchand <david.marchand@redhat.com> wrote: > > On Fri, Oct 23, 2020 at 10:01 AM David Marchand > <david.marchand@redhat.com> wrote: > > > > Following recent increase of an internal array that was limiting CTF event > > descriptions, I had a second look at the code. > > All of this is slow path, so I see no reason in keeping this limitation > > and we can go with dynamic allocations. > > > > While at it, I tweaked the metadata file output. > > > > I consider this -rc2 material. > > Comments? > Thanks. The Generated metadata has issues. Please check Reproducer: echo "trace_autotest" | ./build/app/test/dpdk-test -c 0x3 --trace=.* --no-huge --trace=.* [main][dpdk.org] $ babeltrace /home/jerin/dpdk-traces/rte-2020-10-28-PM-02-02-37/ [error] at line 1303: invalid character '0x11' [error] at line 1303: token "": syntax error, unexpected ERROR [error] Error creating AST [warning] Unable to open trace metadata for path "/home/jerin/dpdk-traces/rte-2020-10-28-PM-02-02-37". [warning] [Context] Cannot open_trace of format ctf at path /home/jerin/dpdk-traces/rte-2020-10-28-PM-02-02-37. [warning] [Context] cannot open trace "/home/jerin/dpdk-traces/rte-2020-10-28-PM-02-02-37" from /home/jerin/dpdk-traces/rte-2020-10-28-PM-02-02-37/ for reading. [error] Cannot open any trace for reading. [error] opening trace "/home/jerin/dpdk-traces/rte-2020-10-28-PM-02-02-37/" for reading. [error] none of the specified trace paths could be opened. > > -- > David Marchand >
On Wed, Oct 28, 2020 at 9:53 AM Jerin Jacob <jerinjacobk@gmail.com> wrote: > > On Wed, Oct 28, 2020 at 1:13 AM David Marchand > <david.marchand@redhat.com> wrote: > > > > On Fri, Oct 23, 2020 at 10:01 AM David Marchand > > <david.marchand@redhat.com> wrote: > > > > > > Following recent increase of an internal array that was limiting CTF event > > > descriptions, I had a second look at the code. > > > All of this is slow path, so I see no reason in keeping this limitation > > > and we can go with dynamic allocations. > > > > > > While at it, I tweaked the metadata file output. > > > > > > I consider this -rc2 material. > > > > Comments? > > Thanks. > > The Generated metadata has issues. Please check > > Reproducer: > > echo "trace_autotest" | ./build/app/test/dpdk-test -c 0x3 --trace=.* > --no-huge --trace=.* Err, indeed, thanks for catching. I did some diff on metadata files, but did not notice this trailing character. This is an issue with the metadata string manipulations, that appears with the last patch... I ended up rewriting most of _ctf.c (removing intermediate buffers allocations) and it works but I'll see if I can pinpoint the issue.
On Wed, Oct 28, 2020 at 2:09 PM David Marchand <david.marchand@redhat.com> wrote: > > echo "trace_autotest" | ./build/app/test/dpdk-test -c 0x3 --trace=.* > > --no-huge --trace=.* > > Err, indeed, thanks for catching. > I did some diff on metadata files, but did not notice this trailing character. > > This is an issue with the metadata string manipulations, that appears > with the last patch... > I ended up rewriting most of _ctf.c (removing intermediate buffers > allocations) and it works but I'll see if I can pinpoint the issue. The problem is in the current HEAD, but it is revealed by my series, maybe because the dynamicity around allocations changed. I see no check on the length of trace->ctf_meta when writing to the metadata file. Did I miss something? int rte_trace_metadata_dump(FILE *f) { ... rc = fprintf(f, "%s", ctf_meta); ... } Breakpoint 1, trace_mkdir () at ../lib/librte_eal/common/eal_common_trace_utils.c:317 317 { (gdb) p strlen(trace_obj_get()->ctf_meta) $2 = 21865 (gdb) p trace_obj_get()->ctf_meta[21865] $3 = 0 '\000' (gdb) set trace_obj_get()->ctf_meta[21865] = 'A' ... $ babeltrace $(ls -1rtd $HOME/dpdk-traces/* |tail -1) [error] at line 1008: token "A": syntax error, unexpected IDENTIFIER [error] Error creating AST ... This fixes it: @@ -37,11 +37,12 @@ meta_copy(char **meta, int *offset, char *str, int rc) if (rc < 0) return rc; - ptr = realloc(ptr, count + rc); + ptr = realloc(ptr, count + rc + 1); if (ptr == NULL) goto free_str; memcpy(RTE_PTR_ADD(ptr, count), str, rc); + ptr[count + rc] = '\0'; count += rc; free(str);
On Wed, Oct 28, 2020 at 4:17 PM David Marchand <david.marchand@redhat.com> wrote: > This fixes it: > @@ -37,11 +37,12 @@ meta_copy(char **meta, int *offset, char *str, int rc) > if (rc < 0) > return rc; > > - ptr = realloc(ptr, count + rc); > + ptr = realloc(ptr, count + rc + 1); > if (ptr == NULL) > goto free_str; > > memcpy(RTE_PTR_ADD(ptr, count), str, rc); > + ptr[count + rc] = '\0'; > count += rc; > free(str); > The other alternative is to prefer libc string formatting functions rather than plain memory alloc + copy + manual null termination: https://github.com/david-marchand/dpdk/commit/traces