diff mbox series

[2/2] malloc: fix realloc padded element size

Message ID 1573570228-15676-2-git-send-email-xuemingl@mellanox.com (mailing list archive)
State Accepted, archived
Delegated to: David Marchand
Headers
Series [1/2] malloc: fix realloc wrong copy size |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/travis-robot success Travis build: passed
ci/Intel-compilation success Compilation OK

Commit Message

Xueming Li Nov. 12, 2019, 2:50 p.m. UTC 
  When resize a memory with next element, the original element size grows.
If the orginal element has padding, the real inner element size didn't
grow as well and this causes trailer verification failure when malloc
debug enabled.

Fixes: af75078fece3 ("first public release")
Cc: stable@dpdk.org

Signed-off-by: Xueming Li <xuemingl@mellanox.com>
---
 lib/librte_eal/common/malloc_elem.c | 5 +++++
 1 file changed, 5 insertions(+)

Comments

Burakov, Anatoly Nov. 14, 2019, 3:11 p.m. UTC | #1 
On 12-Nov-19 2:50 PM, Xueming Li wrote:
> When resize a memory with next element, the original element size grows.
> If the orginal element has padding, the real inner element size didn't
> grow as well and this causes trailer verification failure when malloc
> debug enabled.
> 
> Fixes: af75078fece3 ("first public release")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Xueming Li <xuemingl@mellanox.com>
> ---

Reviewed-by: Anatoly Burakov <anatoly.burakov@intel.com>
David Marchand Nov. 19, 2019, 8:47 p.m. UTC | #2 
On Tue, Nov 12, 2019 at 3:50 PM Xueming Li <xuemingl@mellanox.com> wrote:
>
> When resize a memory with next element, the original element size grows.
> If the orginal element has padding, the real inner element size didn't
> grow as well and this causes trailer verification failure when malloc
> debug enabled.

I did not see this when running the malloc_autotest with debug enabled.
What is missing for me to catch it?


Just a bit chilly to apply this series.
The first patch seems an optimisation.
The second one seems more interesting if we fix the debug mode, but I
suppose we can live without them in 19.11.
Xueming Li Nov. 20, 2019, 2:12 a.m. UTC | #3 
> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Wednesday, November 20, 2019 4:47 AM
> To: Xueming(Steven) Li <xuemingl@mellanox.com>; Anatoly Burakov
> <anatoly.burakov@intel.com>
> Cc: Asaf Penso <asafp@mellanox.com>; dev <dev@dpdk.org>; dpdk stable
> <stable@dpdk.org>
> Subject: Re: [dpdk-stable] [PATCH 2/2] malloc: fix realloc padded element
> size
> 
> On Tue, Nov 12, 2019 at 3:50 PM Xueming Li <xuemingl@mellanox.com>
> wrote:
> >
> > When resize a memory with next element, the original element size grows.
> > If the orginal element has padding, the real inner element size didn't
> > grow as well and this causes trailer verification failure when malloc
> > debug enabled.
> 
> I did not see this when running the malloc_autotest with debug enabled.
> What is missing for me to catch it?
> 
Yes, it happens rarely, depends on memory fragment. I only caught this in middle of a long test.

> 
> Just a bit chilly to apply this series.
> The first patch seems an optimisation.
> The second one seems more interesting if we fix the debug mode, but I
> suppose we can live without them in 19.11.
Few people enable memory debug option, they are there for years.

> 
> 
> --
> David Marchand
David Marchand Nov. 20, 2019, 1:25 p.m. UTC | #4 
On Wed, Nov 20, 2019 at 3:12 AM Xueming(Steven) Li
<xuemingl@mellanox.com> wrote:
>
> > -----Original Message-----
> > From: David Marchand <david.marchand@redhat.com>
> > Sent: Wednesday, November 20, 2019 4:47 AM
> > To: Xueming(Steven) Li <xuemingl@mellanox.com>; Anatoly Burakov
> > <anatoly.burakov@intel.com>
> > Cc: Asaf Penso <asafp@mellanox.com>; dev <dev@dpdk.org>; dpdk stable
> > <stable@dpdk.org>
> > Subject: Re: [dpdk-stable] [PATCH 2/2] malloc: fix realloc padded element
> > size
> >
> > On Tue, Nov 12, 2019 at 3:50 PM Xueming Li <xuemingl@mellanox.com>
> > wrote:
> > >
> > > When resize a memory with next element, the original element size grows.
> > > If the orginal element has padding, the real inner element size didn't
> > > grow as well and this causes trailer verification failure when malloc
> > > debug enabled.
> >
> > I did not see this when running the malloc_autotest with debug enabled.
> > What is missing for me to catch it?
> >
> Yes, it happens rarely, depends on memory fragment. I only caught this in middle of a long test.
>
> >
> > Just a bit chilly to apply this series.
> > The first patch seems an optimisation.
> > The second one seems more interesting if we fix the debug mode, but I
> > suppose we can live without them in 19.11.
> Few people enable memory debug option, they are there for years.

Had a discussion offlist with Anatoly.
Those two issues are hard to catch but the fixes are relevant and
Anatoly is confident.
I will take this in rc3.

Series applied, thanks.
Burakov, Anatoly Nov. 21, 2019, 12:30 p.m. UTC | #5 
On 20-Nov-19 1:25 PM, David Marchand wrote:
> On Wed, Nov 20, 2019 at 3:12 AM Xueming(Steven) Li
> <xuemingl@mellanox.com> wrote:
>>
>>> -----Original Message-----
>>> From: David Marchand <david.marchand@redhat.com>
>>> Sent: Wednesday, November 20, 2019 4:47 AM
>>> To: Xueming(Steven) Li <xuemingl@mellanox.com>; Anatoly Burakov
>>> <anatoly.burakov@intel.com>
>>> Cc: Asaf Penso <asafp@mellanox.com>; dev <dev@dpdk.org>; dpdk stable
>>> <stable@dpdk.org>
>>> Subject: Re: [dpdk-stable] [PATCH 2/2] malloc: fix realloc padded element
>>> size
>>>
>>> On Tue, Nov 12, 2019 at 3:50 PM Xueming Li <xuemingl@mellanox.com>
>>> wrote:
>>>>
>>>> When resize a memory with next element, the original element size grows.
>>>> If the orginal element has padding, the real inner element size didn't
>>>> grow as well and this causes trailer verification failure when malloc
>>>> debug enabled.
>>>
>>> I did not see this when running the malloc_autotest with debug enabled.
>>> What is missing for me to catch it?
>>>
>> Yes, it happens rarely, depends on memory fragment. I only caught this in middle of a long test.
>>
>>>
>>> Just a bit chilly to apply this series.
>>> The first patch seems an optimisation.
>>> The second one seems more interesting if we fix the debug mode, but I
>>> suppose we can live without them in 19.11.
>> Few people enable memory debug option, they are there for years.
> 
> Had a discussion offlist with Anatoly.
> Those two issues are hard to catch but the fixes are relevant and
> Anatoly is confident.
> I will take this in rc3.
> 
> Series applied, thanks.
> 

To test them, i had to modify malloc to always create padded elements :)
Xueming Li Nov. 21, 2019, 12:55 p.m. UTC | #6 
Hi Anatoly,

> -----Original Message-----
> From: Burakov, Anatoly <anatoly.burakov@intel.com>
> Sent: Thursday, November 21, 2019 8:30 PM
> To: David Marchand <david.marchand@redhat.com>; Xueming(Steven) Li
> <xuemingl@mellanox.com>
> Cc: Asaf Penso <asafp@mellanox.com>; dev <dev@dpdk.org>; dpdk stable
> <stable@dpdk.org>
> Subject: Re: [dpdk-stable] [PATCH 2/2] malloc: fix realloc padded element size
> 
> On 20-Nov-19 1:25 PM, David Marchand wrote:
> > On Wed, Nov 20, 2019 at 3:12 AM Xueming(Steven) Li
> > <xuemingl@mellanox.com> wrote:
> >>
> >>> -----Original Message-----
> >>> From: David Marchand <david.marchand@redhat.com>
> >>> Sent: Wednesday, November 20, 2019 4:47 AM
> >>> To: Xueming(Steven) Li <xuemingl@mellanox.com>; Anatoly Burakov
> >>> <anatoly.burakov@intel.com>
> >>> Cc: Asaf Penso <asafp@mellanox.com>; dev <dev@dpdk.org>; dpdk stable
> >>> <stable@dpdk.org>
> >>> Subject: Re: [dpdk-stable] [PATCH 2/2] malloc: fix realloc padded
> >>> element size
> >>>
> >>> On Tue, Nov 12, 2019 at 3:50 PM Xueming Li <xuemingl@mellanox.com>
> >>> wrote:
> >>>>
> >>>> When resize a memory with next element, the original element size grows.
> >>>> If the orginal element has padding, the real inner element size
> >>>> didn't grow as well and this causes trailer verification failure
> >>>> when malloc debug enabled.
> >>>
> >>> I did not see this when running the malloc_autotest with debug enabled.
> >>> What is missing for me to catch it?
> >>>
> >> Yes, it happens rarely, depends on memory fragment. I only caught this in
> middle of a long test.
> >>
> >>>
> >>> Just a bit chilly to apply this series.
> >>> The first patch seems an optimisation.
> >>> The second one seems more interesting if we fix the debug mode, but
> >>> I suppose we can live without them in 19.11.
> >> Few people enable memory debug option, they are there for years.
> >
> > Had a discussion offlist with Anatoly.
> > Those two issues are hard to catch but the fixes are relevant and
> > Anatoly is confident.
> > I will take this in rc3.
> >
> > Series applied, thanks.
> >
> 
> To test them, i had to modify malloc to always create padded elements :)

I fix another issue in element join, as I made some local enhancement patch on memory, it's required, not sure whether it help on public code.

diff --git a/lib/librte_eal/common/malloc_elem.c b/lib/librte_eal/common/malloc_elem.c
index afacb1813c..c3fa0d1039 100644
--- a/lib/librte_eal/common/malloc_elem.c
+++ b/lib/librte_eal/common/malloc_elem.c
@@ -487,6 +487,10 @@ join_elem(struct malloc_elem *elem1, struct malloc_elem *elem2)
        else
                elem1->heap->last = elem1;
        elem1->next = next;
+       if (elem1->pad) {
+               struct malloc_elem *inner = RTE_PTR_ADD(elem1, elem1->pad);
+               inner->size += elem2->size;
+       }
 }



> 
> --
> Thanks,
> Anatoly
Burakov, Anatoly Nov. 21, 2019, 1:15 p.m. UTC | #7 
On 21-Nov-19 12:55 PM, Xueming(Steven) Li wrote:
> Hi Anatoly,
> 
>> -----Original Message-----
>> From: Burakov, Anatoly <anatoly.burakov@intel.com>
>> Sent: Thursday, November 21, 2019 8:30 PM
>> To: David Marchand <david.marchand@redhat.com>; Xueming(Steven) Li
>> <xuemingl@mellanox.com>
>> Cc: Asaf Penso <asafp@mellanox.com>; dev <dev@dpdk.org>; dpdk stable
>> <stable@dpdk.org>
>> Subject: Re: [dpdk-stable] [PATCH 2/2] malloc: fix realloc padded element size
>>
>> On 20-Nov-19 1:25 PM, David Marchand wrote:
>>> On Wed, Nov 20, 2019 at 3:12 AM Xueming(Steven) Li
>>> <xuemingl@mellanox.com> wrote:
>>>>
>>>>> -----Original Message-----
>>>>> From: David Marchand <david.marchand@redhat.com>
>>>>> Sent: Wednesday, November 20, 2019 4:47 AM
>>>>> To: Xueming(Steven) Li <xuemingl@mellanox.com>; Anatoly Burakov
>>>>> <anatoly.burakov@intel.com>
>>>>> Cc: Asaf Penso <asafp@mellanox.com>; dev <dev@dpdk.org>; dpdk stable
>>>>> <stable@dpdk.org>
>>>>> Subject: Re: [dpdk-stable] [PATCH 2/2] malloc: fix realloc padded
>>>>> element size
>>>>>
>>>>> On Tue, Nov 12, 2019 at 3:50 PM Xueming Li <xuemingl@mellanox.com>
>>>>> wrote:
>>>>>>
>>>>>> When resize a memory with next element, the original element size grows.
>>>>>> If the orginal element has padding, the real inner element size
>>>>>> didn't grow as well and this causes trailer verification failure
>>>>>> when malloc debug enabled.
>>>>>
>>>>> I did not see this when running the malloc_autotest with debug enabled.
>>>>> What is missing for me to catch it?
>>>>>
>>>> Yes, it happens rarely, depends on memory fragment. I only caught this in
>> middle of a long test.
>>>>
>>>>>
>>>>> Just a bit chilly to apply this series.
>>>>> The first patch seems an optimisation.
>>>>> The second one seems more interesting if we fix the debug mode, but
>>>>> I suppose we can live without them in 19.11.
>>>> Few people enable memory debug option, they are there for years.
>>>
>>> Had a discussion offlist with Anatoly.
>>> Those two issues are hard to catch but the fixes are relevant and
>>> Anatoly is confident.
>>> I will take this in rc3.
>>>
>>> Series applied, thanks.
>>>
>>
>> To test them, i had to modify malloc to always create padded elements :)
> 
> I fix another issue in element join, as I made some local enhancement patch on memory, it's required, not sure whether it help on public code.
> 
> diff --git a/lib/librte_eal/common/malloc_elem.c b/lib/librte_eal/common/malloc_elem.c
> index afacb1813c..c3fa0d1039 100644
> --- a/lib/librte_eal/common/malloc_elem.c
> +++ b/lib/librte_eal/common/malloc_elem.c
> @@ -487,6 +487,10 @@ join_elem(struct malloc_elem *elem1, struct malloc_elem *elem2)
>          else
>                  elem1->heap->last = elem1;
>          elem1->next = next;
> +       if (elem1->pad) {
> +               struct malloc_elem *inner = RTE_PTR_ADD(elem1, elem1->pad);
> +               inner->size += elem2->size;
> +       }
>   }
> 
> 

This looks like a good change as well - while we /mostly/ join elements 
when they're free, there is one case where we can join a free element 
with one already occupied - and /that/ element could be padded, which we 
currently don't update. So, this patch would help the public code as well.

> 
>>
>> --
>> Thanks,
>> Anatoly
Burakov, Anatoly Nov. 21, 2019, 1:17 p.m. UTC | #8 
On 21-Nov-19 1:15 PM, Burakov, Anatoly wrote:
> On 21-Nov-19 12:55 PM, Xueming(Steven) Li wrote:
>> Hi Anatoly,
>>
>>> -----Original Message-----
>>> From: Burakov, Anatoly <anatoly.burakov@intel.com>
>>> Sent: Thursday, November 21, 2019 8:30 PM
>>> To: David Marchand <david.marchand@redhat.com>; Xueming(Steven) Li
>>> <xuemingl@mellanox.com>
>>> Cc: Asaf Penso <asafp@mellanox.com>; dev <dev@dpdk.org>; dpdk stable
>>> <stable@dpdk.org>
>>> Subject: Re: [dpdk-stable] [PATCH 2/2] malloc: fix realloc padded 
>>> element size
>>>
>>> On 20-Nov-19 1:25 PM, David Marchand wrote:
>>>> On Wed, Nov 20, 2019 at 3:12 AM Xueming(Steven) Li
>>>> <xuemingl@mellanox.com> wrote:
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: David Marchand <david.marchand@redhat.com>
>>>>>> Sent: Wednesday, November 20, 2019 4:47 AM
>>>>>> To: Xueming(Steven) Li <xuemingl@mellanox.com>; Anatoly Burakov
>>>>>> <anatoly.burakov@intel.com>
>>>>>> Cc: Asaf Penso <asafp@mellanox.com>; dev <dev@dpdk.org>; dpdk stable
>>>>>> <stable@dpdk.org>
>>>>>> Subject: Re: [dpdk-stable] [PATCH 2/2] malloc: fix realloc padded
>>>>>> element size
>>>>>>
>>>>>> On Tue, Nov 12, 2019 at 3:50 PM Xueming Li <xuemingl@mellanox.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> When resize a memory with next element, the original element size 
>>>>>>> grows.
>>>>>>> If the orginal element has padding, the real inner element size
>>>>>>> didn't grow as well and this causes trailer verification failure
>>>>>>> when malloc debug enabled.
>>>>>>
>>>>>> I did not see this when running the malloc_autotest with debug 
>>>>>> enabled.
>>>>>> What is missing for me to catch it?
>>>>>>
>>>>> Yes, it happens rarely, depends on memory fragment. I only caught 
>>>>> this in
>>> middle of a long test.
>>>>>
>>>>>>
>>>>>> Just a bit chilly to apply this series.
>>>>>> The first patch seems an optimisation.
>>>>>> The second one seems more interesting if we fix the debug mode, but
>>>>>> I suppose we can live without them in 19.11.
>>>>> Few people enable memory debug option, they are there for years.
>>>>
>>>> Had a discussion offlist with Anatoly.
>>>> Those two issues are hard to catch but the fixes are relevant and
>>>> Anatoly is confident.
>>>> I will take this in rc3.
>>>>
>>>> Series applied, thanks.
>>>>
>>>
>>> To test them, i had to modify malloc to always create padded elements :)
>>
>> I fix another issue in element join, as I made some local enhancement 
>> patch on memory, it's required, not sure whether it help on public code.
>>
>> diff --git a/lib/librte_eal/common/malloc_elem.c 
>> b/lib/librte_eal/common/malloc_elem.c
>> index afacb1813c..c3fa0d1039 100644
>> --- a/lib/librte_eal/common/malloc_elem.c
>> +++ b/lib/librte_eal/common/malloc_elem.c
>> @@ -487,6 +487,10 @@ join_elem(struct malloc_elem *elem1, struct 
>> malloc_elem *elem2)
>>          else
>>                  elem1->heap->last = elem1;
>>          elem1->next = next;
>> +       if (elem1->pad) {
>> +               struct malloc_elem *inner = RTE_PTR_ADD(elem1, 
>> elem1->pad);
>> +               inner->size += elem2->size;
>> +       }
>>   }
>>
>>
> 
> This looks like a good change as well - while we /mostly/ join elements 
> when they're free, there is one case where we can join a free element 
> with one already occupied - and /that/ element could be padded, which we 
> currently don't update. So, this patch would help the public code as well.

(that said, the += should be replaced with explicit calculation of 
correct size, as this joining may be run multiple times, and we don't 
want incorrect size accumulating...)

> 
>>
>>>
>>> -- 
>>> Thanks,
>>> Anatoly
> 
>
Xueming Li Nov. 21, 2019, 2:11 p.m. UTC | #9 
> -----Original Message-----
> From: Burakov, Anatoly <anatoly.burakov@intel.com>
> Sent: Thursday, November 21, 2019 9:17 PM
> To: Xueming(Steven) Li <xuemingl@mellanox.com>; David Marchand
> <david.marchand@redhat.com>
> Cc: Asaf Penso <asafp@mellanox.com>; dev <dev@dpdk.org>; dpdk stable
> <stable@dpdk.org>
> Subject: Re: [dpdk-dev] [dpdk-stable] [PATCH 2/2] malloc: fix realloc padded
> element size
> 
> On 21-Nov-19 1:15 PM, Burakov, Anatoly wrote:
> > On 21-Nov-19 12:55 PM, Xueming(Steven) Li wrote:
> >> Hi Anatoly,
> >>
> >>> -----Original Message-----
> >>> From: Burakov, Anatoly <anatoly.burakov@intel.com>
> >>> Sent: Thursday, November 21, 2019 8:30 PM
> >>> To: David Marchand <david.marchand@redhat.com>; Xueming(Steven)
> Li
> >>> <xuemingl@mellanox.com>
> >>> Cc: Asaf Penso <asafp@mellanox.com>; dev <dev@dpdk.org>; dpdk
> stable
> >>> <stable@dpdk.org>
> >>> Subject: Re: [dpdk-stable] [PATCH 2/2] malloc: fix realloc padded
> >>> element size
> >>>
> >>> On 20-Nov-19 1:25 PM, David Marchand wrote:
> >>>> On Wed, Nov 20, 2019 at 3:12 AM Xueming(Steven) Li
> >>>> <xuemingl@mellanox.com> wrote:
> >>>>>
> >>>>>> -----Original Message-----
> >>>>>> From: David Marchand <david.marchand@redhat.com>
> >>>>>> Sent: Wednesday, November 20, 2019 4:47 AM
> >>>>>> To: Xueming(Steven) Li <xuemingl@mellanox.com>; Anatoly
> Burakov
> >>>>>> <anatoly.burakov@intel.com>
> >>>>>> Cc: Asaf Penso <asafp@mellanox.com>; dev <dev@dpdk.org>; dpdk
> >>>>>> stable <stable@dpdk.org>
> >>>>>> Subject: Re: [dpdk-stable] [PATCH 2/2] malloc: fix realloc padded
> >>>>>> element size
> >>>>>>
> >>>>>> On Tue, Nov 12, 2019 at 3:50 PM Xueming Li
> >>>>>> <xuemingl@mellanox.com>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>> When resize a memory with next element, the original element
> >>>>>>> size grows.
> >>>>>>> If the orginal element has padding, the real inner element size
> >>>>>>> didn't grow as well and this causes trailer verification failure
> >>>>>>> when malloc debug enabled.
> >>>>>>
> >>>>>> I did not see this when running the malloc_autotest with debug
> >>>>>> enabled.
> >>>>>> What is missing for me to catch it?
> >>>>>>
> >>>>> Yes, it happens rarely, depends on memory fragment. I only caught
> >>>>> this in
> >>> middle of a long test.
> >>>>>
> >>>>>>
> >>>>>> Just a bit chilly to apply this series.
> >>>>>> The first patch seems an optimisation.
> >>>>>> The second one seems more interesting if we fix the debug mode,
> >>>>>> but I suppose we can live without them in 19.11.
> >>>>> Few people enable memory debug option, they are there for years.
> >>>>
> >>>> Had a discussion offlist with Anatoly.
> >>>> Those two issues are hard to catch but the fixes are relevant and
> >>>> Anatoly is confident.
> >>>> I will take this in rc3.
> >>>>
> >>>> Series applied, thanks.
> >>>>
> >>>
> >>> To test them, i had to modify malloc to always create padded
> >>> elements :)
> >>
> >> I fix another issue in element join, as I made some local enhancement
> >> patch on memory, it's required, not sure whether it help on public code.
> >>
> >> diff --git a/lib/librte_eal/common/malloc_elem.c
> >> b/lib/librte_eal/common/malloc_elem.c
> >> index afacb1813c..c3fa0d1039 100644
> >> --- a/lib/librte_eal/common/malloc_elem.c
> >> +++ b/lib/librte_eal/common/malloc_elem.c
> >> @@ -487,6 +487,10 @@ join_elem(struct malloc_elem *elem1, struct
> >> malloc_elem *elem2)
> >>          else
> >>                  elem1->heap->last = elem1;
> >>          elem1->next = next;
> >> +       if (elem1->pad) {
> >> +               struct malloc_elem *inner = RTE_PTR_ADD(elem1,
> >> elem1->pad);
> >> +               inner->size += elem2->size;
> >> +       }
> >>   }
> >>
> >>
> >
> > This looks like a good change as well - while we /mostly/ join
> > elements when they're free, there is one case where we can join a free
> > element with one already occupied - and /that/ element could be
> > padded, which we currently don't update. So, this patch would help the
> public code as well.

Correct, it happens in rte_realloc.

> 
> (that said, the += should be replaced with explicit calculation of correct size,
> as this joining may be run multiple times, and we don't want incorrect size
> accumulating...)

I'll update and send a new patch.

> 
> >
> >>
> >>>
> >>> --
> >>> Thanks,
> >>> Anatoly
> >
> >
> 
> 
> --
> Thanks,
> Anatoly
diff mbox series

Patch

diff --git a/lib/librte_eal/common/malloc_elem.c b/lib/librte_eal/common/malloc_elem.c
index 658c9b5b79..afacb1813c 100644
--- a/lib/librte_eal/common/malloc_elem.c
+++ b/lib/librte_eal/common/malloc_elem.c
@@ -307,6 +307,11 @@  split_elem(struct malloc_elem *elem, struct malloc_elem *split_pt)
 	elem->next = split_pt;
 	elem->size = old_elem_size;
 	set_trailer(elem);
+	if (elem->pad) {
+		/* Update inner padding inner element size. */
+		elem = RTE_PTR_ADD(elem, elem->pad);
+		elem->size = old_elem_size - elem->pad;
+	}
 }
 
 /*