diff mbox series

[v2,1/1] fbarray: get fbarrays from containerized secondary

Message ID 1555386203-23776-2-git-send-email-ogawa.yasufumi@lab.ntt.co.jp (mailing list archive)
State Superseded, archived
Delegated to: Thomas Monjalon
Headers
Series Get fbarrays from containerized secondary |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/Intel-compilation success Compilation OK
ci/intel-Performance-Testing success Performance Testing PASS
ci/mellanox-Performance-Testing success Performance Testing PASS

Commit Message

Yasufumi Ogawa April 16, 2019, 3:43 a.m. UTC 
  From: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>

In secondary_msl_create_walk(), it creates a file for fbarrays with its
PID for reserving unique name among secondary processes. However, it
does not work if secondary is run as app container because each of
containerized secondary has PID 1. To reserve unique name, use hostname
instead of PID if the value is 1.

Cc: stable@dpdk.org

Signed-off-by: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
---
 lib/librte_eal/linux/eal/eal_memalloc.c | 25 +++++++++++++++++++++++--
 1 file changed, 23 insertions(+), 2 deletions(-)

Comments

Thomas Monjalon July 4, 2019, 8:17 p.m. UTC | #1 
We need a review here.

16/04/2019 05:43, ogawa.yasufumi@lab.ntt.co.jp:
> From: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
> 
> In secondary_msl_create_walk(), it creates a file for fbarrays with its
> PID for reserving unique name among secondary processes. However, it
> does not work if secondary is run as app container because each of
> containerized secondary has PID 1. To reserve unique name, use hostname
> instead of PID if the value is 1.
> 
> Cc: stable@dpdk.org
> 
> Signed-off-by: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
Burakov, Anatoly July 5, 2019, 8:53 a.m. UTC | #2 
On 16-Apr-19 4:43 AM, ogawa.yasufumi@lab.ntt.co.jp wrote:
> From: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
> 
> In secondary_msl_create_walk(), it creates a file for fbarrays with its
> PID for reserving unique name among secondary processes. However, it
> does not work if secondary is run as app container because each of
> containerized secondary has PID 1. To reserve unique name, use hostname
> instead of PID if the value is 1.
> 
> Cc: stable@dpdk.org
> 
> Signed-off-by: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
> ---

I'm not too well versed in containers - is this hostname 1) always set, 
and 2) always unique?

<snip>

> +	if (getpid() == 1) {
> +		FILE *hn_fp;
> +		hn_fp = fopen("/etc/hostname", "r");
> +		if (hn_fp == NULL) {
> +			RTE_LOG(ERR, EAL,
> +				"Cannot open '/etc/hostname' for secondary\n");
> +			return -1;
> +		}
> +
> +		/* with docker, /etc/hostname just has one entry of hostname */
> +		if (fscanf(hn_fp, "%s", proc_id) == EOF)
> +			return -1;

Wouldn't an error in fscanf() leak the file handle? I think you need to 
fclose() before checking the result.

> +		fclose(hn_fp);
> +	} else
> +		sprintf(proc_id, "%d", (int)getpid());
> +
> +	snprintf(name, RTE_FBARRAY_NAME_LEN, "%s_%s",
> +			primary_msl->memseg_arr.name, proc_id);
>   
>   	ret = rte_fbarray_init(&local_msl->memseg_arr, name,
>   		primary_msl->memseg_arr.len,
>
Yasufumi Ogawa July 9, 2019, 10:22 a.m. UTC | #3 
Hi Anatoly,

On 2019/07/05 17:53, Burakov, Anatoly wrote:
> On 16-Apr-19 4:43 AM, ogawa.yasufumi@lab.ntt.co.jp wrote:
>> From: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
>>
>> In secondary_msl_create_walk(), it creates a file for fbarrays with its
>> PID for reserving unique name among secondary processes. However, it
>> does not work if secondary is run as app container because each of
>> containerized secondary has PID 1. To reserve unique name, use hostname
>> instead of PID if the value is 1.
>>
>> Cc: stable@dpdk.org
>>
>> Signed-off-by: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
>> ---
> 
> I'm not too well versed in containers - is this hostname 1) always set, 
> and 2) always unique?
For docker, 1) hostname is always set. 2) The hostname is decided as 
short form of container ID, so it might not be unique even though very 
low possibility.

I found that we can get whole container ID in `/proc/self/cgroup` as 
discussed [1]. I think using hostname is reasonable way without running 
many secondary processes. However, it might be better to use 64 digits 
full container ID instead of 12 digits short ID if ensure uniqueness 
strongly. What do yo think?

[1] 
https://forums.docker.com/t/get-a-containers-full-id-from-inside-of-itself/37237
> 
> <snip>
> 
>> +    if (getpid() == 1) {
>> +        FILE *hn_fp;
>> +        hn_fp = fopen("/etc/hostname", "r");
>> +        if (hn_fp == NULL) {
>> +            RTE_LOG(ERR, EAL,
>> +                "Cannot open '/etc/hostname' for secondary\n");
>> +            return -1;
>> +        }
>> +
>> +        /* with docker, /etc/hostname just has one entry of hostname */
>> +        if (fscanf(hn_fp, "%s", proc_id) == EOF)
>> +            return -1;
> 
> Wouldn't an error in fscanf() leak the file handle? I think you need to 
> fclose() before checking the result.
I would like to fix it.

Regards,
Yasufumi
> 
>> +        fclose(hn_fp);
>> +    } else
>> +        sprintf(proc_id, "%d", (int)getpid());
>> +
>> +    snprintf(name, RTE_FBARRAY_NAME_LEN, "%s_%s",
>> +            primary_msl->memseg_arr.name, proc_id);
>>       ret = rte_fbarray_init(&local_msl->memseg_arr, name,
>>           primary_msl->memseg_arr.len,
>>
> 
>
Burakov, Anatoly July 9, 2019, 10:24 a.m. UTC | #4 
On 09-Jul-19 11:22 AM, Yasufumi Ogawa wrote:
> Hi Anatoly,
> 
> On 2019/07/05 17:53, Burakov, Anatoly wrote:
>> On 16-Apr-19 4:43 AM, ogawa.yasufumi@lab.ntt.co.jp wrote:
>>> From: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
>>>
>>> In secondary_msl_create_walk(), it creates a file for fbarrays with its
>>> PID for reserving unique name among secondary processes. However, it
>>> does not work if secondary is run as app container because each of
>>> containerized secondary has PID 1. To reserve unique name, use hostname
>>> instead of PID if the value is 1.
>>>
>>> Cc: stable@dpdk.org
>>>
>>> Signed-off-by: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
>>> ---
>>
>> I'm not too well versed in containers - is this hostname 1) always 
>> set, and 2) always unique?
> For docker, 1) hostname is always set. 2) The hostname is decided as 
> short form of container ID, so it might not be unique even though very 
> low possibility.
> 
> I found that we can get whole container ID in `/proc/self/cgroup` as 
> discussed [1]. I think using hostname is reasonable way without running 
> many secondary processes. However, it might be better to use 64 digits 
> full container ID instead of 12 digits short ID if ensure uniqueness 
> strongly. What do yo think?
> 
> [1] 
> https://forums.docker.com/t/get-a-containers-full-id-from-inside-of-itself/37237 
> 

I think it's better to err on the side of caution and guarantee better 
uniqueness. This code will get into an LTS and will be used for years to 
come :)
Burakov, Anatoly July 9, 2019, 10:26 a.m. UTC | #5 
On 09-Jul-19 11:24 AM, Burakov, Anatoly wrote:
> On 09-Jul-19 11:22 AM, Yasufumi Ogawa wrote:
>> Hi Anatoly,
>>
>> On 2019/07/05 17:53, Burakov, Anatoly wrote:
>>> On 16-Apr-19 4:43 AM, ogawa.yasufumi@lab.ntt.co.jp wrote:
>>>> From: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
>>>>
>>>> In secondary_msl_create_walk(), it creates a file for fbarrays with its
>>>> PID for reserving unique name among secondary processes. However, it
>>>> does not work if secondary is run as app container because each of
>>>> containerized secondary has PID 1. To reserve unique name, use hostname
>>>> instead of PID if the value is 1.
>>>>
>>>> Cc: stable@dpdk.org
>>>>
>>>> Signed-off-by: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
>>>> ---
>>>
>>> I'm not too well versed in containers - is this hostname 1) always 
>>> set, and 2) always unique?
>> For docker, 1) hostname is always set. 2) The hostname is decided as 
>> short form of container ID, so it might not be unique even though very 
>> low possibility.
>>
>> I found that we can get whole container ID in `/proc/self/cgroup` as 
>> discussed [1]. I think using hostname is reasonable way without 
>> running many secondary processes. However, it might be better to use 
>> 64 digits full container ID instead of 12 digits short ID if ensure 
>> uniqueness strongly. What do yo think?
>>
>> [1] 
>> https://forums.docker.com/t/get-a-containers-full-id-from-inside-of-itself/37237 
>>
> 
> I think it's better to err on the side of caution and guarantee better 
> uniqueness. This code will get into an LTS and will be used for years to 
> come :)
> 

...however, i think a full 64-digit ID won't even fit into the fbarray 
filename, as i believe it's limited to something like 64 chars. Perhaps 
hostname would be enough after all... or we can increase fbarray name 
length - that would require ABI breakage but the ABI is already broken 
in this release, so it's OK i think.
Yasufumi Ogawa July 11, 2019, 9:37 a.m. UTC | #6 
On 2019/07/09 19:26, Burakov, Anatoly wrote:
> On 09-Jul-19 11:24 AM, Burakov, Anatoly wrote:
>> On 09-Jul-19 11:22 AM, Yasufumi Ogawa wrote:
>>> Hi Anatoly,
>>>
>>> On 2019/07/05 17:53, Burakov, Anatoly wrote:
>>>> On 16-Apr-19 4:43 AM, ogawa.yasufumi@lab.ntt.co.jp wrote:
>>>>> From: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
>>>>>
>>>>> In secondary_msl_create_walk(), it creates a file for fbarrays with 
>>>>> its
>>>>> PID for reserving unique name among secondary processes. However, it
>>>>> does not work if secondary is run as app container because each of
>>>>> containerized secondary has PID 1. To reserve unique name, use 
>>>>> hostname
>>>>> instead of PID if the value is 1.
>>>>>
>>>>> Cc: stable@dpdk.org
>>>>>
>>>>> Signed-off-by: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
>>>>> ---
>>>>
>>>> I'm not too well versed in containers - is this hostname 1) always 
>>>> set, and 2) always unique?
>>> For docker, 1) hostname is always set. 2) The hostname is decided as 
>>> short form of container ID, so it might not be unique even though 
>>> very low possibility.
>>>
>>> I found that we can get whole container ID in `/proc/self/cgroup` as 
>>> discussed [1]. I think using hostname is reasonable way without 
>>> running many secondary processes. However, it might be better to use 
>>> 64 digits full container ID instead of 12 digits short ID if ensure 
>>> uniqueness strongly. What do yo think?
>>>
>>> [1] 
>>> https://forums.docker.com/t/get-a-containers-full-id-from-inside-of-itself/37237 
>>>
>>
>> I think it's better to err on the side of caution and guarantee better 
>> uniqueness. This code will get into an LTS and will be used for years 
>> to come :)
>>
> 
> ...however, i think a full 64-digit ID won't even fit into the fbarray 
> filename, as i believe it's limited to something like 64 chars. Perhaps 
> hostname would be enough after all... or we can increase fbarray name 
> length - that would require ABI breakage but the ABI is already broken 
> in this release, so it's OK i think.
OK.

 >> Wouldn't an error in fscanf() leak the file handle? I think you need 
to fclose() before checking the result.
 > I would like to fix it.
I would like send v3 patch for fixing for fclose().

Thanks,
Yasufumi
Burakov, Anatoly July 11, 2019, 9:43 a.m. UTC | #7 
On 11-Jul-19 10:37 AM, Yasufumi Ogawa wrote:
> On 2019/07/09 19:26, Burakov, Anatoly wrote:
>> On 09-Jul-19 11:24 AM, Burakov, Anatoly wrote:
>>> On 09-Jul-19 11:22 AM, Yasufumi Ogawa wrote:
>>>> Hi Anatoly,
>>>>
>>>> On 2019/07/05 17:53, Burakov, Anatoly wrote:
>>>>> On 16-Apr-19 4:43 AM, ogawa.yasufumi@lab.ntt.co.jp wrote:
>>>>>> From: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
>>>>>>
>>>>>> In secondary_msl_create_walk(), it creates a file for fbarrays 
>>>>>> with its
>>>>>> PID for reserving unique name among secondary processes. However, it
>>>>>> does not work if secondary is run as app container because each of
>>>>>> containerized secondary has PID 1. To reserve unique name, use 
>>>>>> hostname
>>>>>> instead of PID if the value is 1.
>>>>>>
>>>>>> Cc: stable@dpdk.org
>>>>>>
>>>>>> Signed-off-by: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
>>>>>> ---
>>>>>
>>>>> I'm not too well versed in containers - is this hostname 1) always 
>>>>> set, and 2) always unique?
>>>> For docker, 1) hostname is always set. 2) The hostname is decided as 
>>>> short form of container ID, so it might not be unique even though 
>>>> very low possibility.
>>>>
>>>> I found that we can get whole container ID in `/proc/self/cgroup` as 
>>>> discussed [1]. I think using hostname is reasonable way without 
>>>> running many secondary processes. However, it might be better to use 
>>>> 64 digits full container ID instead of 12 digits short ID if ensure 
>>>> uniqueness strongly. What do yo think?
>>>>
>>>> [1] 
>>>> https://forums.docker.com/t/get-a-containers-full-id-from-inside-of-itself/37237 
>>>>
>>>
>>> I think it's better to err on the side of caution and guarantee 
>>> better uniqueness. This code will get into an LTS and will be used 
>>> for years to come :)
>>>
>>
>> ...however, i think a full 64-digit ID won't even fit into the fbarray 
>> filename, as i believe it's limited to something like 64 chars. 
>> Perhaps hostname would be enough after all... or we can increase 
>> fbarray name length - that would require ABI breakage but the ABI is 
>> already broken in this release, so it's OK i think.
> OK.

Just a note: you're targetting this fix towards stable too. For stable, 
you cannot break ABI, so we would have to do with the shorter hostname. 
It's only for 19.08 that you can change fbarray length and use the full 
64-char container ID for uniqueness.

> 
>  >> Wouldn't an error in fscanf() leak the file handle? I think you need 
> to fclose() before checking the result.
>  > I would like to fix it.
> I would like send v3 patch for fixing for fclose().

Please do :)

> 
> Thanks,
> Yasufumi
> 
>
diff mbox series

Patch

diff --git a/lib/librte_eal/linux/eal/eal_memalloc.c b/lib/librte_eal/linux/eal/eal_memalloc.c
index 1e9ebb86d..beec03648 100644
--- a/lib/librte_eal/linux/eal/eal_memalloc.c
+++ b/lib/librte_eal/linux/eal/eal_memalloc.c
@@ -1362,6 +1362,7 @@  secondary_msl_create_walk(const struct rte_memseg_list *msl,
 	struct rte_memseg_list *primary_msl, *local_msl;
 	char name[PATH_MAX];
 	int msl_idx, ret;
+	char proc_id[16];
 
 	if (msl->external)
 		return 0;
@@ -1371,8 +1372,28 @@  secondary_msl_create_walk(const struct rte_memseg_list *msl,
 	local_msl = &local_memsegs[msl_idx];
 
 	/* create distinct fbarrays for each secondary */
-	snprintf(name, RTE_FBARRAY_NAME_LEN, "%s_%i",
-		primary_msl->memseg_arr.name, getpid());
+	/* if run secondary in a container, the name of fbarray file cannot
+	 * be decided with pid because getpid() always returns 1, so use
+	 * hostname as a unique identifier among containers instead.
+	 */
+	if (getpid() == 1) {
+		FILE *hn_fp;
+		hn_fp = fopen("/etc/hostname", "r");
+		if (hn_fp == NULL) {
+			RTE_LOG(ERR, EAL,
+				"Cannot open '/etc/hostname' for secondary\n");
+			return -1;
+		}
+
+		/* with docker, /etc/hostname just has one entry of hostname */
+		if (fscanf(hn_fp, "%s", proc_id) == EOF)
+			return -1;
+		fclose(hn_fp);
+	} else
+		sprintf(proc_id, "%d", (int)getpid());
+
+	snprintf(name, RTE_FBARRAY_NAME_LEN, "%s_%s",
+			primary_msl->memseg_arr.name, proc_id);
 
 	ret = rte_fbarray_init(&local_msl->memseg_arr, name,
 		primary_msl->memseg_arr.len,