[v7,02/10] buildtools: add script for updating symbols abi version
diff mbox series

Message ID 1c066505836ad4cde9e4935881c28a03d8889126.1573230233.git.anatoly.burakov@intel.com
State Superseded, archived
Delegated to: Thomas Monjalon
Headers show
Series
  • Implement the new ABI policy and add helper scripts
Related show

Checks

Context Check Description
ci/Intel-compilation success Compilation OK
ci/checkpatch success coding style OK

Commit Message

Burakov, Anatoly Nov. 8, 2019, 4:25 p.m. UTC
From: Pawel Modrak <pawelx.modrak@intel.com>

Add a script that automatically merges all stable ABI's under one
ABI section with the new version, while leaving experimental
section exactly as it is.

Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
---

Notes:
    v7:
    - Do not remove stable ABI if it was empty
    
    v6:
    - Split map file generation function in two
    - Do not print stable ABI if it wasn't present
    
    v3:
    - Add comments to regex patterns
    
    v2:
    - Reworked script to be pep8-compliant and more reliable

 buildtools/update_version_map_abi.py | 173 +++++++++++++++++++++++++++
 1 file changed, 173 insertions(+)
 create mode 100755 buildtools/update_version_map_abi.py

Comments

Thomas Monjalon Nov. 19, 2019, 2:01 p.m. UTC | #1
08/11/2019 17:25, Anatoly Burakov:
> From: Pawel Modrak <pawelx.modrak@intel.com>
> 
> Add a script that automatically merges all stable ABI's under one
> ABI section with the new version, while leaving experimental
> section exactly as it is.
> 
> Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> Acked-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
> --- /dev/null
> +++ b/buildtools/update_version_map_abi.py
> +"""
> +A Python program to update the ABI version and function names in a DPDK
> +lib_*_version.map file. Called from the buildtools/update_abi.sh utility.
> +"""

Clearly this script is doing more than updating a version number and names.
I already see more in the commit log.
Please would you like to add something in this description?
Burakov, Anatoly Nov. 19, 2019, 3:38 p.m. UTC | #2
On 19-Nov-19 2:01 PM, Thomas Monjalon wrote:
> 08/11/2019 17:25, Anatoly Burakov:
>> From: Pawel Modrak <pawelx.modrak@intel.com>
>>
>> Add a script that automatically merges all stable ABI's under one
>> ABI section with the new version, while leaving experimental
>> section exactly as it is.
>>
>> Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
>> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
>> Acked-by: Bruce Richardson <bruce.richardson@intel.com>
>> ---
>> --- /dev/null
>> +++ b/buildtools/update_version_map_abi.py
>> +"""
>> +A Python program to update the ABI version and function names in a DPDK
>> +lib_*_version.map file. Called from the buildtools/update_abi.sh utility.
>> +"""
> 
> Clearly this script is doing more than updating a version number and names.
> I already see more in the commit log.
> Please would you like to add something in this description?
> 
> 

Not sure what you're referring to. That's exactly what it's doing. What 
else is there that i'm missing?
Thomas Monjalon Nov. 19, 2019, 4:05 p.m. UTC | #3
19/11/2019 16:38, Burakov, Anatoly:
> On 19-Nov-19 2:01 PM, Thomas Monjalon wrote:
> > 08/11/2019 17:25, Anatoly Burakov:
> >> From: Pawel Modrak <pawelx.modrak@intel.com>
> >>
> >> Add a script that automatically merges all stable ABI's under one
> >> ABI section with the new version, while leaving experimental
> >> section exactly as it is.
> >>
> >> Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
> >> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> >> Acked-by: Bruce Richardson <bruce.richardson@intel.com>
> >> ---
> >> --- /dev/null
> >> +++ b/buildtools/update_version_map_abi.py
> >> +"""
> >> +A Python program to update the ABI version and function names in a DPDK
> >> +lib_*_version.map file. Called from the buildtools/update_abi.sh utility.
> >> +"""
> > 
> > Clearly this script is doing more than updating a version number and names.
> > I already see more in the commit log.
> > Please would you like to add something in this description?
> 
> Not sure what you're referring to. That's exactly what it's doing. What 
> else is there that i'm missing?

"merges all stable ABI's under one ABI section"
"while leaving experimental section exactly as it is"

I think this needs to be in the comment at the top of the file,
so we have an introduction for the code in the rest of the file.

Patch
diff mbox series

diff --git a/buildtools/update_version_map_abi.py b/buildtools/update_version_map_abi.py
new file mode 100755
index 0000000000..0f6196140e
--- /dev/null
+++ b/buildtools/update_version_map_abi.py
@@ -0,0 +1,173 @@ 
+#!/usr/bin/env python
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+
+"""
+A Python program to update the ABI version and function names in a DPDK
+lib_*_version.map file. Called from the buildtools/update_abi.sh utility.
+"""
+
+from __future__ import print_function
+import argparse
+import sys
+import re
+
+
+def __parse_map_file(f_in):
+    # match function name, followed by semicolon, followed by EOL, optionally
+    # with whitespace inbetween each item
+    func_line_regex = re.compile(r"\s*"
+                                 r"(?P<func>[a-zA-Z_0-9]+)"
+                                 r"\s*"
+                                 r";"
+                                 r"\s*"
+                                 r"$")
+    # match section name, followed by opening bracked, followed by EOL,
+    # optionally with whitespace inbetween each item
+    section_begin_regex = re.compile(r"\s*"
+                                     r"(?P<version>[a-zA-Z0-9_\.]+)"
+                                     r"\s*"
+                                     r"{"
+                                     r"\s*"
+                                     r"$")
+    # match closing bracket, optionally followed by section name (for when we
+    # inherit from another ABI version), followed by semicolon, followed by
+    # EOL, optionally with whitespace inbetween each item
+    section_end_regex = re.compile(r"\s*"
+                                   r"}"
+                                   r"\s*"
+                                   r"(?P<parent>[a-zA-Z0-9_\.]+)?"
+                                   r"\s*"
+                                   r";"
+                                   r"\s*"
+                                   r"$")
+
+    # for stable ABI, we don't care about which version introduced which
+    # function, we just flatten the list. there are dupes in certain files, so
+    # use a set instead of a list
+    stable_lines = set()
+    # copy experimental section as is
+    experimental_lines = []
+    in_experimental = False
+    has_stable = False
+
+    # gather all functions
+    for line in f_in:
+        # clean up the line
+        line = line.strip('\n').strip()
+
+        # is this an end of section?
+        match = section_end_regex.match(line)
+        if match:
+            # whatever section this was, it's not active any more
+            in_experimental = False
+            continue
+
+        # if we're in the middle of experimental section, we need to copy
+        # the section verbatim, so just add the line
+        if in_experimental:
+            experimental_lines += [line]
+            continue
+
+        # skip empty lines
+        if not line:
+            continue
+
+        # is this a beginning of a new section?
+        match = section_begin_regex.match(line)
+        if match:
+            cur_section = match.group("version")
+            # is it experimental?
+            in_experimental = cur_section == "EXPERIMENTAL"
+            if not in_experimental:
+                has_stable = True
+            continue
+
+        # is this a function?
+        match = func_line_regex.match(line)
+        if match:
+            stable_lines.add(match.group("func"))
+
+    return has_stable, stable_lines, experimental_lines
+
+
+def __generate_stable_abi(f_out, abi_version, lines):
+    # print ABI version header
+    print("DPDK_{} {{".format(abi_version), file=f_out)
+
+    # print global section if it exists
+    if lines:
+        print("\tglobal:", file=f_out)
+        # blank line
+        print(file=f_out)
+
+        # print all stable lines, alphabetically sorted
+        for line in sorted(lines):
+            print("\t{};".format(line), file=f_out)
+
+        # another blank line
+        print(file=f_out)
+
+    # print local section
+    print("\tlocal: *;", file=f_out)
+
+    # end stable version
+    print("};", file=f_out)
+
+
+def __generate_experimental_abi(f_out, lines):
+    # start experimental section
+    print("EXPERIMENTAL {", file=f_out)
+
+    # print all experimental lines as they were
+    for line in lines:
+        # don't print empty whitespace
+        if not line:
+            print("", file=f_out)
+        else:
+            print("\t{}".format(line), file=f_out)
+
+    # end section
+    print("};", file=f_out)
+
+
+def __main():
+    arg_parser = argparse.ArgumentParser(
+        description='Merge versions in linker version script.')
+
+    arg_parser.add_argument("map_file", type=str,
+                            help='path to linker version script file '
+                                 '(pattern: *version.map)')
+    arg_parser.add_argument("abi_version", type=str,
+                            help='target ABI version (pattern: MAJOR.MINOR)')
+
+    parsed = arg_parser.parse_args()
+
+    if not parsed.map_file.endswith('version.map'):
+        print("Invalid input file: {}".format(parsed.map_file),
+              file=sys.stderr)
+        arg_parser.print_help()
+        sys.exit(1)
+
+    if not re.match(r"\d{1,2}\.\d{1,2}", parsed.abi_version):
+        print("Invalid ABI version: {}".format(parsed.abi_version),
+              file=sys.stderr)
+        arg_parser.print_help()
+        sys.exit(1)
+
+    with open(parsed.map_file) as f_in:
+        has_stable, stable_lines, experimental_lines = __parse_map_file(f_in)
+
+    with open(parsed.map_file, 'w') as f_out:
+        need_newline = has_stable and experimental_lines
+        if has_stable:
+            __generate_stable_abi(f_out, parsed.abi_version, stable_lines)
+        if need_newline:
+            # separate sections with a newline
+            print(file=f_out)
+        if experimental_lines:
+            __generate_experimental_abi(f_out, experimental_lines)
+
+
+if __name__ == "__main__":
+    __main()