[v10,2/3] devtools: script to send notifications of expired symbols

  Use this script with the output of the DPDK symbol tool, to notify
maintainers of expired symbols by email. You need to define the environment
variable DPDK_GETMAINTAINER_PATH for this tool to work.

Use terminal output to review the emails before sending.
e.g.
$ devtools/symbol-tool.py list-expired --format-output csv \
| DPDK_GETMAINTAINER_PATH=<somewhere>/get_maintainer.pl \
devtools/notify_expired_symbols.py --format-output terminal

Then use email output to send the emails to the maintainers.
e.g.
$ devtools/symbol-tool.py list-expired --format-output csv \
| DPDK_GETMAINTAINER_PATH=<somewhere>/get_maintainer.pl \
devtools/notify_expired_symbols.py --format-output email \
--smtp-server <server> --sender <someone@somewhere.com> \
--password <password> --cc <someone@somewhere.com>

Signed-off-by: Ray Kinsella <mdr@ashroe.eu>
---
 devtools/notify-symbol-maintainers.py | 256 ++++++++++++++++++++++++++
 1 file changed, 256 insertions(+)
 create mode 100755 devtools/notify-symbol-maintainers.py
  

Ray Kinsella <mdr@ashroe.eu> writes:

> Use this script with the output of the DPDK symbol tool, to notify
> maintainers of expired symbols by email. You need to define the environment
> variable DPDK_GETMAINTAINER_PATH for this tool to work.
>
> Use terminal output to review the emails before sending.
> e.g.
> $ devtools/symbol-tool.py list-expired --format-output csv \
> | DPDK_GETMAINTAINER_PATH=<somewhere>/get_maintainer.pl \
> devtools/notify_expired_symbols.py --format-output terminal
>
> Then use email output to send the emails to the maintainers.
> e.g.
> $ devtools/symbol-tool.py list-expired --format-output csv \
> | DPDK_GETMAINTAINER_PATH=<somewhere>/get_maintainer.pl \
> devtools/notify_expired_symbols.py --format-output email \
> --smtp-server <server> --sender <someone@somewhere.com> \
> --password <password> --cc <someone@somewhere.com>
>
> Signed-off-by: Ray Kinsella <mdr@ashroe.eu>
> ---
>  devtools/notify-symbol-maintainers.py | 256 ++++++++++++++++++++++++++
>  1 file changed, 256 insertions(+)
>  create mode 100755 devtools/notify-symbol-maintainers.py
>
> diff --git a/devtools/notify-symbol-maintainers.py b/devtools/notify-symbol-maintainers.py
> new file mode 100755
> index 0000000000..ee554687ff
> --- /dev/null
> +++ b/devtools/notify-symbol-maintainers.py
> @@ -0,0 +1,256 @@
> +#!/usr/bin/env python3
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2021 Intel Corporation
> +# pylint: disable=invalid-name
> +'''Tool to notify maintainers of expired symbols'''
> +import smtplib
> +import ssl
> +import sys
> +import subprocess
> +import argparse
> +from argparse import RawTextHelpFormatter
> +import time
> +from email.message import EmailMessage
> +
> +DESCRIPTION = '''
> +Use this script with the output of the DPDK symbol tool, to notify maintainers
> +and contributors of expired symbols by email. You need to define the environment
> +variable DPDK_GETMAINTAINER_PATH for this tool to work.
> +
> +Use terminal output to review the emails before sending.
> +e.g.
> +$ devtools/symbol-tool.py list-expired --format-output csv \\
> +| DPDK_GETMAINTAINER_PATH=<somewhere>/get_maintainer.pl \\
> +{s} --format-output terminal
> +
> +Then use email output to send the emails to the maintainers.
> +e.g.
> +$ devtools/symbol-tool.py list-expired --format-output csv \\
> +| DPDK_GETMAINTAINER_PATH=<somewhere>/get_maintainer.pl \\
> +{s} --format-output email \\
> +--smtp-server <server> --sender <someone@somewhere.com> --password <password> \\
> +--cc <someone@somewhere.com>
> +'''
> +
> +EMAIL_TEMPLATE = '''Hi there,
> +
> +Please note the symbols listed below have expired. In line with the DPDK ABI
> +policy, they should be scheduled for removal, in the next DPDK release.
> +
> +For more information, please see the DPDK ABI Policy, section 3.5.3.
> +https://doc.dpdk.org/guides/contributing/abi_policy.html
> +
> +Thanks,
> +
> +The DPDK Symbol Bot
> +
> +'''
> +
> +ABI_POLICY = 'doc/guides/contributing/abi_policy.rst'
> +MAINTAINERS = 'MAINTAINERS'
> +get_maintainer = ['devtools/get-maintainer.sh', \
> +                  '--email', '-f']

Maybe it's best to make this something that can be overridden.  There's
a series to change the .sh files to .py files.  Perhaps an environment
variable or argument?

> +def _get_maintainers(libpath):
> +    '''Get the maintainers for given library'''
> +    try:
> +        cmd = get_maintainer + [libpath]
> +        result = subprocess.run(cmd, \
> +                                stdout=subprocess.PIPE, \
> +                                stderr=subprocess.PIPE,
> +                                check=True)
> +    except subprocess.CalledProcessError:
> +        return None

You might consider handling

   except FileNotFoundError:
      ....

With a graceful exit and error message.  In case the get_maintainers
path changes.

> +    if result is None:
> +        return None
> +
> +    email = result.stdout.decode('utf-8')
> +    if email == '':
> +        return None
> +
> +    email = list(filter(None,email.split('\n')))
> +    return email
> +
> +default_maintainers = _get_maintainers(ABI_POLICY) + \
> +    _get_maintainers(MAINTAINERS)
> +
> +def get_maintainers(libpath):
> +    '''Get the maintainers for given library'''
> +    maintainers=_get_maintainers(libpath)
> +
> +    if maintainers is None:
> +        maintainers = default_maintainers
> +
> +    return maintainers
> +
> +def get_message(library, symbols, config):
> +    '''Build email message from symbols, config and maintainers'''
> +    contributors = {}
> +    message = {}
> +    maintainers = get_maintainers(library)
> +
> +    if maintainers != default_maintainers:
> +        message['CC'] = default_maintainers.copy()
> +
> +    if 'CC' in config:
> +        message.setdefault('CC',[]).append(config['CC'])
> +
> +    message['Subject'] = 'Expired symbols in {}\n'.format(library)
> +
> +    body = EMAIL_TEMPLATE
> +    body += '{:<50}{:<25}{:<25}\n'.format('Symbol','Contributor','Email')
> +    for sym in symbols:
> +        body += ('{:<50}{:<25}{:<25}\n'.format(sym,\
> +                                               symbols[sym]['name'],
> +                                               symbols[sym]['email'],
> +        ))
> +        email = symbols[sym]['email']
> +        contributors[email] = ''
> +
> +    contributors = list(contributors.keys())
> +
> +    message['To'] = maintainers + contributors
> +    message['Body'] = body
> +
> +    return message
> +
> +class OutputEmail():
> +    '''Format the output for email'''
> +    def __init__(self, config):
> +        self.config = config
> +
> +        self.terminal = OutputTerminal(config)
> +        context = ssl.create_default_context()
> +
> +        # Try to log in to server and send email
> +        try:
> +            self.server = smtplib.SMTP(config['smtp_server'], 587)
> +            self.server.starttls(context=context) # Secure the connection
> +            self.server.login(config['sender'], config['password'])
> +        except Exception as exception:
> +            print(exception)
> +            raise exception
> +
> +    def message(self,message):
> +        '''send email'''
> +        self.terminal.message(message)
> +
> +        msg = EmailMessage()
> +        msg.set_content(message.pop('Body'))
> +
> +        for key in message.keys():
> +            msg[key] = message[key]
> +
> +        msg['From'] = self.config['sender']
> +        msg['Reply-To'] = 'no-reply@dpdk.org'
> +
> +        self.server.send_message(msg)
> +
> +        time.sleep(1)

Why this sleep is needed?

> +
> +    def __del__(self):
> +        self.server.quit()
> +
> +class OutputTerminal(): # pylint: disable=too-few-public-methods
> +    '''Format the output for the terminal'''
> +    def __init__(self, config):
> +        self.config = config
> +
> +    def message(self,message):
> +        '''Print email to terminal'''
> +
> +        terminal = 'To:' + ', '.join(message['To']) + '\n'
> +        if 'sender' in self.config.keys():
> +            terminal += 'From:' + self.config['sender'] + '\n'
> +
> +        terminal += 'Reply-To:' + 'no-reply@dpdk.org' + '\n'
> +
> +        if 'CC' in message:
> +            terminal += 'CC:' + ', '.join(message['CC']) + '\n'
> +
> +        terminal += 'Subject:' + message['Subject'] + '\n'
> +        terminal += 'Body:' + message['Body'] + '\n'
> +
> +        print(terminal)
> +        print('-' * 80)
> +
> +def parse_config(args):
> +    '''put the command line args in the right places'''
> +    config = {}
> +    error_msg = None
> +
> +    outputs = {
> +        None : OutputTerminal,
> +        'terminal' : OutputTerminal,
> +        'email' : OutputEmail
> +    }
> +
> +    if args.format_output == 'email':
> +        if args.smtp_server is None:
> +            error_msg = 'SMTP server'
> +        else:
> +            config['smtp_server'] = args.smtp_server
> +
> +        if args.sender is None:
> +            error_msg = 'sender'
> +        else:
> +            config['sender'] = args.sender
> +
> +        if args.password is None:
> +            error_msg = 'password'
> +        else:
> +            config['password'] = args.password
> +
> +    if args.cc is not None:
> +        config['CC'] = args.cc
> +
> +    if error_msg is not None:
> +        print('Please specify a {} for email output'.format(error_msg))
> +        return None
> +
> +    config['output'] = outputs[args.format_output]
> +    return config
> +
> +def main():
> +    '''Main entry point'''
> +    parser = argparse.ArgumentParser(description=DESCRIPTION.format(s=__file__), \
> +                                     formatter_class=RawTextHelpFormatter)
> +    parser.add_argument('--format-output', choices=['terminal','email'], \
> +                        default='terminal')
> +    parser.add_argument('--smtp-server')
> +    parser.add_argument('--password')
> +    parser.add_argument('--sender')
> +    parser.add_argument('--cc')
> +
> +    args = parser.parse_args()
> +    config = parse_config(args)
> +    if config is None:
> +        return
> +
> +    symbols = {}
> +    lastlib = library = ''
> +
> +    output = config['output'](config)
> +
> +    for line in sys.stdin:
> +        line = line.rstrip('\n')
> +
> +        if line.find('mapfile') >= 0:
> +            continue
> +        library, symbol, name, email = line.split(',')
> +
> +        if library != lastlib:
> +            message = get_message(lastlib, symbols, config)
> +            output.message(message)
> +            symbols = {}
> +
> +        lastlib = library
> +        symbols[symbol] = {'name' : name, 'email' : email}
> +
> +    #print the last library
> +    message = get_message(lastlib, symbols, config)
> +    output.message(message)
> +
> +if __name__ == '__main__':
> +    main()
  

Hello Ray,

On Tue, Aug 31, 2021 at 4:51 PM Ray Kinsella <mdr@ashroe.eu> wrote:
>
> Use this script with the output of the DPDK symbol tool, to notify
> maintainers of expired symbols by email. You need to define the environment
> variable DPDK_GETMAINTAINER_PATH for this tool to work.
>
> Use terminal output to review the emails before sending.

Two comments:
- there are references of a previous name for the script,
%s/notify_expired_symbols.py/notify-symbol-maintainers.py/g

- and a reminder for the empty report that we received yesterday.
I think this can be reproduced with:

$ DPDK_GETMAINTAINER_PATH=devtools/get_maintainer.pl
devtools/notify-symbol-maintainers.py --format-output terminal <<EOF
> mapfile,expired (v21.08,v19.11),contributor name,contributor email
> lib/rib,rte_rib6_get_ip,Stephen Hemminger,stephen@networkplumber.org
> EOF
To:Ray Kinsella <mdr@ashroe.eu>, Thomas Monjalon <thomas@monjalon.net>
Reply-To:no-reply@dpdk.org
Subject:Expired symbols in

Body:Hi there,

Please note the symbols listed below have expired. In line with the DPDK ABI
policy, they should be scheduled for removal, in the next DPDK release.

For more information, please see the DPDK ABI Policy, section 3.5.3.
https://doc.dpdk.org/guides/contributing/abi_policy.html

Thanks,

The DPDK Symbol Bot

Symbol                                            Contributor
    Email


--------------------------------------------------------------------------------

^^^^
Here, empty report.

To:Vladimir Medvedkin <vladimir.medvedkin@intel.com>, stephen@networkplumber.org
Reply-To:no-reply@dpdk.org
CC:Ray Kinsella <mdr@ashroe.eu>, Thomas Monjalon <thomas@monjalon.net>
Subject:Expired symbols in lib/rib

Body:Hi there,

Please note the symbols listed below have expired. In line with the DPDK ABI
policy, they should be scheduled for removal, in the next DPDK release.

For more information, please see the DPDK ABI Policy, section 3.5.3.
https://doc.dpdk.org/guides/contributing/abi_policy.html

Thanks,

The DPDK Symbol Bot

Symbol                                            Contributor
    Email
rte_rib6_get_ip                                   Stephen Hemminger
    stephen@networkplumber.org


--------------------------------------------------------------------------------
  

On 01/09/2021 13:46, Aaron Conole wrote:
> Ray Kinsella <mdr@ashroe.eu> writes:
> 
>> Use this script with the output of the DPDK symbol tool, to notify
>> maintainers of expired symbols by email. You need to define the environment
>> variable DPDK_GETMAINTAINER_PATH for this tool to work.
>>
>> Use terminal output to review the emails before sending.
>> e.g.
>> $ devtools/symbol-tool.py list-expired --format-output csv \
>> | DPDK_GETMAINTAINER_PATH=<somewhere>/get_maintainer.pl \
>> devtools/notify_expired_symbols.py --format-output terminal
>>
>> Then use email output to send the emails to the maintainers.
>> e.g.
>> $ devtools/symbol-tool.py list-expired --format-output csv \
>> | DPDK_GETMAINTAINER_PATH=<somewhere>/get_maintainer.pl \
>> devtools/notify_expired_symbols.py --format-output email \
>> --smtp-server <server> --sender <someone@somewhere.com> \
>> --password <password> --cc <someone@somewhere.com>
>>
>> Signed-off-by: Ray Kinsella <mdr@ashroe.eu>
>> ---
>>  devtools/notify-symbol-maintainers.py | 256 ++++++++++++++++++++++++++
>>  1 file changed, 256 insertions(+)
>>  create mode 100755 devtools/notify-symbol-maintainers.py
>>
>> diff --git a/devtools/notify-symbol-maintainers.py b/devtools/notify-symbol-maintainers.py
>> new file mode 100755
>> index 0000000000..ee554687ff
>> --- /dev/null
>> +++ b/devtools/notify-symbol-maintainers.py
>> @@ -0,0 +1,256 @@
>> +#!/usr/bin/env python3
>> +# SPDX-License-Identifier: BSD-3-Clause
>> +# Copyright(c) 2021 Intel Corporation
>> +# pylint: disable=invalid-name
>> +'''Tool to notify maintainers of expired symbols'''
>> +import smtplib
>> +import ssl
>> +import sys
>> +import subprocess
>> +import argparse
>> +from argparse import RawTextHelpFormatter
>> +import time
>> +from email.message import EmailMessage
>> +
>> +DESCRIPTION = '''
>> +Use this script with the output of the DPDK symbol tool, to notify maintainers
>> +and contributors of expired symbols by email. You need to define the environment
>> +variable DPDK_GETMAINTAINER_PATH for this tool to work.
>> +
>> +Use terminal output to review the emails before sending.
>> +e.g.
>> +$ devtools/symbol-tool.py list-expired --format-output csv \\
>> +| DPDK_GETMAINTAINER_PATH=<somewhere>/get_maintainer.pl \\
>> +{s} --format-output terminal
>> +
>> +Then use email output to send the emails to the maintainers.
>> +e.g.
>> +$ devtools/symbol-tool.py list-expired --format-output csv \\
>> +| DPDK_GETMAINTAINER_PATH=<somewhere>/get_maintainer.pl \\
>> +{s} --format-output email \\
>> +--smtp-server <server> --sender <someone@somewhere.com> --password <password> \\
>> +--cc <someone@somewhere.com>
>> +'''
>> +
>> +EMAIL_TEMPLATE = '''Hi there,
>> +
>> +Please note the symbols listed below have expired. In line with the DPDK ABI
>> +policy, they should be scheduled for removal, in the next DPDK release.
>> +
>> +For more information, please see the DPDK ABI Policy, section 3.5.3.
>> +https://doc.dpdk.org/guides/contributing/abi_policy.html
>> +
>> +Thanks,
>> +
>> +The DPDK Symbol Bot
>> +
>> +'''
>> +
>> +ABI_POLICY = 'doc/guides/contributing/abi_policy.rst'
>> +MAINTAINERS = 'MAINTAINERS'
>> +get_maintainer = ['devtools/get-maintainer.sh', \
>> +                  '--email', '-f']
> 
> Maybe it's best to make this something that can be overridden.  There's
> a series to change the .sh files to .py files.  Perhaps an environment
> variable or argument?

ACK

> 
>> +def _get_maintainers(libpath):
>> +    '''Get the maintainers for given library'''
>> +    try:
>> +        cmd = get_maintainer + [libpath]
>> +        result = subprocess.run(cmd, \
>> +                                stdout=subprocess.PIPE, \
>> +                                stderr=subprocess.PIPE,
>> +                                check=True)
>> +    except subprocess.CalledProcessError:
>> +        return None
> 
> You might consider handling
> 
>    except FileNotFoundError:
>       ....
> 
> With a graceful exit and error message.  In case the get_maintainers
> path changes.

ACK

> 
>> +    if result is None:
>> +        return None
>> +
>> +    email = result.stdout.decode('utf-8')
>> +    if email == '':
>> +        return None
>> +
>> +    email = list(filter(None,email.split('\n')))
>> +    return email
>> +
>> +default_maintainers = _get_maintainers(ABI_POLICY) + \
>> +    _get_maintainers(MAINTAINERS)
>> +
>> +def get_maintainers(libpath):
>> +    '''Get the maintainers for given library'''
>> +    maintainers=_get_maintainers(libpath)
>> +
>> +    if maintainers is None:
>> +        maintainers = default_maintainers
>> +
>> +    return maintainers
>> +
>> +def get_message(library, symbols, config):
>> +    '''Build email message from symbols, config and maintainers'''
>> +    contributors = {}
>> +    message = {}
>> +    maintainers = get_maintainers(library)
>> +
>> +    if maintainers != default_maintainers:
>> +        message['CC'] = default_maintainers.copy()
>> +
>> +    if 'CC' in config:
>> +        message.setdefault('CC',[]).append(config['CC'])
>> +
>> +    message['Subject'] = 'Expired symbols in {}\n'.format(library)
>> +
>> +    body = EMAIL_TEMPLATE
>> +    body += '{:<50}{:<25}{:<25}\n'.format('Symbol','Contributor','Email')
>> +    for sym in symbols:
>> +        body += ('{:<50}{:<25}{:<25}\n'.format(sym,\
>> +                                               symbols[sym]['name'],
>> +                                               symbols[sym]['email'],
>> +        ))
>> +        email = symbols[sym]['email']
>> +        contributors[email] = ''
>> +
>> +    contributors = list(contributors.keys())
>> +
>> +    message['To'] = maintainers + contributors
>> +    message['Body'] = body
>> +
>> +    return message
>> +
>> +class OutputEmail():
>> +    '''Format the output for email'''
>> +    def __init__(self, config):
>> +        self.config = config
>> +
>> +        self.terminal = OutputTerminal(config)
>> +        context = ssl.create_default_context()
>> +
>> +        # Try to log in to server and send email
>> +        try:
>> +            self.server = smtplib.SMTP(config['smtp_server'], 587)
>> +            self.server.starttls(context=context) # Secure the connection
>> +            self.server.login(config['sender'], config['password'])
>> +        except Exception as exception:
>> +            print(exception)
>> +            raise exception
>> +
>> +    def message(self,message):
>> +        '''send email'''
>> +        self.terminal.message(message)
>> +
>> +        msg = EmailMessage()
>> +        msg.set_content(message.pop('Body'))
>> +
>> +        for key in message.keys():
>> +            msg[key] = message[key]
>> +
>> +        msg['From'] = self.config['sender']
>> +        msg['Reply-To'] = 'no-reply@dpdk.org'
>> +
>> +        self.server.send_message(msg)
>> +
>> +        time.sleep(1)
> 
> Why this sleep is needed?

Don't hammer the mail server :-)

> 
>> +
>> +    def __del__(self):
>> +        self.server.quit()
>> +
>> +class OutputTerminal(): # pylint: disable=too-few-public-methods
>> +    '''Format the output for the terminal'''
>> +    def __init__(self, config):
>> +        self.config = config
>> +
>> +    def message(self,message):
>> +        '''Print email to terminal'''
>> +
>> +        terminal = 'To:' + ', '.join(message['To']) + '\n'
>> +        if 'sender' in self.config.keys():
>> +            terminal += 'From:' + self.config['sender'] + '\n'
>> +
>> +        terminal += 'Reply-To:' + 'no-reply@dpdk.org' + '\n'
>> +
>> +        if 'CC' in message:
>> +            terminal += 'CC:' + ', '.join(message['CC']) + '\n'
>> +
>> +        terminal += 'Subject:' + message['Subject'] + '\n'
>> +        terminal += 'Body:' + message['Body'] + '\n'
>> +
>> +        print(terminal)
>> +        print('-' * 80)
>> +
>> +def parse_config(args):
>> +    '''put the command line args in the right places'''
>> +    config = {}
>> +    error_msg = None
>> +
>> +    outputs = {
>> +        None : OutputTerminal,
>> +        'terminal' : OutputTerminal,
>> +        'email' : OutputEmail
>> +    }
>> +
>> +    if args.format_output == 'email':
>> +        if args.smtp_server is None:
>> +            error_msg = 'SMTP server'
>> +        else:
>> +            config['smtp_server'] = args.smtp_server
>> +
>> +        if args.sender is None:
>> +            error_msg = 'sender'
>> +        else:
>> +            config['sender'] = args.sender
>> +
>> +        if args.password is None:
>> +            error_msg = 'password'
>> +        else:
>> +            config['password'] = args.password
>> +
>> +    if args.cc is not None:
>> +        config['CC'] = args.cc
>> +
>> +    if error_msg is not None:
>> +        print('Please specify a {} for email output'.format(error_msg))
>> +        return None
>> +
>> +    config['output'] = outputs[args.format_output]
>> +    return config
>> +
>> +def main():
>> +    '''Main entry point'''
>> +    parser = argparse.ArgumentParser(description=DESCRIPTION.format(s=__file__), \
>> +                                     formatter_class=RawTextHelpFormatter)
>> +    parser.add_argument('--format-output', choices=['terminal','email'], \
>> +                        default='terminal')
>> +    parser.add_argument('--smtp-server')
>> +    parser.add_argument('--password')
>> +    parser.add_argument('--sender')
>> +    parser.add_argument('--cc')
>> +
>> +    args = parser.parse_args()
>> +    config = parse_config(args)
>> +    if config is None:
>> +        return
>> +
>> +    symbols = {}
>> +    lastlib = library = ''
>> +
>> +    output = config['output'](config)
>> +
>> +    for line in sys.stdin:
>> +        line = line.rstrip('\n')
>> +
>> +        if line.find('mapfile') >= 0:
>> +            continue
>> +        library, symbol, name, email = line.split(',')
>> +
>> +        if library != lastlib:
>> +            message = get_message(lastlib, symbols, config)
>> +            output.message(message)
>> +            symbols = {}
>> +
>> +        lastlib = library
>> +        symbols[symbol] = {'name' : name, 'email' : email}
>> +
>> +    #print the last library
>> +    message = get_message(lastlib, symbols, config)
>> +    output.message(message)
>> +
>> +if __name__ == '__main__':
>> +    main()
>
  

Hi David,


On 01/09/2021 14:01, David Marchand wrote:
> Hello Ray,
> 
> On Tue, Aug 31, 2021 at 4:51 PM Ray Kinsella <mdr@ashroe.eu> wrote:
>>
>> Use this script with the output of the DPDK symbol tool, to notify
>> maintainers of expired symbols by email. You need to define the environment
>> variable DPDK_GETMAINTAINER_PATH for this tool to work.
>>
>> Use terminal output to review the emails before sending.

Just realized I missed this. 

> 
> Two comments:
> - there are references of a previous name for the script,
> %s/notify_expired_symbols.py/notify-symbol-maintainers.py/g

Fixed in v11 = I used __file__ instead.

> - and a reminder for the empty report that we received yesterday.
> I think this can be reproduced with:

Yes - I remember that, I will fix in v12.

> 
> $ DPDK_GETMAINTAINER_PATH=devtools/get_maintainer.pl
> devtools/notify-symbol-maintainers.py --format-output terminal <<EOF
>> mapfile,expired (v21.08,v19.11),contributor name,contributor email
>> lib/rib,rte_rib6_get_ip,Stephen Hemminger,stephen@networkplumber.org
>> EOF
> To:Ray Kinsella <mdr@ashroe.eu>, Thomas Monjalon <thomas@monjalon.net>
> Reply-To:no-reply@dpdk.org
> Subject:Expired symbols in
> 
> Body:Hi there,
> 
> Please note the symbols listed below have expired. In line with the DPDK ABI
> policy, they should be scheduled for removal, in the next DPDK release.
> 
> For more information, please see the DPDK ABI Policy, section 3.5.3.
> https://doc.dpdk.org/guides/contributing/abi_policy.html
> 
> Thanks,
> 
> The DPDK Symbol Bot
> 
> Symbol                                            Contributor
>     Email
> 
> 
> --------------------------------------------------------------------------------
> 
> ^^^^
> Here, empty report.
> 
> To:Vladimir Medvedkin <vladimir.medvedkin@intel.com>, stephen@networkplumber.org
> Reply-To:no-reply@dpdk.org
> CC:Ray Kinsella <mdr@ashroe.eu>, Thomas Monjalon <thomas@monjalon.net>
> Subject:Expired symbols in lib/rib
> 
> Body:Hi there,
> 
> Please note the symbols listed below have expired. In line with the DPDK ABI
> policy, they should be scheduled for removal, in the next DPDK release.
> 
> For more information, please see the DPDK ABI Policy, section 3.5.3.
> https://doc.dpdk.org/guides/contributing/abi_policy.html
> 
> Thanks,
> 
> The DPDK Symbol Bot
> 
> Symbol                                            Contributor
>     Email
> rte_rib6_get_ip                                   Stephen Hemminger
>     stephen@networkplumber.org
> 
> 
> --------------------------------------------------------------------------------
> 
>
  

On 01/09/2021 13:46, Aaron Conole wrote:
> Ray Kinsella <mdr@ashroe.eu> writes:
> 
>> Use this script with the output of the DPDK symbol tool, to notify
>> maintainers of expired symbols by email. You need to define the environment
>> variable DPDK_GETMAINTAINER_PATH for this tool to work.
>>
>> Use terminal output to review the emails before sending.
>> e.g.
>> $ devtools/symbol-tool.py list-expired --format-output csv \
>> | DPDK_GETMAINTAINER_PATH=<somewhere>/get_maintainer.pl \
>> devtools/notify_expired_symbols.py --format-output terminal
>>
>> Then use email output to send the emails to the maintainers.
>> e.g.
>> $ devtools/symbol-tool.py list-expired --format-output csv \
>> | DPDK_GETMAINTAINER_PATH=<somewhere>/get_maintainer.pl \
>> devtools/notify_expired_symbols.py --format-output email \
>> --smtp-server <server> --sender <someone@somewhere.com> \
>> --password <password> --cc <someone@somewhere.com>
>>
>> Signed-off-by: Ray Kinsella <mdr@ashroe.eu>
>> ---
>>  devtools/notify-symbol-maintainers.py | 256 ++++++++++++++++++++++++++
>>  1 file changed, 256 insertions(+)
>>  create mode 100755 devtools/notify-symbol-maintainers.py
>>
>> diff --git a/devtools/notify-symbol-maintainers.py b/devtools/notify-symbol-maintainers.py
>> new file mode 100755
>> index 0000000000..ee554687ff
>> --- /dev/null
>> +++ b/devtools/notify-symbol-maintainers.py
>> @@ -0,0 +1,256 @@
>> +#!/usr/bin/env python3
>> +# SPDX-License-Identifier: BSD-3-Clause
>> +# Copyright(c) 2021 Intel Corporation
>> +# pylint: disable=invalid-name
>> +'''Tool to notify maintainers of expired symbols'''
>> +import smtplib
>> +import ssl
>> +import sys
>> +import subprocess
>> +import argparse
>> +from argparse import RawTextHelpFormatter
>> +import time
>> +from email.message import EmailMessage
>> +
>> +DESCRIPTION = '''
>> +Use this script with the output of the DPDK symbol tool, to notify maintainers
>> +and contributors of expired symbols by email. You need to define the environment
>> +variable DPDK_GETMAINTAINER_PATH for this tool to work.
>> +
>> +Use terminal output to review the emails before sending.
>> +e.g.
>> +$ devtools/symbol-tool.py list-expired --format-output csv \\
>> +| DPDK_GETMAINTAINER_PATH=<somewhere>/get_maintainer.pl \\
>> +{s} --format-output terminal
>> +
>> +Then use email output to send the emails to the maintainers.
>> +e.g.
>> +$ devtools/symbol-tool.py list-expired --format-output csv \\
>> +| DPDK_GETMAINTAINER_PATH=<somewhere>/get_maintainer.pl \\
>> +{s} --format-output email \\
>> +--smtp-server <server> --sender <someone@somewhere.com> --password <password> \\
>> +--cc <someone@somewhere.com>
>> +'''
>> +
>> +EMAIL_TEMPLATE = '''Hi there,
>> +
>> +Please note the symbols listed below have expired. In line with the DPDK ABI
>> +policy, they should be scheduled for removal, in the next DPDK release.
>> +
>> +For more information, please see the DPDK ABI Policy, section 3.5.3.
>> +https://doc.dpdk.org/guides/contributing/abi_policy.html
>> +
>> +Thanks,
>> +
>> +The DPDK Symbol Bot
>> +
>> +'''
>> +
>> +ABI_POLICY = 'doc/guides/contributing/abi_policy.rst'
>> +MAINTAINERS = 'MAINTAINERS'
>> +get_maintainer = ['devtools/get-maintainer.sh', \
>> +                  '--email', '-f']
> 
> Maybe it's best to make this something that can be overridden.  There's
> a series to change the .sh files to .py files.  Perhaps an environment
> variable or argument?
> 
>> +def _get_maintainers(libpath):
>> +    '''Get the maintainers for given library'''
>> +    try:
>> +        cmd = get_maintainer + [libpath]
>> +        result = subprocess.run(cmd, \
>> +                                stdout=subprocess.PIPE, \
>> +                                stderr=subprocess.PIPE,
>> +                                check=True)
>> +    except subprocess.CalledProcessError:
>> +        return None
> 
> You might consider handling
> 
>    except FileNotFoundError:
>       ....
> 
> With a graceful exit and error message.  In case the get_maintainers
> path changes.
> 

So FYI - this get's into the weed a bit.
As there is already a DPDK_GETMAINTAINER_PATH environment variable,
what would you call a new variable.

So instead I added logic for the script to sanity check that _everything_
is defined and where it expects it to be, and then complain loudly 
and die when it is not.

The devtools scripts already cross-reference either each, so I'd expect
any changes changing to get-maintainers.sh to get-maintainers.py to take
care of cross-references. 

Ray K
  

On 01/09/2021 14:01, David Marchand wrote:
> Hello Ray,
> 
> On Tue, Aug 31, 2021 at 4:51 PM Ray Kinsella <mdr@ashroe.eu> wrote:
>>
>> Use this script with the output of the DPDK symbol tool, to notify
>> maintainers of expired symbols by email. You need to define the environment
>> variable DPDK_GETMAINTAINER_PATH for this tool to work.
>>
>> Use terminal output to review the emails before sending.
> 
> Two comments:
> - there are references of a previous name for the script,
> %s/notify_expired_symbols.py/notify-symbol-maintainers.py/g
> 
> - and a reminder for the empty report that we received yesterday.
> I think this can be reproduced with:
> 

I err'ed - this taken care of v11 also.

Ray K