more documentation

This commit is contained in:
JanLJL
2020-03-05 18:39:38 +01:00
parent 3fb053fa79
commit dcd5b8fd61
11 changed files with 126 additions and 25 deletions

View File

@@ -218,7 +218,7 @@ This can be achieved by running OSACA with the command line option ``--import MI
``ARCH`` defines the abbreviation of the target architecture for which the instructions will be added and file must be the path to the generated output file of the benchmark. ``ARCH`` defines the abbreviation of the target architecture for which the instructions will be added and file must be the path to the generated output file of the benchmark.
The format of this file has to match either the basic command line output of ibench, e.g., The format of this file has to match either the basic command line output of ibench, e.g.,
.. code-block:: .. code-block:: bash
[INSTRUCTION FORM]-TP: 0.500 (clock cycles) [DEBUG - result: 1.000000] [INSTRUCTION FORM]-TP: 0.500 (clock cycles) [DEBUG - result: 1.000000]
[INSTRUCTION FORM]-LT: 4.000 (clock cycles) [DEBUG - result: 1.000000] [INSTRUCTION FORM]-LT: 4.000 (clock cycles) [DEBUG - result: 1.000000]
@@ -226,7 +226,7 @@ The format of this file has to match either the basic command line output of ibe
or the command line output of asmbench including the name of the instruction form in a separate line at the or the command line output of asmbench including the name of the instruction form in a separate line at the
beginning, e.g.: beginning, e.g.:
.. code-block:: .. code-block:: bash
[INSTRUCTION FORM] [INSTRUCTION FORM]
Latency: 4.00 cycle Latency: 4.00 cycle
@@ -276,7 +276,7 @@ Database check
Since a manual adjustment of the ISA DB is currently indispensable when adding new instruction forms, Since a manual adjustment of the ISA DB is currently indispensable when adding new instruction forms,
OSACA provides a database sanity check using the --db-check flag. It can be executed via: OSACA provides a database sanity check using the --db-check flag. It can be executed via:
.. code-block:: .. code-block:: bash
osaca --arch ARCH --db-check [-v] file osaca --arch ARCH --db-check [-v] file
@@ -304,7 +304,7 @@ The code shows a simple scalar multiplication of a vector ``b`` and a floating-p
The result is written in vector ``a``. The result is written in vector ``a``.
After including the OSACA byte marker into the assembly, one can start the analysis typing After including the OSACA byte marker into the assembly, one can start the analysis typing
.. code:: bash .. code-block:: bash
osaca --arch CSX PATH/TO/FILE osaca --arch CSX PATH/TO/FILE
@@ -312,7 +312,7 @@ in the command line.
The output is: The output is:
.. code-block:: .. code-block:: bash
Open Source Architecture Code Analyzer (OSACA) - v0.3 Open Source Architecture Code Analyzer (OSACA) - v0.3
Analyzed file: scale.s.csx.O3.s Analyzed file: scale.s.csx.O3.s

View File

@@ -60,7 +60,7 @@ html_theme = 'sphinx_rtd_theme'
# Add any paths that contain custom static files (such as style sheets) here, # Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files, # relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css". # so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static'] html_static_path = []
htmlhelp_basename = 'osaca_doc' htmlhelp_basename = 'osaca_doc'
html_sidebars = {'**': ['globaltoc.html', 'relations.html', 'sourcelink.html', 'searchbox.html']} html_sidebars = {'**': ['globaltoc.html', 'relations.html', 'sourcelink.html', 'searchbox.html']}

View File

@@ -1,5 +1,5 @@
OSACA |br| Open Source Architecture Code Analyzer OSACA |br| Open Source Architecture Code Analyzer
====================== =================================================
.. toctree:: .. toctree::
:maxdepth: 2 :maxdepth: 2

View File

@@ -12,12 +12,23 @@ For an innermost loop kernel in assembly, this tool allows automatic instruction
.. image:: https://travis-ci.org/RRZE-HPC/OSACA.svg?branch=master .. image:: https://travis-ci.org/RRZE-HPC/OSACA.svg?branch=master
:target: https://travis-ci.org/RRZE-HPC/OSACA :target: https://travis-ci.org/RRZE-HPC/OSACA
:alt: Build Status
.. image:: https://codecov.io/github/RRZE-HPC/OSACA/coverage.svg?branch=master .. image:: https://codecov.io/github/RRZE-HPC/OSACA/coverage.svg?branch=master
:target: https://codecov.io/github/RRZE-HPC/OSACA?branch=master :target: https://codecov.io/github/RRZE-HPC/OSACA?branch=master
:alt: Code Coverage
.. image:: https://readthedocs.org/projects/osaca/badge/?version=latest
:target: https://osaca.readthedocs.io/en/latest/?badge=latest
:alt: Documentation Status
.. image:: https://img.shields.io/badge/read-the_docs-blue
:target: https://osaca.readthedocs.io/
:alt: Docs
.. image:: https://img.shields.io/badge/code%20style-black-000000.svg .. image:: https://img.shields.io/badge/code%20style-black-000000.svg
:target: https://github.com/ambv/black :target: https://github.com/ambv/black
:alt: Code Style
Getting started Getting started
=============== ===============
@@ -207,7 +218,7 @@ This can be achieved by running OSACA with the command line option ``--import MI
``ARCH`` defines the abbreviation of the target architecture for which the instructions will be added and file must be the path to the generated output file of the benchmark. ``ARCH`` defines the abbreviation of the target architecture for which the instructions will be added and file must be the path to the generated output file of the benchmark.
The format of this file has to match either the basic command line output of ibench, e.g., The format of this file has to match either the basic command line output of ibench, e.g.,
.. code-block:: .. code-block:: bash
[INSTRUCTION FORM]-TP: 0.500 (clock cycles) [DEBUG - result: 1.000000] [INSTRUCTION FORM]-TP: 0.500 (clock cycles) [DEBUG - result: 1.000000]
[INSTRUCTION FORM]-LT: 4.000 (clock cycles) [DEBUG - result: 1.000000] [INSTRUCTION FORM]-LT: 4.000 (clock cycles) [DEBUG - result: 1.000000]
@@ -215,7 +226,7 @@ The format of this file has to match either the basic command line output of ibe
or the command line output of asmbench including the name of the instruction form in a separate line at the or the command line output of asmbench including the name of the instruction form in a separate line at the
beginning, e.g.: beginning, e.g.:
.. code-block:: .. code-block:: bash
[INSTRUCTION FORM] [INSTRUCTION FORM]
Latency: 4.00 cycle Latency: 4.00 cycle
@@ -265,7 +276,7 @@ Database check
Since a manual adjustment of the ISA DB is currently indispensable when adding new instruction forms, Since a manual adjustment of the ISA DB is currently indispensable when adding new instruction forms,
OSACA provides a database sanity check using the --db-check flag. It can be executed via: OSACA provides a database sanity check using the --db-check flag. It can be executed via:
.. code-block:: .. code-block:: bash
osaca --arch ARCH --db-check [-v] file osaca --arch ARCH --db-check [-v] file
@@ -293,7 +304,7 @@ The code shows a simple scalar multiplication of a vector ``b`` and a floating-p
The result is written in vector ``a``. The result is written in vector ``a``.
After including the OSACA byte marker into the assembly, one can start the analysis typing After including the OSACA byte marker into the assembly, one can start the analysis typing
.. code:: bash .. code-block:: bash
osaca --arch CSX PATH/TO/FILE osaca --arch CSX PATH/TO/FILE
@@ -301,7 +312,7 @@ in the command line.
The output is: The output is:
.. code-block:: .. code-block:: bash
Open Source Architecture Code Analyzer (OSACA) - v0.3 Open Source Architecture Code Analyzer (OSACA) - v0.3
Analyzed file: scale.s.csx.O3.s Analyzed file: scale.s.csx.O3.s

View File

@@ -19,6 +19,9 @@ def sanity_check(arch: str, verbose=False, output_file=sys.stdout):
:type arch: str :type arch: str
:param verbose: verbose output flag, defaults to `False` :param verbose: verbose output flag, defaults to `False`
:type verbose: bool, optional :type verbose: bool, optional
:param output_file: output stream specifying where to write output, defaults to :class:`sys. stdout`
:type output_file: stream, optional
""" """
# load arch machine model # load arch machine model
arch_mm = MachineModel(arch=arch) arch_mm = MachineModel(arch=arch)
@@ -184,6 +187,10 @@ def _get_ibench_output(input_data, isa):
def _validate_measurement(measurement, mode): def _validate_measurement(measurement, mode):
"""
Check if latency has a maximum deviation of 0.05% and throughput is a reciprocal of a
an integer number.
"""
if mode == 'lt': if mode == 'lt':
if ( if (
math.floor(measurement) * 1.05 >= measurement math.floor(measurement) * 1.05 >= measurement
@@ -204,6 +211,7 @@ def _validate_measurement(measurement, mode):
def _create_db_operand(operand, isa): def _create_db_operand(operand, isa):
"""Get DB operand by input string and ISA."""
if isa == 'aarch64': if isa == 'aarch64':
return _create_db_operand_aarch64(operand) return _create_db_operand_aarch64(operand)
elif isa == 'x86': elif isa == 'x86':
@@ -211,6 +219,7 @@ def _create_db_operand(operand, isa):
def _create_db_operand_aarch64(operand): def _create_db_operand_aarch64(operand):
"""Get DB operand for AArch64 by operand string."""
if operand == 'i': if operand == 'i':
return {'class': 'immediate', 'imd': 'int'} return {'class': 'immediate', 'imd': 'int'}
elif operand in 'wxbhsdq': elif operand in 'wxbhsdq':
@@ -236,6 +245,7 @@ def _create_db_operand_aarch64(operand):
def _create_db_operand_x86(operand): def _create_db_operand_x86(operand):
"""Get DB operand for AArch64 by operand string."""
if operand == 'r': if operand == 'r':
return {'class': 'register', 'name': 'gpr'} return {'class': 'register', 'name': 'gpr'}
elif operand in 'xyz': elif operand in 'xyz':
@@ -260,6 +270,7 @@ def _create_db_operand_x86(operand):
def _check_sanity_arch_db(arch_mm, isa_mm): def _check_sanity_arch_db(arch_mm, isa_mm):
"""Do sanity check for ArchDB by given ISA."""
suspicious_prefixes_x86 = ['vfm', 'fm'] suspicious_prefixes_x86 = ['vfm', 'fm']
suspicious_prefixes_arm = ['fml', 'ldp', 'stp', 'str'] suspicious_prefixes_arm = ['fml', 'ldp', 'stp', 'str']
if arch_mm.get_ISA().lower() == 'aarch64': if arch_mm.get_ISA().lower() == 'aarch64':
@@ -318,6 +329,7 @@ def _check_sanity_arch_db(arch_mm, isa_mm):
def _check_sanity_isa_db(arch_mm, isa_mm): def _check_sanity_isa_db(arch_mm, isa_mm):
"""Do sanity check for an ISA DB."""
# returned lists # returned lists
duplicate_instr_isa = [] duplicate_instr_isa = []
only_in_isa = [] only_in_isa = []
@@ -343,6 +355,7 @@ def _check_sanity_isa_db(arch_mm, isa_mm):
def _get_sanity_report( def _get_sanity_report(
total, m_tp, m_l, m_pp, suspic_instr, dup_arch, dup_isa, only_isa, verbose=False, colors=False total, m_tp, m_l, m_pp, suspic_instr, dup_arch, dup_isa, only_isa, verbose=False, colors=False
): ):
"""Get sanity summary report."""
s = '' s = ''
# non-verbose summary # non-verbose summary
s += 'SUMMARY\n----------------------\n' s += 'SUMMARY\n----------------------\n'
@@ -376,6 +389,7 @@ def _get_sanity_report(
def _get_sanity_report_verbose( def _get_sanity_report_verbose(
total, m_tp, m_l, m_pp, suspic_instr, dup_arch, dup_isa, only_isa, colors=False total, m_tp, m_l, m_pp, suspic_instr, dup_arch, dup_isa, only_isa, colors=False
): ):
"""Get the verbose part of the sanity report with all missing instruction forms."""
BRIGHT_CYAN = '\033[1;36;1m' if colors else '' BRIGHT_CYAN = '\033[1;36;1m' if colors else ''
BRIGHT_BLUE = '\033[1;34;1m' if colors else '' BRIGHT_BLUE = '\033[1;34;1m' if colors else ''
BRIGHT_RED = '\033[1;31;1m' if colors else '' BRIGHT_RED = '\033[1;31;1m' if colors else ''
@@ -418,6 +432,7 @@ def _get_sanity_report_verbose(
def _get_full_instruction_name(instruction_form): def _get_full_instruction_name(instruction_form):
"""Get full instruction form name/identifier string out of given instruction form."""
operands = [] operands = []
for op in instruction_form['operands']: for op in instruction_form['operands']:
op_attrs = [ op_attrs = [
@@ -429,16 +444,19 @@ def _get_full_instruction_name(instruction_form):
def __represent_none(self, data): def __represent_none(self, data):
"""Get YAML None representation."""
return self.represent_scalar(u'tag:yaml.org,2002:null', u'~') return self.represent_scalar(u'tag:yaml.org,2002:null', u'~')
def _create_yaml_object(): def _create_yaml_object():
"""Create YAML module with None representation."""
yaml_obj = ruamel.yaml.YAML() yaml_obj = ruamel.yaml.YAML()
yaml_obj.representer.add_representer(type(None), __represent_none) yaml_obj.representer.add_representer(type(None), __represent_none)
return yaml_obj return yaml_obj
def __dump_data_to_yaml(filepath, data): def __dump_data_to_yaml(filepath, data):
"""Dump data to YAML file at given filepath."""
# first add 'normal' meta data in the right order (no ordered dict yet) # first add 'normal' meta data in the right order (no ordered dict yet)
meta_data = dict(data) meta_data = dict(data)
del meta_data['instruction_forms'] del meta_data['instruction_forms']

View File

@@ -159,7 +159,7 @@ def import_data(benchmark_type, arch, filepath, output_file=sys.stdout):
:param filepath: filepath of the output file" :param filepath: filepath of the output file"
:type filepath: str :type filepath: str
:param output_file: output stream specifying where to write output, defaults to :class:`sys.stdout` :param output_file: output stream specifying where to write output, defaults to :class:`sys.stdout`
:type output_file: stream :type output_file: stream, optional
""" """
if benchmark_type.lower() == 'ibench': if benchmark_type.lower() == 'ibench':
import_benchmark_output(arch, 'ibench', filepath, output=output_file) import_benchmark_output(arch, 'ibench', filepath, output=output_file)
@@ -210,6 +210,7 @@ def inspect(args, output_file=sys.stdout):
:param args: arguments given from :class:`~argparse.ArgumentParser` after parsing :param args: arguments given from :class:`~argparse.ArgumentParser` after parsing
:param output_file: Define the stream for output, defaults to :class:`sys.stdout` :param output_file: Define the stream for output, defaults to :class:`sys.stdout`
:type output_file: stream, optional
""" """
arch = args.arch arch = args.arch
isa = MachineModel.get_isa_for_arch(arch) isa = MachineModel.get_isa_for_arch(arch)
@@ -251,6 +252,7 @@ def run(args, output_file=sys.stdout):
:param args: arguments given from :class:`~argparse.ArgumentParser` after parsing :param args: arguments given from :class:`~argparse.ArgumentParser` after parsing
:param output_file: Define the stream for output, defaults to :class:`sys.stdout` :param output_file: Define the stream for output, defaults to :class:`sys.stdout`
:type output_file: stream, optional
""" """
if args.check_db: if args.check_db:
# Sanity check on DB # Sanity check on DB

View File

@@ -1,4 +1,5 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""Attribute Dictionary to access dictionary entries as attributes."""
class AttrDict(dict): class AttrDict(dict):
@@ -8,14 +9,19 @@ class AttrDict(dict):
@staticmethod @staticmethod
def convert_dict(dictionary): def convert_dict(dictionary):
"""
Convert given dictionary to `AttrDict`.
:param dictionary: `dict` to be converted
:type dictionary: `dict`
:returns: `AttrDict` representation of ``dictionary``
"""
if isinstance(dictionary, type(list())): if isinstance(dictionary, type(list())):
return [AttrDict.convert_dict(x) for x in dictionary] return [AttrDict.convert_dict(x) for x in dictionary]
if isinstance(dictionary, type(dict())): if isinstance(dictionary, type(dict())):
for key in list(dictionary.keys()): for key in list(dictionary.keys()):
entry = dictionary[key] entry = dictionary[key]
if isinstance(entry, type(dict())) or isinstance( if isinstance(entry, type(dict())) or isinstance(entry, type(AttrDict())):
entry, type(AttrDict())
):
dictionary[key] = AttrDict.convert_dict(dictionary[key]) dictionary[key] = AttrDict.convert_dict(dictionary[key])
if isinstance(entry, type(list())): if isinstance(entry, type(list())):
dictionary[key] = [AttrDict.convert_dict(x) for x in entry] dictionary[key] = [AttrDict.convert_dict(x) for x in entry]

View File

@@ -1,4 +1,5 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""Parser superclass of specific parsers."""
class BaseParser(object): class BaseParser(object):
@@ -17,14 +18,14 @@ class BaseParser(object):
self.construct_parser() self.construct_parser()
def parse_file(self, file_content, start_line=0): def parse_file(self, file_content, start_line=0):
''' """
Parse assembly file. This includes *not* extracting of the marked kernel and Parse assembly file. This includes *not* extracting of the marked kernel and
the parsing of the instruction forms. the parsing of the instruction forms.
:param str file_content: assembly code :param str file_content: assembly code
:param int start_line: offset, if first line in file_content is meant to be not 1 :param int start_line: offset, if first line in file_content is meant to be not 1
:return: list of instruction forms :return: list of instruction forms
''' """
# Create instruction form list # Create instruction form list
asm_instructions = [] asm_instructions = []
lines = file_content.split('\n') lines = file_content.split('\n')

View File

@@ -12,6 +12,7 @@ class ParserAArch64v81(BaseParser):
self.isa = 'aarch64' self.isa = 'aarch64'
def construct_parser(self): def construct_parser(self):
"""Create parser for ARM AArch64 ISA."""
# Comment # Comment
symbol_comment = '//' symbol_comment = '//'
self.comment = pp.Literal(symbol_comment) + pp.Group( self.comment = pp.Literal(symbol_comment) + pp.Group(
@@ -181,8 +182,9 @@ class ParserAArch64v81(BaseParser):
Parse line and return instruction form. Parse line and return instruction form.
:param str line: line of assembly code :param str line: line of assembly code
:param int line_id: default None, identifier of instruction form :param line_number: identifier of instruction form, defautls to None
:return: parsed instruction form :type line_number: int, optional
:return: `dict` -- parsed asm line (comment, label, directive or instruction form)
""" """
instruction_form = AttrDict( instruction_form = AttrDict(
{ {
@@ -263,6 +265,12 @@ class ParserAArch64v81(BaseParser):
return instruction_form return instruction_form
def parse_instruction(self, instruction): def parse_instruction(self, instruction):
"""
Parse instruction in asm line.
:param str instruction: Assembly line string.
:returns: `dict` -- parsed instruction form
"""
result = self.instruction_parser.parseString(instruction, parseAll=True).asDict() result = self.instruction_parser.parseString(instruction, parseAll=True).asDict()
result = AttrDict.convert_dict(result) result = AttrDict.convert_dict(result)
operands = [] operands = []
@@ -292,6 +300,7 @@ class ParserAArch64v81(BaseParser):
return return_dict return return_dict
def process_operand(self, operand): def process_operand(self, operand):
"""Post-process operand"""
# structure memory addresses # structure memory addresses
if self.MEMORY_ID in operand: if self.MEMORY_ID in operand:
return self.process_memory_address(operand[self.MEMORY_ID]) return self.process_memory_address(operand[self.MEMORY_ID])
@@ -311,6 +320,7 @@ class ParserAArch64v81(BaseParser):
return operand return operand
def process_memory_address(self, memory_address): def process_memory_address(self, memory_address):
"""Post-process memory address operand"""
# Remove unnecessarily created dictionary entries during parsing # Remove unnecessarily created dictionary entries during parsing
offset = None if 'offset' not in memory_address else memory_address['offset'] offset = None if 'offset' not in memory_address else memory_address['offset']
base = None if 'base' not in memory_address else memory_address['base'] base = None if 'base' not in memory_address else memory_address['base']
@@ -333,11 +343,13 @@ class ParserAArch64v81(BaseParser):
return AttrDict({self.MEMORY_ID: new_dict}) return AttrDict({self.MEMORY_ID: new_dict})
def process_sp_register(self, register): def process_sp_register(self, register):
"""Post-process stack pointer register"""
reg = register reg = register
reg['prefix'] = 'x' reg['prefix'] = 'x'
return AttrDict({self.REGISTER_ID: reg}) return AttrDict({self.REGISTER_ID: reg})
def process_register_list(self, register_list): def process_register_list(self, register_list):
"""Post-process register lists (e.g., {r0,r3,r5}) and register ranges (e.g., {r0-r7})"""
# Remove unnecessarily created dictionary entries during parsing # Remove unnecessarily created dictionary entries during parsing
vlist = [] vlist = []
dict_name = '' dict_name = ''
@@ -354,6 +366,7 @@ class ParserAArch64v81(BaseParser):
return AttrDict({self.REGISTER_ID: new_dict}) return AttrDict({self.REGISTER_ID: new_dict})
def process_immediate(self, immediate): def process_immediate(self, immediate):
"""Post-process immediate operand"""
dict_name = '' dict_name = ''
if 'identifier' in immediate: if 'identifier' in immediate:
# actually an identifier, change declaration # actually an identifier, change declaration
@@ -378,11 +391,13 @@ class ParserAArch64v81(BaseParser):
) )
def process_label(self, label): def process_label(self, label):
"""Post-process label asm line"""
# remove duplicated 'name' level due to identifier # remove duplicated 'name' level due to identifier
label['name'] = label['name']['name'] label['name'] = label['name']['name']
return AttrDict({self.LABEL_ID: label}) return AttrDict({self.LABEL_ID: label})
def get_full_reg_name(self, register): def get_full_reg_name(self, register):
"""Return one register name string including all attributes"""
if 'lanes' in register: if 'lanes' in register:
return ( return (
register['prefix'] register['prefix']
@@ -394,19 +409,21 @@ class ParserAArch64v81(BaseParser):
return register['prefix'] + str(register['name']) return register['prefix'] + str(register['name'])
def normalize_imd(self, imd): def normalize_imd(self, imd):
"""Normalize immediate to decimal based representation"""
if 'value' in imd: if 'value' in imd:
if imd['value'].lower().startswith('0x'): if imd['value'].lower().startswith('0x'):
# hex, return decimal # hex, return decimal
return int(imd['value'], 16) return int(imd['value'], 16)
return int(imd['value'], 10) return int(imd['value'], 10)
elif 'float' in imd: elif 'float' in imd:
return self.ieee_to_int(imd['float']) return self.ieee_to_float(imd['float'])
elif 'double' in imd: elif 'double' in imd:
return self.ieee_to_int(imd['double']) return self.ieee_to_float(imd['double'])
# identifier # identifier
return imd return imd
def ieee_to_int(self, ieee_val): def ieee_to_float(self, ieee_val):
"""Convert IEEE representation to python float"""
exponent = int(ieee_val['exponent'], 10) exponent = int(ieee_val['exponent'], 10)
if ieee_val['e_sign'] == '-': if ieee_val['e_sign'] == '-':
exponent *= -1 exponent *= -1
@@ -416,16 +433,19 @@ class ParserAArch64v81(BaseParser):
raise NotImplementedError raise NotImplementedError
def is_gpr(self, register): def is_gpr(self, register):
"""Check if register is a general purpose register"""
if register['prefix'] in 'wx': if register['prefix'] in 'wx':
return True return True
return False return False
def is_vector_register(self, register): def is_vector_register(self, register):
"""Check if register is a vector register"""
if register['prefix'] in 'bhsdqv': if register['prefix'] in 'bhsdqv':
return True return True
return False return False
def is_flag_dependend_of(self, flag_a, flag_b): def is_flag_dependend_of(self, flag_a, flag_b):
"""Check if ``flag_a`` is dependent on ``flag_b``"""
# we assume flags are independent of each other, e.g., CF can be read while ZF gets written # we assume flags are independent of each other, e.g., CF can be read while ZF gets written
# TODO validate this assumption # TODO validate this assumption
if flag_a.name == flag_b.name: if flag_a.name == flag_b.name:
@@ -433,6 +453,7 @@ class ParserAArch64v81(BaseParser):
return False return False
def is_reg_dependend_of(self, reg_a, reg_b): def is_reg_dependend_of(self, reg_a, reg_b):
"""Check if ``reg_a`` is dependent on ``reg_b``"""
prefixes_gpr = 'wx' prefixes_gpr = 'wx'
prefixes_vec = 'bhsdqv' prefixes_vec = 'bhsdqv'
if reg_a['name'] == reg_b['name']: if reg_a['name'] == reg_b['name']:
@@ -443,4 +464,5 @@ class ParserAArch64v81(BaseParser):
return False return False
def get_reg_type(self, register): def get_reg_type(self, register):
"""Get register type"""
return register['prefix'] return register['prefix']

View File

@@ -13,6 +13,7 @@ class ParserX86ATT(BaseParser):
self.isa = 'x86' self.isa = 'x86'
def construct_parser(self): def construct_parser(self):
"""Create parser for ARM AArch64 ISA."""
decimal_number = pp.Combine( decimal_number = pp.Combine(
pp.Optional(pp.Literal('-')) + pp.Word(pp.nums) pp.Optional(pp.Literal('-')) + pp.Word(pp.nums)
).setResultsName('value') ).setResultsName('value')
@@ -148,6 +149,7 @@ class ParserX86ATT(BaseParser):
) )
def parse_register(self, register_string): def parse_register(self, register_string):
"""Parse register string"""
try: try:
return self.process_operand( return self.process_operand(
self.register.parseString(register_string, parseAll=True).asDict() self.register.parseString(register_string, parseAll=True).asDict()
@@ -160,8 +162,9 @@ class ParserX86ATT(BaseParser):
Parse line and return instruction form. Parse line and return instruction form.
:param str line: line of assembly code :param str line: line of assembly code
:param int line_id: default None, identifier of instruction form :param line_number: default None, identifier of instruction form
:return: parsed instruction form :type line_number: int, optional
:return: ``dict`` -- parsed asm line (comment, label, directive or instruction form)
""" """
instruction_form = AttrDict( instruction_form = AttrDict(
{ {
@@ -232,6 +235,12 @@ class ParserX86ATT(BaseParser):
return instruction_form return instruction_form
def parse_instruction(self, instruction): def parse_instruction(self, instruction):
"""
Parse instruction in asm line.
:param str instruction: Assembly line string.
:returns: `dict` -- parsed instruction form
"""
result = self.instruction_parser.parseString(instruction, parseAll=True).asDict() result = self.instruction_parser.parseString(instruction, parseAll=True).asDict()
result = AttrDict.convert_dict(result) result = AttrDict.convert_dict(result)
operands = [] operands = []
@@ -260,6 +269,7 @@ class ParserX86ATT(BaseParser):
return return_dict return return_dict
def process_operand(self, operand): def process_operand(self, operand):
"""Post-process operand"""
# For the moment, only used to structure memory addresses # For the moment, only used to structure memory addresses
if self.MEMORY_ID in operand: if self.MEMORY_ID in operand:
return self.process_memory_address(operand[self.MEMORY_ID]) return self.process_memory_address(operand[self.MEMORY_ID])
@@ -270,6 +280,7 @@ class ParserX86ATT(BaseParser):
return operand return operand
def process_memory_address(self, memory_address): def process_memory_address(self, memory_address):
"""Post-process memory address operand"""
# Remove unecessarily created dictionary entries during memory address parsing # Remove unecessarily created dictionary entries during memory address parsing
offset = None if 'offset' not in memory_address else memory_address['offset'] offset = None if 'offset' not in memory_address else memory_address['offset']
base = None if 'base' not in memory_address else memory_address['base'] base = None if 'base' not in memory_address else memory_address['base']
@@ -284,11 +295,13 @@ class ParserX86ATT(BaseParser):
return AttrDict({self.MEMORY_ID: new_dict}) return AttrDict({self.MEMORY_ID: new_dict})
def process_label(self, label): def process_label(self, label):
"""Post-process label asm line"""
# remove duplicated 'name' level due to identifier # remove duplicated 'name' level due to identifier
label['name'] = label['name']['name'] label['name'] = label['name']['name']
return AttrDict({self.LABEL_ID: label}) return AttrDict({self.LABEL_ID: label})
def process_immediate(self, immediate): def process_immediate(self, immediate):
"""Post-process immediate operand"""
if 'identifier' in immediate: if 'identifier' in immediate:
# actually an identifier, change declaration # actually an identifier, change declaration
return immediate return immediate
@@ -296,10 +309,12 @@ class ParserX86ATT(BaseParser):
return AttrDict({self.IMMEDIATE_ID: immediate}) return AttrDict({self.IMMEDIATE_ID: immediate})
def get_full_reg_name(self, register): def get_full_reg_name(self, register):
"""Return one register name string including all attributes"""
# nothing to do # nothing to do
return register['name'] return register['name']
def normalize_imd(self, imd): def normalize_imd(self, imd):
"""Normalize immediate to decimal based representation"""
if 'value' in imd: if 'value' in imd:
if imd['value'].lower().startswith('0x'): if imd['value'].lower().startswith('0x'):
# hex, return decimal # hex, return decimal
@@ -309,6 +324,7 @@ class ParserX86ATT(BaseParser):
return imd return imd
def is_flag_dependend_of(self, flag_a, flag_b): def is_flag_dependend_of(self, flag_a, flag_b):
"""Check if ``flag_a`` is dependent on ``flag_b``"""
# we assume flags are independent of each other, e.g., CF can be read while ZF gets written # we assume flags are independent of each other, e.g., CF can be read while ZF gets written
# TODO validate this assumption # TODO validate this assumption
if flag_a.name == flag_b.name: if flag_a.name == flag_b.name:
@@ -316,6 +332,7 @@ class ParserX86ATT(BaseParser):
return False return False
def is_reg_dependend_of(self, reg_a, reg_b): def is_reg_dependend_of(self, reg_a, reg_b):
"""Check if ``reg_a`` is dependent on ``reg_b``"""
# Check if they are the same registers # Check if they are the same registers
if reg_a.name == reg_b.name: if reg_a.name == reg_b.name:
return True return True
@@ -359,11 +376,13 @@ class ParserX86ATT(BaseParser):
return False return False
def is_basic_gpr(self, register): def is_basic_gpr(self, register):
"""Check if register is a basic general purpose register (ebi, rax, ...)"""
if any(char.isdigit() for char in register['name']): if any(char.isdigit() for char in register['name']):
return False return False
return True return True
def is_gpr(self, register): def is_gpr(self, register):
"""Check if register is a general purpose register"""
if register is None: if register is None:
return False return False
gpr_parser = ( gpr_parser = (
@@ -381,6 +400,7 @@ class ParserX86ATT(BaseParser):
return False return False
def is_vector_register(self, register): def is_vector_register(self, register):
"""Check if register is a vector register"""
if register is None: if register is None:
return False return False
if register['name'].rstrip(string.digits).lower() in ['mm', 'xmm', 'ymm', 'zmm']: if register['name'].rstrip(string.digits).lower() in ['mm', 'xmm', 'ymm', 'zmm']:
@@ -388,6 +408,7 @@ class ParserX86ATT(BaseParser):
return False return False
def get_reg_type(self, register): def get_reg_type(self, register):
"""Ger register type"""
if register is None: if register is None:
return False return False
if self.is_gpr(register): if self.is_gpr(register):

View File

@@ -1,4 +1,5 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""Semantics opbject responsible for architecture specific semantic operations"""
import warnings import warnings
from functools import reduce from functools import reduce
@@ -19,6 +20,12 @@ class ArchSemantics(ISASemantics):
# SUMMARY FUNCTION # SUMMARY FUNCTION
def add_semantics(self, kernel): def add_semantics(self, kernel):
"""
Applies performance data (throughput, latency, port pressure) and source/destination
distribution to each instruction of a given kernel.
:param list kernel: kernel to apply semantics
"""
for instruction_form in kernel: for instruction_form in kernel:
self.assign_src_dst(instruction_form) self.assign_src_dst(instruction_form)
self.assign_tp_lt(instruction_form) self.assign_tp_lt(instruction_form)
@@ -26,6 +33,11 @@ class ArchSemantics(ISASemantics):
self.set_hidden_loads(kernel) self.set_hidden_loads(kernel)
def assign_optimal_throughput(self, kernel): def assign_optimal_throughput(self, kernel):
"""
Assign optimal throughput port pressure to a kernel. This is done in steps of ``0.01cy``.
:param list kernel: kernel to apply optimal port utilization
"""
INC = 0.01 INC = 0.01
kernel.reverse() kernel.reverse()
port_list = self._machine_model.get_ports() port_list = self._machine_model.get_ports()
@@ -74,6 +86,7 @@ class ArchSemantics(ISASemantics):
kernel.reverse() kernel.reverse()
def set_hidden_loads(self, kernel): def set_hidden_loads(self, kernel):
"""Hide loads behind stores if architecture supports hidden loads (depricated)"""
loads = [instr for instr in kernel if INSTR_FLAGS.HAS_LD in instr['flags']] loads = [instr for instr in kernel if INSTR_FLAGS.HAS_LD in instr['flags']]
stores = [instr for instr in kernel if INSTR_FLAGS.HAS_ST in instr['flags']] stores = [instr for instr in kernel if INSTR_FLAGS.HAS_ST in instr['flags']]
# Filter instructions including load and store # Filter instructions including load and store
@@ -114,6 +127,7 @@ class ArchSemantics(ISASemantics):
# get parser result and assign throughput and latency value to instruction form # get parser result and assign throughput and latency value to instruction form
# mark instruction form with semantic flags # mark instruction form with semantic flags
def assign_tp_lt(self, instruction_form): def assign_tp_lt(self, instruction_form):
"""Assign throughput and latency to an instruction form."""
flags = [] flags = []
port_number = len(self._machine_model['ports']) port_number = len(self._machine_model['ports'])
if instruction_form['instruction'] is None: if instruction_form['instruction'] is None:
@@ -298,6 +312,7 @@ class ArchSemantics(ISASemantics):
instruction_form['latency_lcd'] = 0 instruction_form['latency_lcd'] = 0
def _handle_instruction_found(self, instruction_data, port_number, instruction_form, flags): def _handle_instruction_found(self, instruction_data, port_number, instruction_form, flags):
"""Apply performance data to instruction if it was found in the archDB"""
throughput = instruction_data['throughput'] throughput = instruction_data['throughput']
port_pressure = self._machine_model.average_port_pressure( port_pressure = self._machine_model.average_port_pressure(
instruction_data['port_pressure'] instruction_data['port_pressure']
@@ -334,14 +349,17 @@ class ArchSemantics(ISASemantics):
return throughput, port_pressure, latency, latency_wo_load return throughput, port_pressure, latency, latency_wo_load
def substitute_mem_address(self, operands): def substitute_mem_address(self, operands):
"""Create memory wildcard for all memory operands"""
# reg_ops = [op for op in operands if 'register' in op] # reg_ops = [op for op in operands if 'register' in op]
# reg_type = self._parser.get_reg_type(reg_ops[0]['register']) # reg_type = self._parser.get_reg_type(reg_ops[0]['register'])
return [self._create_reg_wildcard() if 'memory' in op else op for op in operands] return [self._create_reg_wildcard() if 'memory' in op else op for op in operands]
def _create_reg_wildcard(self): def _create_reg_wildcard(self):
"""Wildcard constructor"""
return {'*': '*'} return {'*': '*'}
def convert_op_to_reg(self, reg_type, reg_id='0'): def convert_op_to_reg(self, reg_type, reg_id='0'):
"""Create register operand for a memory addressing operand"""
if self._isa == 'x86': if self._isa == 'x86':
if reg_type == 'gpr': if reg_type == 'gpr':
register = {'register': {'name': 'r' + str(int(reg_id) + 9)}} register = {'register': {'name': 'r' + str(int(reg_id) + 9)}}
@@ -352,6 +370,7 @@ class ArchSemantics(ISASemantics):
return register return register
def _nullify_data_ports(self, port_pressure): def _nullify_data_ports(self, port_pressure):
"""Set all ports to 0.0 for the ports of a machine model"""
data_ports = self._machine_model.get_data_ports() data_ports = self._machine_model.get_data_ports()
for port in data_ports: for port in data_ports:
index = self._machine_model.get_ports().index(port) index = self._machine_model.get_ports().index(port)
@@ -381,6 +400,7 @@ class ArchSemantics(ISASemantics):
@staticmethod @staticmethod
def get_throughput_sum(kernel): def get_throughput_sum(kernel):
"""Get the overall throughput sum separated by port of all instructions of a kernel."""
tp_sum = reduce( tp_sum = reduce(
(lambda x, y: [sum(z) for z in zip(x, y)]), (lambda x, y: [sum(z) for z in zip(x, y)]),
[instr['port_pressure'] for instr in kernel], [instr['port_pressure'] for instr in kernel],