.. note:: This is a WIP Beta version. All content is subject to change.

.. _developer_guide:

===============
Developer Guide
===============


.. image:: ../images/components/component_overview_v2.png
    :alt: Schematic component overview


This section will teach you how to develop custom component implementations and integrate them into CIRCA's approximation flow. The covered topics are:

* `Basics`_
* `Important data structures and classes`_
* `Exchangeable components`_
* `Extensible components`_

You can find more detailed technical information on the framework's modules in the :ref:`module index <modules>`.


Basics
======

CIRCA is designed to be: 
	
	* *General*: The framework should not be restricted to certain circuit types, error metrics, approximation and search techniques, or specific target technologies. 

	* *Modular*: The framework architecture should enable the exchange of certain processing steps without affecting other steps. Modularity is key for the evaluation and the comparison of different techniques under a consistent experimental setup.

	* *Compatible*: The framework, in particular its inputs and outputs, should connect to other, widely-used academic and commercial front-end and back-end tools, e.g., tools synthesizing circuits for ASIC or FPGA technology.

	* *Extensible*: The framework should facilitate the swift implementation and evaluation of new techniques.

To achieve this, CIRCA's approximation flow is handled by independent stages and processing blocks (short: components) which communicate using well- defined interfaces. Each component can be extended by custom implementations and provides a specific piece of functionality which is needed for the approximation process. Integrating your own component implementations allows you to easily test different search strategies, approximation techniques, and quality assurance methods.

The framework provides the necessary structure for the components to work independently. CIRCA utilizes data structures as well as classes which can be accessed and interpreted by all components, enabling them to work together effectively.

.. todo:: 
	
	* Depict the dev. flow and then for each stage a close-up

Component implementation
------------------------

The customizable components of CIRCA are classes, implementing a specific interface. The interfaces are specified by *abstract base classes* from which the custom implementations must be derived. They were implemented using Python's `Abstract Base Classes`_ module, so their subclasses have to implement some methods in order to be instantiatable. The concept of abstract base classes ensures that new implementations provide the functions used by CIRCA in its standard approximation flow.

To create a custom component implementation, simply create a subclass of the abstract base class for that component and make sure to implement all of the base class's abstract methods (note that it is possible to derive from an already existing subclass to minimize implementation effort).

Each abstract base class provides a dictionary ``SUPPORTED_SUBCLASSES``. The key corresponds to the method's name used in the configuration file, e.g., *AnnotatedCandidates*. The value is a lamda expression, calling the constructor of the corresponding implementation. Once an implementation is finished, i.e., the subclass, add the key-value pair to the dictionary to make the implementation available.

.. When the class is finished, its name or ID must be added to some static data structure of the abstract base class so that it can be recognized when it is found in the configuration file. The data structure and requirement of an ID depends on the component.

.. todo:: Add an example of an implementation of a subclass of an ABC?

Component classification
------------------------

We differentiate between two types of customizable components: `exchangeable components <Exchangeable components_>`_ and `extensible components <Extensible components_>`_.

	* Exchangeable components are the stages and processing blocks *Input*, *QualityAssurance*, *Estimation*, *Search*, and *Output*. Exactly one implementation of each component is used when CIRCA is executed. The used implementations have a great impact on the approximation process and on the quality of the outcome.

	* Extensible components can be seen as functionality which is either used or not used in an approximation process. Extensible components include *approximators* from the *Approximation* block and *ErrorMetrics*. While the number of exchangleable components in an approximation process is restrricted to exactly one, the user has no upper limit on the number of employed extensible components.


Important data structures and classes
=====================================

In order to implement your custom components, you need to know the data structures, classes, and interfaces you will be working with. Since most of them are already documented extensively in the code, their implementation details will be omitted here. Instead, to improve the understanding of the objects, an overview of the scope of the particular object is given from a high-level perspective.

GeneralInformation
------------------

The ``GeneralInformation`` class acts as a container for information used by several components and is instantiated once in the *Input* stage. After instantiation, it is passed to every other stage when they are created.

The ``GeneralInformation`` object's content includes paths to the configuration and input circuit files, information on the input circuit, and references to the data structures *Candidates*, *Variants*, as well as nodes and circuits.

.. It also offers a public :ref:`method <information>` to extract primary input and output information from the input circuit automatically. This method must be called before the ``GeneralInformation`` instance's ``pi`` and ``po`` attributes contain a value other than ``None``.

When creating an *Input* component implementation, your job is to setup the ``GeneralInformation`` instance and usually also extract PI/PO (primary input/primary output) information, either manually or using the existing method. The PI/PO information is needed, for example, in the quality assurance step. For the automatic extraction of PI/PO information from Verilog code, ``GeneralInformation`` provides a method. 

Candidates
----------

A *candidate* is a part of the input circuit which can be replaced by an approximate component. Usually, the arithmetic units in the data path of a circuit are specified as candidates. The selection of candidates in a circuit is crucial since the selection of candidates directly influences the quality of the approximated outcome.

Candidates are represented by the ``Candidate`` class. Each ``Candidate`` instance must have a unique name and specify local quality constraints, i.e., local error metrics with error bounds. The local quality constraints of a candidate Every candidate has a reference to its original implementation (a circuit file containing only the candidate from the input circuit). From the candidate's original implementation, the candidate's variants are generated and stored in a candidate-specific directory.

A candidate can represent different parts of a circuit, e.g., a register, a wire, or an entire circuit module. Depending on the component or part of the circuit represented by the candidate, the candidate may have to be treated differently and may offer different methods. Thus, the ``Candidate`` class is abstract and cannot be instantiated. A specific implementation of a candidate is described in a subclass, derived from the ``Candidate`` class. In this way, CIRCA does not restrict candidates to a specific parts of a circuit. Furthermore, the candidate's type can be used to derive a unique name for the candidates.

All candidates are stored in a ``CandidateSet``, which is a basic data structure containing the ``Candidate`` instances and the path to the directory in which the candidate files are stored. It is accessible through the ``GeneralInformation`` instance. Candidates can be added and removed by all components.

Variants
--------

A *variant* is an unique instance of a candidate with a certain degree of approximation applied. The relation of a variant to its candidate can be compared to the relation between an object and its class. The candidate specifies the part of the input circuit to approximate, at least one approximation method and some quality constraints. The variants are the resulting circuits, each having only one approximation method and possibly adjusted quality constraints.

Variants do not need a name, since they generate a unique ID value based on their candidate's name, their approximation method, and their local quality constraints. Since the ID can become very long, a hash value is generated which can be used instead. The ``Variant`` instances have a reference to their candidate and to their circuit file.

``Variant`` objects can be created in two ways: A ``Candidate`` instance can generate its own original variant, which is basically a copy of the candidate's part of the input circuit. Each ``Variant`` instance can then generate its "neighboring" variants by either switching the approximation method or increasing one error bound by one step. These variants are called *neigbors* or *children* of the variant generating them. The variants created this way do not have a circuit file however. They must be chosen for the *Approximation* stage where these files are created.

All variants are stored in a ``VariantSet``, a data structure similar to the ``CandidateSet`` with a slightly different internal structure. It can also be accessed through the ``GeneralInformation`` instance and manipulated by all stages. Throughout the approximation process, the *Approximation* stage creates new variants and adds them to the ``VariantSet``; hence, the set is continuously extended. This can be seen as an on-the-fly generation of an approximate component library. That is, if an existing approximate component library is loaded into CIRCA, it would either represent or even replace the ``VariantSet``. 

*Note that a variant's ID (or hash) is unique. Thus, each variant is only created once in an approximation process which improves efficiency.*


Nodes and Circuits
------------------

In the approximation process, approximated variants of all specified candidates are generated. Then, CIRCA replaces the original implementation of a candidate with its approximated variants to create an approximated circuit. These *approximated circuits* are represented by instances of the ``Circuit`` class. An instance holds information about the circuit file and the result of the verification, i.e., whether the circuit's quality has been validated by the *QualityAssurance* stage and whether the circuit has passed the check.

All ``Circuit`` instances are stored in an ``ApproximatedCircuits`` instance representing the directory in which the circuit files are stored. A circuit can be stored in Verilog and in BLIF format.

An approximated circuit is uniquely described by the combination of variants used to implement the candidates. We denote this combination as *circuit configuration*. Since the search space is modeled as a graph structure, we identify, in the context of the search, configurations with ``Node``\ s. A ``Node`` is a representation of a circuit in another context, providing different information and functionality. Once a ``Node``'s circuit has been generated, the corresponding ``Circuit`` instance and the ``Node`` are connected by bidirectional references. The ``Node`` class specifies the interface for traversing the search space and working with circuits without having to provide detailed circuit information on this abstract level. The components use ``Node``\ s as input and output of their functions, i.e., as an interface.

A ``Node`` has a method for generating its children, similar to the ``Variant`` class. The children are those nodes, providing a circuit configuration with only one candidate implementing a different variant than the parent and this variant has a local error bound increased by one step compared to the parent's variant. For example, consider a node with the configuration {C\ :sub:`1, WC1`; C\ :sub:`2, WC3`}, C\ :sub:`1, WC1` being a candidate's variant with an worst-case error bound of *1* and C\ :sub:`2, WC3` a candidate's variant with a worst-case error bound of *3*. Then, the children of this node are: {C\ :sub:`1, WC2`; C\ :sub:`2, WC3`} and {C\ :sub:`1, WC1`; C\ :sub:`2, WC4`} (assuming a step-size of *1*). Note that this means that a ``Node`` instance could generate a child ``Node`` with the same configuration as its parent, by changing a candidate to its previous variant. Avoiding these circles is part of the *Search* stage component and can be accomplished using the ``Node``'s ``id`` attribute.

There is no central data structure storing all ``Node``\ s as this is considered to be the *Search* component's task and many nodes are likely to be never actually used. Make sure to look into the ``Node`` documentation if you want to implement your own stage component.

.. _dev_guide_exc_comps:

Exchangeable components
=======================

Exchangeable components share many properties which must be considered when developing a custom implementation. Here you can find the features they have in common and detailed documentation of all five components.

Shared features
---------------
All abstract base classes of exchangeable components provide a ``factory`` function which is called for component instantiation. A ``factory`` function takes the config. file as an input and uses the ``Method`` parameter from the corresponding config. section to instantiate the correct subclass. 

All supported implementations are listed in the base class's ``SUPPORTED_SUBCLASSES`` attribute, a dictionary storing the names of all available implementations as keys and constructor references as values, using lambda expressions. The function also extracts all key-value pairs from the component's config section and passes them to the constructor, stored in a dictionary as well. 

So, when creating a new component implementation, the first step is to add a new entry to the ``SUPPORTED_SUBCLASSES`` dictionary

.. code-block:: python

    "MyComponent": lambda s: MyComponent(s)

where ``MyComponent`` is the name of your class and ``MyComponent(s)`` is the constructor of the class, receiving the key-value pairs of the config. section. Also make sure that your ``__init__`` method has the correct signature with only the config. option dictionary as parameter:

.. code-block:: python

    class MyComponent(ComponentABC):
        def __init__(self, settings_dict):

Note that ``ComponentABC`` can be the component's abstract base class or one of the existing subclasses, if you want to base your implementation on something more substantial.

The exchangeable components also share an optional ``setup`` method. The ``setup`` function is called immediately after instantiation of the component and can be used to prepare the component for the approximation process. The function takes - at least - the ``GeneralInformation`` object as input and is used to perform further setup steps.

Component details
-----------------
.. toctree::
    :maxdepth: 1

    Input <dev_guide_input>
    QualityAssurance <dev_guide_qa>
    (Estimation) <dev_guide_estimation>
    Search <dev_guide_search>
    Output <dev_guide_output>


.. _dev_guide_ext_comps:

Extensible components
=====================

Extensible components have very little in common since they are used for very specialized purposes and not in a generalized manner like the exchangeable stage components. They are used on the same level as many of the specialized data structures which cannot be customized (but may be made customizable in the future). You can create as many custom implementations for these as you like to enrich the framework with new possibilities and use them independently with built in or custom component implementations. However, it can be useful to create approximation methods or error metrics that are especially suitable to use in conjunction with a specific custom component.

.. toctree::
    :maxdepth: 1

    Approximators <dev_guide_approximators>
    ErrorMetrics <dev_guide_error_metrics>


Utils
=====

The CIRCA framework provides some utility modules for working with the configuration file,
generating log messages and accessing the file system and OS functions.

.. todo:: Add references to module documentations

Configuration utils ``cfg_util``
--------------------------------
The ``cfg_util`` module provides a configuration file parser ``CfgParser`` to easily get option
values or dictionaries of options and their values from a whole section of the config file. It also
includes a parser function ``parseQualityConstraintSpecifier`` to parse the values from a quality
constraint specification string as defined in the :ref:`user guide's subsection about the configuration file <quality_constraint_syntax>`


Logging utils ``logutils``
--------------------------
CIRCA comes with a relatively extensive logging functionality that lets you write logging output to
files or to the console with five priority levels. Log messages to the console are also colored
based on their level. To include logging in a custom module, simply include this code in the
module:

 .. code-block:: python

    import logging
    from circa.utils.logutils import IndentationAdapter
    log = logging.getLogger("circa.base.<your_module_name>")
    log = IndentationAdapter(log, offset=7)

Where ``<your_module_name>`` is the name you want to appear as source of the log message. You can
adjust the ``offset`` parameter of the ``IndentationAdapter`` to increase or decrease the amount of
indentation that is prepended to the message. For writing a log message, use one of the following
functions:

 .. code-block:: python

    log.debug("Message of low importance, only for debugging")
    log.info("General message for the user, can be ignored safely")
    log.warning("Something is not right but the process will continue")
    log.error("Something unexpected happened that might lead to a crash")
    log.critical("Something very bad happened and the process will be stopped")

.. Maybe add these later
.. Shell utility ``shutil2``


.. Constants ``constants``



.. _Abstract Base Classes: https://docs.python.org/3/library/abc.html