12. Contributing: Developer’s Guide

Everybody is encouraged to contribute to the RAFCON project. However, collaboration needs some guidelines. We try to collect all this information in this document, so please stick to this.

When contributing to this repository, please first discuss the change you wish to make via issue, email, or any other method with the owners of this repository before making a change.

Please note that we have a code of conduct, please follow it in all your interactions with the project.

Please check our code style, before making any changes to the code. Please read the commit guidelines, before submitting any commit.

12.1. Getting Started

For an introduction of how to install RAFCON as a user, have a look at the website. Also, to get the correct dependencies, follow the instruction given on this site.

The following describes how to get the latest RAFCON version per GitHub.

First, change to the directory in which you want to clone RAFCON:

$ cd /some/personal/path/

Next, clone the RAFCON repository. You can either use the HTTPS URL:

$ git clone https://github.com/DLR-RM/RAFCON.git

or the SSH URL:

$ git clone git@github.com:DLR-RM/RAFCON.git

This must of course only be done once. If you want to get the latest commits after you have cloned the repository, use

$ cd /some/personal/path/rafcon
$ git pull

In order to run RAFCON from the local code base, you have to setup the environment:

$ export PYTHONPATH=/some/personal/path/rafcon/source:$PYTHONPATH
$ export PATH=/some/personal/path/rafcon/bin:$PATH

Now you can run rafcon to start the RAFCON-GUI or just run rafcon_core to only launch the core. Hereby, rafcon just links to the file /some/personal/path/rafcon/source/rafcon/gui/start.py and rafcon_core points to /some/personal/path/rafcon/source/rafcon/core/start.py, so you could also call these files directly.

IMPORTANT: If you start rafcon for the first time start it this way:

$ RAFCON_CHECK_INSTALLATION=True rafcon

This will install all fonts and gtk-styles.

12.2. Install RAFCON from Sources

RAFCON can be also installed from our GitHub releases.

If you don’t want to edit the source code of RAFCON, it can be installed directly from source:

pip install /install/directory/rafcon/ --user

If you want to be able to change the source code, you can install RAFCON in editable mode.

pip install --editable /install/directory/rafcon/ --user

Any changes in /install/directory/rafcon/source will take effect when launching RAFCON.

12.3. Running and writing tests

Running tests with tox

The simplest and most reliable way of running the tests is using tox. If you have not installed tox, do so using

$ pip install tox

Then, in the simplest case you just call tox in the root directory of the RAFCON repository:

$ tox

This will run the following environments:

  • py27, py3[4-7]: Runs the test using the according Python interpreter

  • coverage: Runs the tests using Python 2.7 with a coverage report

  • docs: Builds the documentation and verifies all links

  • check: Verifies the sdist and wheel file

Specific environments can be run with the -e option:

$ tox -e 2.7,3.4
$ tox -e docs

When running the tests (py27, py3[4-7] or coverage), you can pass custom options to pytest by listing them after tox [tox options] --. The default pytest options are -vx -m "(core or gui or share_elements) and not unstable".

$ tox -e 2.7 -- -x -m "core"
$ tox -- -s -k "test__destruct"

Tox creates a virtualenv for each environment, based on the dependencies defined in pyproject.toml and tox.ini. These are only generated on the first run of an environment. If the dependencies change, you need to tell tox to recreate the environments using the -r/--recreate option:

$ tox -re py2.7

The RAFCON tests are annotated with a number of markers, allowing you to select specific tests:

  • core, gui, share_elements, network: Tests located in a folder with that name

  • unstable: GUI tests that do not run very reliable (e.g. because of the window manager)

Pytest allows you to select tests based on markers using the -m option. Markers can be combined using not, and, or and paranthesis:

$ tox -e 2.7 -- -x -m "gui and not unstable"

Writing tests

RAFCON provides a lot of tests in the tests/ folder. Many of these tests are integration tests, unit tests are unfortunately often missing. If a test only uses imports from rafcon.core, it is to be placed in tests/core/, otherwise in tests/gui/.

RAFCON uses pytest as testing framework. It e.g. auto detects your test files starting with test_*. Please have a look at the documentation before writing tests: https://pytest.org/

GUI tests

When you want to write an integration test using the GUI, a custom fixture named gui is provided (tests/gui/conftest.py). Simply add gui as parameter to your test (no import is required for tests residing beneath test/gui/). The fixture automatically starts the GUI before the test and closes it thereafter.

Important: Do not import any module from rafcon.gui outside of a function! Otherwise, models might be created withing the wrong thread, which leads to gtkmvc forwarding observer notifications asynchronously (via idle_add) instead of synchronously.

When calling an operation that causes changes (in the core, models, GUI), you need to add the operation to the GTK queue and wait until the operation has finished. This is simply done by calling gui(function_reference, *args, **kwargs) instead of function_reference(*args, **kwargs).

If your test commands causes any logger.warning or logger.error, you need to specify the expected numbers. Do so by calling gui.expected_warnings += 1, respectively gui.expected_errors += 1, directly after the command that causes the warning/error.

The fixture will load the default core and gui config options and the libraries generic and unit_test_state_machines. If you want to override certain settings or add more libraries, use the following decorator:

@pytest.mark.parametrize('gui', [{
    "gui_config": {
        'AUTO_BACKUP_ENABLED': True,
        'HISTORY_ENABLED': True
    },
    "libraries": {
        "ros": os.path.join(testing_utils.EXAMPLES_PATH, "libraries", "ros_libraries"),
        "turtle_libraries": os.path.join(testing_utils.EXAMPLES_PATH, "libraries", "turtle_libraries")
    }
}], indirect=True, ids=["with history, auto backup, ros and turtle libraries"])
def test_name(gui):
    pass  # test code

Using the ids argument, you can specify a label for your configuration. Other possible keys are core_config (dict), runtime_config (dict) and with_gui (bool, for tests that operate on models but do not require the controllers and views). It is also possible to combine this with parameter sets:

config_options = {
    "gui_config": {
        'HISTORY_ENABLED': True
    }
}
@pytest.mark.parametrize("gui,state_path,recursive,rel_size", [
    (config_options, state_path_root, False, (40, 40)),
    (config_options, state_path_root, True, (40, 40)),
    (config_options, state_path_P, False, (20, 20)),
    (config_options, state_path_P, True, (20, 20)),
], indirect=["gui"])
def test_name(gui, state_path, recursive, rel_size, monkeypatch):
    pass  # test code

Note that in this case, you need to set the indirect parameter to ["gui"].

The gui fixture offers some features:

  • if you want to restart the GUI within a test, call gui.restart()

  • the fixture provides shorthand access the gui singletons via gui.singletons and core singletons via gui.core_singletons, without requiring any further imports.

  • if you want to run a test after the GUI was closed, you can set the function to be run via gui.post_test = functools.partial(function_reference, *args, **kwargs)

12.4. Helper Function: Convert pixels to cairo units

This function can be used when developing with cairo:

def pixel_to_cairo(self, pixel):
    """Helper function to convert pixels to cairo units

    The conversion is depending on the view. The critical parameter is the current zooming factor. The equation is:
    cairo units = pixels / zoom factor

    :param float pixel: Number of pixels to convert
    :return: Number of cairo units corresponding to given number of pixels
    :rtype: float
    """
    zoom = self.get_zoom_factor()
    return pixel / zoom

12.5. Code of Conduct

Our Pledge

In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.

Our Standards

Examples of behavior that contributes to creating a positive environment include:

  • Using welcoming and inclusive language

  • Being respectful of differing viewpoints and experiences

  • Gracefully accepting constructive criticism

  • Focusing on what is best for the community

  • Showing empathy towards other community members

Examples of unacceptable behavior by participants include:

  • The use of sexualized language or imagery and unwelcome sexual attention or advances

  • Trolling, insulting/derogatory comments, and personal or political attacks

  • Public or private harassment

  • Publishing others’ private information, such as a physical or electronic address, without explicit permission

  • Other conduct which could reasonably be considered inappropriate in a professional setting

Our Responsibilities

Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.

Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.

Scope

This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.

Enforcement

Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at rafcon@dlr.de. All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.

Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project’s leadership.

Attribution

This Code of Conduct is adapted from the Contributor Covenant, version 1.4, available at https://www.contributor-covenant.org/version/1/4/ <https://www.contributor-covenant.org/version/1/4/>`__

12.6. Bugs & Feature request

Please use GitHub Issues to report bugs. This page can also be used for feature requests.

12.7. Code style

Style guides

We follow the rules PEP 8 (Style Guide for Python Code) and the DLR Coding Conventions with some minor exceptions:

  • Line length is limited to 120 characters (instead of 79)

  • no __version__ in header (except rafcon/__init__.py)

For docstrings (and generally for the documentation), we format using reStructuredText for Sphinx. More information on this is available in another PEP 287.

PyCharm Coding Style/Conventions

Our recommended tool for development is PyCharm. This IDE makes it easy for you to follow our style guides. We prepared a settings file for “Code styles” and “Inspection profiles”, which you just need to import: File > Import Settings > [project root/.idea/recommended_code_style_and_inspections.jar.

12.8. Git commit guidelines

General Git commit conventions

Git is used as our version control system. Please keep the following points in mind when committing to the repository:

  • if you are not a core developer, all changes must be bundled in pull requests

  • therefore, a new branch is required for new features and bug fixes

  • try to keep pull requests small, this eases the review and will speed up the merge of the branch

  • before submitting a pull request, make sure that all tests are passed

  • new features require new unit tests

  • each functional/logical change gets its own commit (try to keep them small)

  • no excessive use of logger.debug outputs (in commits) and never commit print commands

Git commit messages

When looking at the Git commit history, it should be easy to see what changes have been performed. This is why it is important to have good commit messages.

What to do:

  • Use imperative (Add file …)

  • First line is the caption of the commit (should be less than 72 chars)

    • summarizes the commit

    • mentions which code is affected (e.g. by stating the changes module/class)

  • Second line is blank

  • Following lines give a longer description and list changes in detail (use “- ” or “* ” as bullet points)

    • Why is the change necessary

    • How does it address the issue

    • Possible side effects

    • Give Issue/Feature-ID of GitHub Issues

      • Clos[e[s]]|Fix[e[s]]|Resolve[e[s]] #1234567 – use one of the keywords to automatically close an issue

      • Relates to issue #1234567 – state issue id also when the issue is not supposed to be closed

What not to do:

  • Try to avoid using the -m <msg>/--message=<msg> flag to git commit

  • Do not assume that the reader knows the details about bugs, features or previous commits

  • Neither assume that the reader looks at the commited code

Setting up your system

Add this line to your ~/.vimrc to add spell checking and automatic wrapping to your commit messages.

autocmd Filetype gitcommit setlocal spell textwidth=72

Explaining examples

This is a good example for a commit message:

Example: Capitalized, short (<72 chars) summary

More detailed explanatory text. Wrap it to about 72 characters. Think
of first line as the subject of an email and the rest of the text as
the body. The blank line separating the summary from the body is
critical (unless you omit the body entirely); tools like rebase can get
confused if you run the two together.

Further paragraphs come after blank lines.

- Dash '-' and asterisk '*' are both fine, followed by a space
- Use a hanging indent

Th following link shows some bad examples: https://wiki.openstack.org/wiki/GitCommitMessages#Some_examples_of_bad_practice

Sources

12.9. Steps for releasing

Steps to perform, when releasing a new version of RAFCON (this section is only for releasing the tool inside our institute):

  1. Decide about the new version number

Should the release be a patch or minor release? Or even a major release? Check the latest version number and adjust it appropriately.

  1. Create a release Branch

Create a new branch from the latest commit you want to include into the new release. Optionally, if there are new untested feature, which you don’t want to include into the release, first go back to a previous commit:

$ git checkout [hash]

Then, create the new branch and check it out:

$ git checkout -b release-[new version number]

  1. Fix relevant issues (optional)

Within your new branch, you shouldn’t integrate any new features, but only solve issues. The develop and feature-xy branches should be used for new features. Take your time to ensure that the most critical issues are fixed and RAFCON is running stable.

  1. Create a test state machine for the new version (only minor/major releases)

For each minor release, a state machine must be created to ensure backwards compatibility using a special test.

Therefore, open the state machine in [project directory]/tests/assets/unit_test_state_machines/backward_compatibility/[latest version number] and save it to the same folder with the correct version as library name. Commit your changes.

  1. Check tests

Run all tests and verify that they do all pass. If not, fix them! Also check the BuildBot. Commit your changes.

  1. Check the changelog

Open [project directory]/doc/Changelog.rst and verify that all changes are included within the correct version number. Compare with git log and the latest closed issues on GitHub. Commit your changes.

  1. Build style files

Build *.css files from *.scss files.

$ ./compile_scss.sh
$ git add share/themes/RAFCON/gtk-3.0/*.css --force

  1. Apply the version number

  1. If the dev dependencies have not yet been installed via pdm, then run pdm install --dev --no-editable

  2. Update the version number by running pdm run bump2version [major / minor / or patch]

  3. Update the date-released in [project directory]/CITATION.cff.

  4. Run cffconvert --format zenodo --outfile .zenodo.json (see “Making software citable”, requires Python 3)

  5. Commit and push your changes.

  1. Merge to master

When everything is prepared, you can merge the release branch into the master branch:

$ git push release-[new version number]
$ git checkout master
$ git pull master
$ git merge release-[new version number]
$ git push

  1. Merge to develop

Merge all changes back into the develop branch:

$ git checkout develop
$ git pull
$ git merge release-[new version number]]
$ git push

  1. Publish new release to PyPi

Create a new distribution file and publish it on PyPi:

$ rm dist/*
$ pdm build
$ twine upload dist/*

  1. Publish to GitHub

Publish the changes to GitHub and GitHub Enterprise (assuming github is your GitHub remote name):

$ git push github
$ git checkout master
$ git push github

Make a release on GitHub by navigating to https://github.com/DLR-RM/RAFCON/releases/new. Enter the new version number in the “Tag version” field. Optioanlly add a release title and decription. Click “Publish release”.

  1. Force build of GitHub pages

Push an empty commit to the gh-pages branch:

$ git checkout gh-pages
$ git commit -m 'rebuild pages' --allow-empty
$ git push
$ git push github

12.10. Translating RAFCON

RAFCON is prepared for internationalization (i18n) and in fact ships with a basic German translation. This translation can be extended and new translations can be added. We are happy about your support here!

Add translatable strings

In order to allow for translating strings in RAFCON, they need to be marked in the source code. All you need to do for this is wrapping it in the gettext function _(), e.g. _("Hello world").

Strings in glade files do not need to be marked, but the label property needs an attribute translatable="yes":

<object class="GtkButton">
    <property name="label" translatable="yes">Translatable button label</property>
    ...
</object>

Generating a translation template

If new translatable strings have been added or you want to create translations for a new language, a POT file (Portable Object Template) has to be created:

cd /dir/to/rafcon/repository
./bin/i18n-gen-msg

This will create a rafcon.pot file in the source/rafcon/locale folder.

Updating an existing translation

If you want to update a translation, open the according PO file, located in source/rafcon/locale (e.g. de.po) with Poedit. Then go to Catalog => Update from POT file and select the generated rafcon.pot file. This step is optional, if no new translatable strings have been added.

With Poedit, you can add new translations for strings comfortably. Alternatively, you can directly edit the PO files with the text editor of your choice.

Creating a new translation

Open Poedit. Go to File => New From POT/PO File… and enter the name of the new language. Start adding your translations. When you are done, store the file in source/rafcon/locale with the language’ locale as name, e.g. de.po for the German translation.

12.11. HowTo on creating a splashscreen

This is a guide on creating a splash screen picture using the GIMP template.

Location The template file is located in source/rafcon/gui/assets/themes/templates. The standard place for images to be loaded into the startup splashscreen is source/rafcon/gui/assets/splashscreens.

File The template is a .xcf which is the native project format of GIMP. This guide is also using GIMP.

Process

  1. Open the template with GIMP.

  2. First select single window mode: Windows -> Single Window Mode. This will save you a lot of time and struggle.

  3. Check if the Layer widget is present. It should contain five items, one layer Background and four RAFCON_logo layers. If not present, press Ctrl + L.

  4. Now go to File -> Open as Layers and select a picture of your choice.

  5. Check if the image is placed between Background and RAFCON_Logo in the Layer widget. If not, drag it in the correct position.

  6. Now select the layer with your picture in the Layer widget.

    • Go to the Tools widget and select the Scale tool.

    • Click on your picture with the Scale tool and fit it in the 570x320px field.

    • If your not satisfied with your result, try the “Move” tool and move the layer around.

  7. Notice the small eye in the Layer widget next to a layer? Activate the visibility of a logo that serves your needs the best.

  8. If everything is done, ignore the invisible layers and click on File -> Export As. Export the picture as whatever python is able to load into a pixelbuffer, .jpg and .png work fine.

  9. Voila, there is your normed splash screen!