Development

Install the development environment

If you want to try agilepy’s new features that are not officially released yet, a develpoment environment called agilepy-environment is available into Anaconda cloud. It contains all the dependencies unless agilepy, which must be installed by hand cloning the repository.

Anaconda

conda config --add channels conda-forge
conda config --add channels plotly
conda create -n agilepydev -c agilescience agiletools agilepy-dataset
conda activate agilepydev
git clone https://github.com/AGILESCIENCE/Agilepy.git
cd Agilepy && git checkout develop
conda env update -f environment.yml
python setup.py develop

Docker

docker pull agilescience/agilepy-recipe:latest
mkdir shared_dir && cd shared_dir && git clone https://github.com/AGILESCIENCE/Agilepy.git \
&& cd Agilepy && git checkout develop

docker run --rm -it -p 8888:8888 \
-e DISPLAY=$DISPLAY \
-v /tmp/.X11-unix:/tmp/.X11-unix:rw \
-v $SHARED_DIR_PATH:/shared_dir \
agilescience/agilepy-recipe:latest

## -- Inside the container --
conda activate agilepydev
cd /shared_dir/Agilepy python setup.py develop

jupyter notebook --port 8889 --ip 0.0.0.0  --allow-root

Now you have the agilepy’s latest version installed in your environment.

Git flow

Branches

Two main branches:

  • master: contains only production releases.

  • develop: contains commits that will be included in the next production release.

Two support branches:

  • feature branch: each new feature (Trello’s card) should be developed in its own feature branch, branching from develop and merged back into it. The feature branch are not pushed into the remote.

  • hotfix branch: if an hotfix is needed it should be develop in its own branch, branching from master and merged back to it.

Git flow

Versioning

The master branch contains only production releases: when the develop branch (or hotfix branch) is merged to master a new release tag must be created. Its name follows the semantic versioning.

x.y.z

Incrementing:

  • x version when you make incompatible API changes,

  • y version when you add functionality in a backwards compatible manner, and

  • z version when you make backwards compatible bug fixes.

Branches names

The master and the develop branch have an infinite lifetime, hence their name is fixed.

The feature branch takes the following format:

feature-#<card-number>-<short-description>

e.g. feature-#61-new-cool-feature

The hotfix branch name takes the following format:

hotfix-#<card-number>-<release-number>

e.g. hotfix-#57-1.0.0

The release number is the one of the production release from which it originates from.

Getting started

Development of a new feature

Create a new feature branch:

git checkout develop
git pull origin develop
git checkout -b feature-#61-new-cool-feature develop

Development and testing of the new feature.

When you have finished, update the CHANGELOG.md and commit your changes.

vim CHANGELOG
git commit -m "feature-#61-new-cool-feature done"

In the meantime it is possible that someone else have pushed his work into the develop branch. In this case you have to merge the changes in your feature branch.

git pull **origin** develop

Finally you can merge your feature branch back to develop branch.

git merge --no-ff feature-#61-new-cool-feature
git branch -d feature-#61-new-cool-feature
git push origin develop

Add configuration parameters

Let’s say we want to add the following configuration section to the AGAnalysis’ configuration file.

ap:
    radius: 0.25
    timeslot: 3600

  • Add the new section to the AGAnalysis.getConfiguration() method.

  • Add the type of the configuration parameters within the AGAnalysisConfig.checkOptionsType() method (in the corresponding lists).

  • If the parameters need some kind of validation (this is not the case), add a new method in ValidationStrategies and call it within the AGAnalysisConfig.validateConfiguration() (check examples).

  • If the parameters need some kind of transformation (this is not the case), add a new method in CompletionStrategies and call it within the AGAnalysisConfig.completeConfiguration() (check examples).

  • Add the new configuration section to all the unit test configuration files.

  • Document the new configuration parameters within the manual/configuration_file.rst file.

Add a new science tool

Let’s say we want to add a new (c++) science tool: AG_ap.

  • Add a new class within the api/ScienceTools.py script. You need to implement some abstract methods.

  • You can use the new class as follows:

apTool = AP("AG_ap", self.logger)
apTool.configureTool(self.config)
if not apTool.allRequiredOptionsSet(self.config):
    raise ScienceToolInputArgMissing("Some options have not been set.")
products = apTool.call()

Release of a new version

Change the version of the software in setup.py. The version increment must be take in account all the commits of the develop branch. You can check the CHANGELOG.md to facilitate this process. Please, add the new tag within the CHANGELOG.md file.

git checkout master
git merge --no-ff develop
git tag -a <new-tag>
git push origin <new-tag>

DevOps

A high level description of agilepy’s devops is in the image below:

Git flow

This scheme workflow produces three images:

  • base_image: It’s an image with all the dependencies except Agilepy python library, it’s used for developing purposes only by developers. Base image is built after a new commit in agilepy-recipe repository.

  • latest code image: It’s the base_image with Agilepy’s develop branch at latest commit, useful for using or testing agilepy’s updates not officially released. This image is not supported nor stable and is built by dockerhub after github’s testing pipelines are successful.

  • released image: The base_image with Agilepy’s release tag. By default the community shall be download this image. It’s built when a new tag is created.