Version 4 (modified by minondoa, 5 years ago) (diff)



The aim of this section will be to explain the process we follow to contribute to the development of the DRM4G.

We use something similar to a gitflow workflow. Internally we synchronize our work with GitBucket, which is what will be considered as the central repository of the Santander Meteorology Group.

  • You can find the project here, but to be able to access it you must first get a GitHub account and get access to the private repository.

Necessary Steps

There are a lot of tutorials showing how to use git, including our own, but this is more straightforward explanation on how to get started.

In Linux operating systems:

  • Open a terminal on the folder in which you wish your local copy of the repository to be stored (it's recommended to use an empty folder) and run the following commands:
git init  #to initialize an empty Git repository
git remote add origin #to make your local repository point to the remote repository in GitBucket
git checkout master #to create a local repository of the master branch
git checkout develop #to create a local repository of the develop branch

If you want to see the changes your doing, check the file ".git/config" under your repository directory.

With this, you will now have DRM4G's source code at your disposal.

From here you should create branches for every new feature you'd like to include to the DRM4G.

  • To create a new branch and switch to it you can use the command git checkout -b <branch_name>.
  • For a more in depth tutorial on how branching works, click here.

Testing the DRM4G

Once you've made the changes you wanted to, you'll want to install your version to be sure that your new feature is working properly.

Just in case you would like to try out different versions, we recommend you use a virtual environment to test it.

  • Here you can find a tutorial on how to install a virtual environment, or you can look for one on your own.

Before you can install and try out your own version, you'll have to build your own package:

  • Open a terminal in the folder where your repository is located.
  • Run the command python sdist

This will create a a distribution package under a folder called dist.

Installing your version in a virtual environment

Go to wherever you have your virtual environment, open a terminal and execute the following commands:

source bin/activate
export DRM4G_DIR = $PWD/conf 
pip install path/to/drm4g/package

DRM4G_DIR is where the configuration files will be installed. More information here.

And that's it. Now you can use and test your own version of DRM4G.

For other ways to install the DRM4G, you can check here.

How to debug code

You have at your disposal a number of ways to check what could have gone wrong when something breaks.

Via the DRM4G CLI

  • All of the drm4g commands can be executed in debug mode by adding the option "--dbg".

Via the logger

  • If there has been any error, chances are that you can find the cause by looking at DRM4G's log files, found in "$DRM4G_DIR/.drm4g/var"
    • The DRM4G is divided into different parts, and each one of them has its own logger.
    • By default, the logger level is set to INFO for all of them. To see all of the log messages that the DRM4G can record, you'll have to modify the file "$DRM4G_DIR/.drm4g/etc/logger.conf" and change the level of the logger to "DEBUG" for each part that you wish to check.

Via the job's logs

  • In addition, all of the individual jobs submitted have their own log files. They can help you see in which phase did the program stop working.
    • They are grouped in folders by their job ID, every hundred jobs. They can be located in "$DRM4G_DIR/.drm4g/var".
      • The folders will look like this: "000-099", "100-199", ...

Our Git workflow

In this section you'll find the guidelines of how we tackle the development process.

As mentioned above, to us, our central repository will be a private one hosted by GitBucket, but in addition we have a second public one hosted in GitHub to make the DRM4G accessible to anyone that might want to contribute to the project.

Both this two repositories have at least two branches, the master and the develop branches, that must be synchronized at all times.

Commits in the master branch represent all the stable versions of the DRM4G. This means that merges to this branch are only done when ready to publish a release.

The develop branch reflects the whole evolution of the project. It shows all of the new features and bug fixes that have been included. This is the main branch where all of the work will be done.

Getting ready for a release

Once all the new changes for the next release have been added to the develop branch and it is at a stable point, it's time to publish a new release.

From this point on, all the following changes done on this branch will be for the following release. So, to avoid halting the development of the project a new branch is created indicating the new release number.

  • The nomenclature will be a string followed by the new version in numbers in the form of DRM4G-MAJOR.MINOR.PATCH and will have its number increment following this guidelines:
    • MAJOR version when you add some new functionality or you make incompatible API changes,
    • MINOR version when you improve some part of the DRM4G's functionality in a backwards-compatible manner
    • PATCH version when you make backwards-compatible bug fixes.
  • For creating tags, the same naming convention will be followed, albeit just the digits will be used.

It is in this branch where the version number of all the files will be modified, and where the code will be brought to a release ready state. That means that it is where minor bug fixes will be made, where comments and unnecessary code snippets will be removed and other maintenance tasks will be carried out.

  • During this process you might want to merge the new version number or some of the bug fixes on to the develop branch so that future features may incorporate them.

When ready, all that needs to be done is merge it into the master, create a tag, update the remote repositories and publish the new release.

git checkout master
git merge --squash DRM4G-X.X.X
git commit -v
git tag X.X.X
git push origin master
git push origin --tags
git push github master
git push github --tags
  • When performing the commit, the commit message will include all the previous commits made, but at the beginning a detailed message describing what is being added should be included.
  • Squashing all the commits when doing the merge, will ensure that every commit in the master is a stable version and will make the log history cleaner, allowing use to easily see the changes made between commits.

Updating the develop will be a bit different. If it is ever needed to rollback to a previous version, its commit has to be accessible. So merges to the develop branch won't be squashed.

git checkout develop
git merge --no-ff DRM4G-X.X.X
git tag X.X.X
git push origin develop
git push origin --tags
git push github develop
git push github --tags

Publishing a release

After having updated the master, it's time to publish the release.

  1. First, the package has to be uploaded to PyPI, there's a few ways to do this, but you'll need to have an account with access to the groups PyPI page.
  2. Run the command python register or modify your $HOME/.pypirc file and add your credentials to be able to submit package data to PyPI.
    • This step is only needed the first time.
  3. Finally submit the distribution files.
    • Run the command python sdist upload
    • Run the command python sdist and then upload the package though PyPI's user interface.

Attachments (6)

Download all attachments as: .zip