Skip to content

Koji software building

The Linux Software Building Service is based on Koji, a system to build RPM software packages for Linux OS distribution and IT department Computing Infrastructure. It uses mock to create chroot environments to perform software pakage builds.

From Koji's website:

Koji is an RPM-based build system. Koji's goal is to provide a flexible, secure, and reproducible way to build software.

Check Koji's upstream documentation for detailed information on how to use this tool.

User requirements

  • Koji is accessible via https://koji.cern.ch
  • The koji client is available via the koji package, which is already preinstalled in lxplus. It can be installed manually via yum install koji
  • Before using koji, you require an account in koji - please request an account through Linuxsupport SNOW here
  • If required, configure your koji client as per these instructions. Alternatively you may use lxplus or aiadm
  • To confirm that you account is ready you may run koji list-permissions --mine. Providing everything is configured correctly you should have 'build' permissions

Repositories

  • All tags in koji have a corresponding repository on http://linuxsoft.cern.ch/internal/repos
  • These repositories are synced automatically, but there may be delays of up to 30 mins depending on the size of the repo
  • We do not provide .repo files automatically, an example repo file can be seen here if you follow the AI workflow
  • If your machine is puppet managed, consider investigating the 'osrepo' module

RPM 101

  • A rpm is composed of a NAME, VERSION, RELEASE (which contains a DISTTAG)
ksh-20120801-10.el6_5.4
NAME=ksh
VERSION=20120801
RELEASE=10.el6_5.4
DISTTAG=el6_5
  • A rpm is built with rpmbuild -ba. If it doesn't work on your machine it will not work in Koji. However if it works on your machine it may not work on Koji (dependencies you have installed but not documented in the spec file, afs dependency, tag not defined, etc..)
  • To generate a clean buildroot koji uses mock
  • A rpm N V R (name, version, release) is unique in koji.
  • When you release a new RPM, bump the version and/or release number and associate it with a Changelog entry in the .spec file.

Workflow

As agreed with the AI team, the default workflow is the following:

workflow

  • We consider your machine runs puppet and uses the provided repositories definition
  • The tag used in the example is ai6 but it can be yours. Please open a ticket if you want to request a new tag.
  • IMPORTANT: If you request a new tag it is your responsibility to use the best practice of deploying your repo file via the ai osrepos puppet module

We will now see how to execute step 1, 3 and 5 from the image above:

Building your RPM

There are several ways to build your RPM using Koji. The recommended method is using RPMCI, which automates many of the tasks and will improve your workflow. The other methods are documented for completenes, but are not recommended.

RPMCI: Automating your RPM builds like a pro

We provide GitLab CI templating for you to use on your RPM building processes. Using our template, you'll automatically get a full Gitlab Continuous Integration pipeline for building your RPM, testing it and promoting it to QA and stable for multiple operating system versions.

This repo can help you centralise and simplify your GitLab CI scripts. Please refer to the repo's README for further instructions, or to the example RPM template to get started.

  • NOTE: All the build operations can be executed with --scratch to test if your package will build correctly, however please be aware that koji is not designed to be CI server
  • First add packages to your tag, as an example we are going to use the ai6 tag

    koji add-pkg --owner=ai-team ai6-testing mypkg
    koji add-pkg --owner=ai-team ai6-qa mypkg
    koji add-pkg --owner=ai-team ai6-stable mypkg
    
  • This operation is only needed once, when you add a new package

  • Note: There is no notion of a group in koji, only users. --owner can only be set to a valid user name. If you would like to generate builds using a 'team' username, please contact LinuxSupport to create the user for you
  • Build you package koji build ai6 mypkg-1.2-4.el6.src.rpm
  • Your package will build and providing it was successful the rpm will be available in the ai6-testing repository
  • An example to build via gitlab is koji build ai6 git+ssh://git@gitlab.cern.ch:7999/ai-config-team/ai-tools.git#8.12-1
  • This command will build from the ai-config-team/ai-tools repo from the 8.12-1 branch

  • Notes:

    • Your Gitlab project must be configured with visibility set to Internal (i.e. all authenticated CERN users can access the project) or Public (i.e. all Internet users can access the project without authentication). See Gitlab documentation for more details on visibility levels
    • Alternatively, you may use a project with visibility set to Private and explicitly grant access to user koji support, but take into account that the source RPM built by Koji will be open to all CERN users anyway

Promoting a build

  1. Tag your build to -qa:
    1. If you're using the recommended RPMCI method, you will find a Gitlab CI stage called "Deploy_qa" in your pipeline (once you push a git tag) to promote your package to -qa.
    2. If you are not, you will have to tag the build manually: koji tag-build ai6-qa mypkg-1.2-4.el7
  2. Your package will be available in -qa in a few minutes. You can find it looking for your repo in linuxsoft.
  3. Create a CRM ticket following this template with Proposed date for production set in a weeks time:

    promotion

  4. Wait a week and check if your ticket is not a blocker for anyone else

  5. Tag it to -stable:
    1. If you're using the recommended RPMCI method, you will find a Gitlab CI stage called "Deploy_stable" in your pipeline (same one from step 1) to promote your package to -stable.
    2. If you are not, you will have to tag the build manually: koji tag-build ai6-stable mypkg-1.2-4.el7
  6. Your package will be available in -stable in a few minutes.

Requesting a new tag

  • Complete this SNOW form
  • All tag names should be short and will have the distribution major release number in it's final name (ai6.ai7 etc..)