Uploaded image for project: 'IGB'
  1. IGB
  2. IGBF-3495

Setup a CI/CD pipeline for HTSJDK-IGB

    Details

    • Type: Task
    • Status: Needs 1st Level Review (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None

      Description

      • Create a CI/CD pipeline for htsjdk-igb using GitHub actions
      • Whenever a push action occurs in the repository, this pipeline is triggered.

        Attachments

        1. architecture.jpeg
          architecture.jpeg
          181 kB
        2. artifacts_image.png
          artifacts_image.png
          43 kB
        3. build.png
          build.png
          32 kB
        4. pipeline.png
          pipeline.png
          76 kB

          Activity

          Hide
          kgopu Kaushik Gopu added a comment - - edited
          • [~aloraine], I created a sample pipeline that triggers when a push to the repository happens.
          • The pipeline has only one single step, to execute the build command.
          • I have attached the image of the pipeline.yml for your reference.
          • The output generated by the action (pipeline.yml) can be found here.
          • I would like to discuss the pipeline requirements with you in detail.
          Show
          kgopu Kaushik Gopu added a comment - - edited [~aloraine] , I created a sample pipeline that triggers when a push to the repository happens. The pipeline has only one single step, to execute the build command. I have attached the image of the pipeline.yml for your reference. The output generated by the action (pipeline.yml) can be found here . I would like to discuss the pipeline requirements with you in detail.
          Hide
          kgopu Kaushik Gopu added a comment - - edited

          Notes for CI/CD Pipeline:

          Architecture:

          • Event: Events trigger workflows. Ex: push, pull-request.
          • Workflow: A workflow is like a pipeline. It is composed of one or many jobs
          • Runner: A physical or virtual hardware where the code for a workflow is executed. Each job can be executed on a different runner.
          • Jobs: Jobs aggregate steps and define which runner to execute them on

          Getting familiar with GitHub action clauses/keywords

          • On: The on keyword and the lines that follow it define which types of triggers the workflow will match on and start executing
          • Name: To define the name of the step or a Job
          • Jobs: Composed of one or more steps
          • Steps: Steps are the basic unit of execution. They consist of either invocations of predefined actions or a shell command to be run on the runner
          • Uses: For pulling pre-defined actions from the internet (similar to using APIs that do the job for us).
          • With: To pass parameters as an input to pre-defined action defined in the "Uses" clause
          • Run: To execute shell commands

          Workflow setup:

          • Workflow files are written in YAML, and they must be located in the .github/workflows directory

          Workflow of htsjdk-igb pipeline:

          You can find the workflow file of htsjdk-igb here.

          • This workflow gets triggered for push or pull-request event (The branch prefix must start with "igb-*".)
          • GitHub provides an OS to execute the workflow ( look for runs-on step)
          • actions/checkout@v3: pre-defined action for checking out a repo
          • actions/setup-java@v3: pre-defined action to set up Java environment. This action requires input parameters, such as Java version and distribution, which are sent using the "with" clause.
          • Now, the environment set-up is ready to build the project. The ./gradlew build -x test command is executed in run step
          • Implemented a verification step to ensure the build/libs directory contains all jars, including Source and Java Doc.

          Downloading Jars:

          • After a successful build, you can able to view and download the artifact(see image).
          Show
          kgopu Kaushik Gopu added a comment - - edited Notes for CI/CD Pipeline: Architecture: Event: Events trigger workflows. Ex: push, pull-request. Workflow: A workflow is like a pipeline. It is composed of one or many jobs Runner: A physical or virtual hardware where the code for a workflow is executed. Each job can be executed on a different runner. Jobs: Jobs aggregate steps and define which runner to execute them on Getting familiar with GitHub action clauses/keywords On: The on keyword and the lines that follow it define which types of triggers the workflow will match on and start executing Name: To define the name of the step or a Job Jobs: Composed of one or more steps Steps: Steps are the basic unit of execution. They consist of either invocations of predefined actions or a shell command to be run on the runner Uses: For pulling pre-defined actions from the internet (similar to using APIs that do the job for us). With: To pass parameters as an input to pre-defined action defined in the "Uses" clause Run: To execute shell commands Workflow setup: Workflow files are written in YAML, and they must be located in the .github/workflows directory Workflow of htsjdk-igb pipeline: You can find the workflow file of htsjdk-igb here . This workflow gets triggered for push or pull-request event (The branch prefix must start with "igb-*".) GitHub provides an OS to execute the workflow ( look for runs-on step) actions/checkout@v3: pre-defined action for checking out a repo actions/setup-java@v3: pre-defined action to set up Java environment. This action requires input parameters, such as Java version and distribution, which are sent using the "with" clause. Now, the environment set-up is ready to build the project. The ./gradlew build -x test command is executed in run step Implemented a verification step to ensure the build/libs directory contains all jars, including Source and Java Doc. Downloading Jars: After a successful build, you can able to view and download the artifact(see image).
          Hide
          kgopu Kaushik Gopu added a comment -

          Added detailed notes on GitHub actions and workflow. Let me know if you have trouble understanding them.

          Show
          kgopu Kaushik Gopu added a comment - Added detailed notes on GitHub actions and workflow. Let me know if you have trouble understanding them.
          Hide
          ann.loraine Ann Loraine added a comment -

          Moving to backlog as I do not have time to complete the work described above.

          Show
          ann.loraine Ann Loraine added a comment - Moving to backlog as I do not have time to complete the work described above.

            People

            • Assignee:
              Unassigned
              Reporter:
              kgopu Kaushik Gopu
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: