Uploaded image for project: 'IGB'
  1. IGB
  2. IGBF-1773 Create new repos for optionalPlugins
  3. IGBF-1827

Figure out how to set up bitbucket pipeline build for App repositories

    Details

    • Type: Sub-task
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
    • Story Points:
      1.5
    • Sprint:
      Summer 2019 Sprint 8, Summer 2019 Sprint 9, Summer 2019 Sprint 10, Summer 2019 Sprint 11

      Description

      Investigate and develop proof-of-concept for a bitbucket pipeline configuration that will allow Apps to be built automatically when a user pushes to their remote.

      Requirements:

      • The pipeline should create the App jar file and also an OBR index for the app.
      • The pipeline should copy the App jar and OBR index file named repository.xml to the downloads directory of the repository for that App.

      Investigate:

      • Can the OBR index file provide an absolute URL to the jar file hosted in the Bitbucket repository's download section? If yes, let's do this for the bitbucket pipelines build to enable testing.
      • Can a developer indicate the URL for the downloads section? If yes, then during testing we could add this location as a test App repository location.

        Attachments

          Issue Links

            Activity

            Hide
            ann.loraine Ann Loraine added a comment -

            IGB Apps now have example bitbucket_pipeline.yml files demonstrating how to

            • build App jar file and OBR index file "repository.xml"
            • copy same to downloads section

            Currently, a user can add the downloads URL to IGB (using App Manager GUI or Settings) to check that the App can be installed.

            Example:

            https://bitbucket.org/lorainelab/merge-annotation-operator/downloads/

            Note: Final slash character is required! (We may fix in later versions of IGB)

            Show
            ann.loraine Ann Loraine added a comment - IGB Apps now have example bitbucket_pipeline.yml files demonstrating how to build App jar file and OBR index file "repository.xml" copy same to downloads section Currently, a user can add the downloads URL to IGB (using App Manager GUI or Settings) to check that the App can be installed. Example: https://bitbucket.org/lorainelab/merge-annotation-operator/downloads/ Note: Final slash character is required! (We may fix in later versions of IGB)
            Hide
            prutha Prutha Kulkarni (Inactive) added a comment -

            For testing:
            1) Create a for the 23andMe app repository.
            2) You can just set up the repository variables for 23andMe. As the configuration file is already there in the repository, you just need to enable pipelines in the repository.
            Once it is enabled the build setup will start and after its completion you can check the downloads section for the JAR and repository.xml
            3) You can copy this downloads section link and paste it into the manage repositories section of the IGB app manager.
            4) Install the app and test it in IGB.
            For more details on the testing steps, please refer the comments section of IGBF-1822 ticket.

            Show
            prutha Prutha Kulkarni (Inactive) added a comment - For testing: 1) Create a for the 23andMe app repository. 2) You can just set up the repository variables for 23andMe. As the configuration file is already there in the repository, you just need to enable pipelines in the repository. Once it is enabled the build setup will start and after its completion you can check the downloads section for the JAR and repository.xml 3) You can copy this downloads section link and paste it into the manage repositories section of the IGB app manager. 4) Install the app and test it in IGB. For more details on the testing steps, please refer the comments section of IGBF-1822 ticket.
            Hide
            prutha Prutha Kulkarni (Inactive) added a comment -

            Thanks for the suggestion Prof. [~aloraine]. It worked.
            So, I forked the 23andme-snp-converter repository and setup the pipeline for the same.
            It is creating the required files under the download section and we can directly provide the "downloads" section path in the IGB app manager in order to test the app.
            Now submitting a pull request to push those changes onto the master.

            Show
            prutha Prutha Kulkarni (Inactive) added a comment - Thanks for the suggestion Prof. [~aloraine] . It worked. So, I forked the 23andme-snp-converter repository and setup the pipeline for the same. It is creating the required files under the download section and we can directly provide the "downloads" section path in the IGB app manager in order to test the app. Now submitting a pull request to push those changes onto the master.
            Hide
            prutha Prutha Kulkarni (Inactive) added a comment -

            Sure Prof. [~aloraine] I will fork the 23andMe repository to avoid permission problems.
            Also, the repository.xml is present at that location but I guess it is not accessible. I'll try to investigate on that.

            Show
            prutha Prutha Kulkarni (Inactive) added a comment - Sure Prof. [~aloraine] I will fork the 23andMe repository to avoid permission problems. Also, the repository.xml is present at that location but I guess it is not accessible. I'll try to investigate on that.
            Hide
            ann.loraine Ann Loraine added a comment -

            Another possibly useful suggestion:

            Show
            ann.loraine Ann Loraine added a comment - Another possibly useful suggestion: From the error, it looks like the repository.xml file – which IGB expects to find at URL https://bitbucket.org/.../downloads/repository.xml – is not present or accessible from that address.
            Hide
            ann.loraine Ann Loraine added a comment -

            A potentially usful suggestion:

            • It will probably be easier to make a fork and use that for testing the build and copy to downloads location.

            Reason:

            I think you will be less likely to encounter permissions problems since you would "own" the fork in that case. I'm not sure if bitbucket will allow a non-admin user to modify a team repository, and both 23andMe and "Hello World" App repositories are "owned" by the Loraine Lab team. You are a member and have write-access but not admin-access to same.

            If you do make a fork, it is completely fine to issue pull requests from your fork's master to the team repo's master and then accept your own pulls – or you could ask another devloper to review if you want another pair of eyes.

            Show
            ann.loraine Ann Loraine added a comment - A potentially usful suggestion: It will probably be easier to make a fork and use that for testing the build and copy to downloads location. Reason: I think you will be less likely to encounter permissions problems since you would "own" the fork in that case. I'm not sure if bitbucket will allow a non-admin user to modify a team repository, and both 23andMe and "Hello World" App repositories are "owned" by the Loraine Lab team. You are a member and have write-access but not admin-access to same. If you do make a fork, it is completely fine to issue pull requests from your fork's master to the team repo's master and then accept your own pulls – or you could ask another devloper to review if you want another pair of eyes.
            Hide
            prutha Prutha Kulkarni (Inactive) added a comment -

            Done with the creation of JAR file as well as the OBR index for the app. Setup pipeline to copy both the files to downloads section of the bitbucket.
            Now investigating on how we can provide a URL of the bitbucket repository's download section directly to the app manager in IGB for testing.
            Following exception thrown when we check on the downloads repository path from IGB's app manager :
            10:53:12.286 ERROR o.l.i.p.m.RepositoryInfoManager - Expected element 'repository' at the root of the document
            java.lang.Exception: Expected element 'repository' at the root of the document
            at org.apache.felix.bundlerepository.impl.PullParser.parseRepository(PullParser.java:50) ~[na:na]
            at org.apache.felix.bundlerepository.impl.DataModelHelperImpl.repository(DataModelHelperImpl.java:168) ~[na:na]
            at org.apache.felix.bundlerepository.impl.DataModelHelperImpl.repository(DataModelHelperImpl.java:141) ~[na:na]
            at org.apache.felix.bundlerepository.impl.RepositoryAdminImpl$1.run(RepositoryAdminImpl.java:114) ~[na:na]
            at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_101]
            at org.apache.felix.bundlerepository.impl.RepositoryAdminImpl.addRepository(RepositoryAdminImpl.java:110) ~[na:na]
            at org.apache.felix.bundlerepository.impl.RepositoryAdminImpl.addRepository(RepositoryAdminImpl.java:98) ~[na:na]
            at org.lorainelab.igb.plugin.manager.RepositoryInfoManager$1.runInBackground(RepositoryInfoManager.java:59) [plugins-manager-9.1.0.jar:na]
            at org.lorainelab.igb.plugin.manager.RepositoryInfoManager$1.runInBackground(RepositoryInfoManager.java:55) [plugins-manager-9.1.0.jar:na]
            at com.affymetrix.genometry.thread.CThreadWorker.doInBackground(CThreadWorker.java:73) [genometry-9.1.0.jar:na]
            at javax.swing.SwingWorker$1.call(SwingWorker.java:295) [na:1.8.0_101]
            at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_101]
            at javax.swing.SwingWorker.run(SwingWorker.java:334) [na:1.8.0_101]
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_101]
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_101]
            at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]

            Debugging for the same.

            Show
            prutha Prutha Kulkarni (Inactive) added a comment - Done with the creation of JAR file as well as the OBR index for the app. Setup pipeline to copy both the files to downloads section of the bitbucket. Now investigating on how we can provide a URL of the bitbucket repository's download section directly to the app manager in IGB for testing. Following exception thrown when we check on the downloads repository path from IGB's app manager : 10:53:12.286 ERROR o.l.i.p.m.RepositoryInfoManager - Expected element 'repository' at the root of the document java.lang.Exception: Expected element 'repository' at the root of the document at org.apache.felix.bundlerepository.impl.PullParser.parseRepository(PullParser.java:50) ~ [na:na] at org.apache.felix.bundlerepository.impl.DataModelHelperImpl.repository(DataModelHelperImpl.java:168) ~ [na:na] at org.apache.felix.bundlerepository.impl.DataModelHelperImpl.repository(DataModelHelperImpl.java:141) ~ [na:na] at org.apache.felix.bundlerepository.impl.RepositoryAdminImpl$1.run(RepositoryAdminImpl.java:114) ~ [na:na] at java.security.AccessController.doPrivileged(Native Method) ~ [na:1.8.0_101] at org.apache.felix.bundlerepository.impl.RepositoryAdminImpl.addRepository(RepositoryAdminImpl.java:110) ~ [na:na] at org.apache.felix.bundlerepository.impl.RepositoryAdminImpl.addRepository(RepositoryAdminImpl.java:98) ~ [na:na] at org.lorainelab.igb.plugin.manager.RepositoryInfoManager$1.runInBackground(RepositoryInfoManager.java:59) [plugins-manager-9.1.0.jar:na] at org.lorainelab.igb.plugin.manager.RepositoryInfoManager$1.runInBackground(RepositoryInfoManager.java:55) [plugins-manager-9.1.0.jar:na] at com.affymetrix.genometry.thread.CThreadWorker.doInBackground(CThreadWorker.java:73) [genometry-9.1.0.jar:na] at javax.swing.SwingWorker$1.call(SwingWorker.java:295) [na:1.8.0_101] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_101] at javax.swing.SwingWorker.run(SwingWorker.java:334) [na:1.8.0_101] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_101] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_101] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101] Debugging for the same.

              People

              • Assignee:
                Unassigned
                Reporter:
                ann.loraine Ann Loraine
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: