Details
-
Type:
Improvement
-
Status: Closed (View Workflow)
-
Priority:
Critical
-
Resolution: Done
-
Affects Version/s: None
-
Fix Version/s: None
-
Labels:None
-
Story Points:1
-
Sprint:Fall 2018 Sprint 3, Fall 2018 Sprint 4
Description
We are now using packages from htsjdk artifact in the IGB project. However, there is a problem. It is not an OSGi bundle. To handle this, we are now importing the htsjdk into the "shared-lib-wrapper" module of IGB project. The pom.xml file of this module uses the BND-tool maven plug-in to packages htsjdk along with many other completely unrelated artifacts into one large bundle, which then gets loaded into the OSGi run-time along with all the other bundles that make up the IGB application.
This is not ideal. Maintaining the "shared-lib-wrapper" module is made extra complicated by this approach. For example, if some of these libraries rely on different versions of the same artifact, the approach won't work.
To simplify our codebase, we should transform these 3rd-party jar artifacts into proper OSGi bundles.
To do this, let's "fork" the htsjdk library and modify/enhance their build process to create a proper OSGi bundle.
Following this, we should add it to our nexus maven repository (https://nexus.bioviz.org) so that the new htsjdk OSGi bundle is available to IGB developers.
Attachments
Issue Links
- relates to
-
IGBF-1457 Startup Error in Console (ERROR com.affymetrix.main.OSGiHandler)
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
IGBF-1417 Create new artifact with bigbed and bigwig parsing code from IGV codebase
-
- Closed
-
-
-
- Closed
-
Activity
| Field | Original Value | New Value |
|---|---|---|
| Sprint | Fall 2018 Sprint 3 [ 53 ] |
| Rank | Ranked higher |
| Rank | Ranked higher |
| Status | Open [ 1 ] | In Progress [ 3 ] |
| Status | In Progress [ 3 ] | Open [ 1 ] |
| Status | Open [ 1 ] | In Progress [ 3 ] |
| Assignee | Srishti Tiwari [ stiwari8 ] |
Hello Dr. Loraine,
After creating OSGi bundle for htsjdk bundle I realize the imported dependency that htsjdk has, is also not OSGi bundle and to use it in our system we must make them an OSGi bundle. One way to go about it would be to exclude those from imported packages if they are not coming in our workflow. But if we wish to use htsjdk extensively in future anyway, we should have a look at those bundles.
Please have a look at the repository of this jar gov.nih.nlm.ncbi.ngs-java :
https://github.com/ncbi/ngs
Thanks,
Srishti
I figured the usage of gov.nih.nlm.ncbi.ngs-java in htsjdk is only limited to reading SRA files and nothing else. I see that we don't deal with SRA file formats. So If we are not planning to use it in our system, it will not be very useful to include that dependency.
Please review the following branches :
https://github.com/srishti-tiwari/htsjdk/tree/igb-2.16.1
https://bitbucket.org/srishtitiwari/integrated-genome-browser/src/IGBF-1418/
Thanks,
Srishti
Pull request submitted.
https://bitbucket.org/lorainelab/integrated-genome-browser/pull-requests/638/igbf-1418/diff
| Status | In Progress [ 3 ] | Pull Request Submitted [ 10101 ] |
| Status | Pull Request Submitted [ 10101 ] | In Progress [ 3 ] |
| Status | In Progress [ 3 ] | Pull Request Submitted [ 10101 ] |
| Status | Pull Request Submitted [ 10101 ] | Reviewing Pull Request [ 10303 ] |
| Status | Reviewing Pull Request [ 10303 ] | Needs Testing [ 10002 ] |
| Assignee | Srishti Tiwari [ stiwari8 ] |
| Rank | Ranked higher |
| Sprint | Fall 2018 Sprint 3 [ 53 ] | Fall 2018 Sprint 3, Fall 2018 Sprint 4 [ 53, 54 ] |
| Rank | Ranked higher |
I merged a pull request from Srishti's branch https://github.com/srishti-tiwari/htsjdk/tree/igb-2.16.1 into https://github.com/lorainelab/htsjdk/tree/igb-2.16.1
IGB-friendly htsjdk-igb artifact is now uploaded to https://nexus.bioviz.org/repository/maven-releases/ with coordinates:
<groupId>htsjdk-igb</groupId>
<artifactId>htsjdk-igb</artifactId>
<version>2.16.1</version>
<packaging>bundle</packaging>
| Assignee | Kiran Korey [ kkorey ] |
| Status | Needs Testing [ 10002 ] | Testing In Progress [ 10003 ] |
| Status | Testing In Progress [ 10003 ] | Open [ 1 ] |
I uploaded the artifact manually using the Nexus GUI. However, a checksum did not get generated for the artifact. This is a issue with Nexus, which has been fixed in newer versions.
I will upgrade Nexus, but for now, let's add code to the build.gradle file to upload the artifacts. If we do this, the checksum will be generated & included in the upload. Let's also upload the source code.
For details, see my comment on https://jira.transvar.org/browse/IGBF-1419.
Note: I re-opened IGBF-1419 pending fix of this issue.
| Assignee | Kiran Korey [ kkorey ] | Ann Loraine [ aloraine ] |
| Assignee | Ann Loraine [ aloraine ] |
| Assignee | Srishti Tiwari [ stiwari8 ] |
| Status | Open [ 1 ] | In Progress [ 3 ] |
| Status | In Progress [ 3 ] | Needs 1st Level Review [ 10005 ] |
I added the changes to upload archive using build.gradle. The checksum is getting generated when the artifacts are uploaded to the test repository.
Please refer:
https://nexus.bioviz.org/repository/Test_IGB_Artifact_Deployment/
| Status | Needs 1st Level Review [ 10005 ] | Reviewing [ 10301 ] |
| Status | Reviewing [ 10301 ] | Ready for Pull Request [ 10304 ] |
Please refer to this link :
https://github.com/gradle/gradle/issues/6849
Since this Gradle issue is still open, we need to do a workaround, untill the above issue is fixed.
So I added a dummy scala file just to get the build work successfully. This needs to be removed when the issue is fixed in Gradle.
Please try following commands:
> gradle jar
> gradle uploadArchives
| Status | Ready for Pull Request [ 10304 ] | Needs 1st Level Review [ 10005 ] |
| Resolution | Done [ 10000 ] | |
| Status | Needs 1st Level Review [ 10005 ] | Closed [ 6 ] |
| Workflow | Loraine Lab Workflow [ 18124 ] | Fall 2019 Workflow Update [ 19950 ] |
| Workflow | Fall 2019 Workflow Update [ 19950 ] | Revised Fall 2019 Workflow Update [ 22071 ] |
For documentation on OSGi, see Loraine Lab ebooks repository. (Ask Dr. Loraine)