Uploaded image for project: 'IGB'
  1. IGB
  2. IGBF-1321

Test for errors due to htsjdk library update

    Details

    • Story Points:
      2
    • Sprint:
      B - Summer 2018, Summer 2018 Part 2

      Description

      The resolution for issue IGBF-1098 involved updating the samtools library.

      Thoroughly testing the impact of changing the library is an extensive process.

        Attachments

          Issue Links

            Activity

            Hide
            ieclabau Ivory Blakley (Inactive) added a comment -

            Because this change could have impacts that we don't notice, I think it would be best for this change to sit on the development version (not release version) for another cycle. I think it should be released in IGB 9.0.2.

            Show
            ieclabau Ivory Blakley (Inactive) added a comment - Because this change could have impacts that we don't notice, I think it would be best for this change to sit on the development version (not release version) for another cycle. I think it should be released in IGB 9.0.2.
            Hide
            ieclabau Ivory Blakley (Inactive) added a comment -
            Show
            ieclabau Ivory Blakley (Inactive) added a comment - Some links... Kiran's branch with the update: https://bitbucket.org/kkorey/kkorey-igb/branch/IGBF-1098-sam-so Samtools java library documentation: http://www.javadoc.io/doc/com.github.samtools/htsjdk/2.16.0 Google document summarizing testing process (share link): https://docs.google.com/document/d/19_EYNgiJr-GJT5zSQ99ogrLw8_aO5rl2ejXiNUYwq9M/edit?usp=sharing
            Hide
            ieclabau Ivory Blakley (Inactive) added a comment -

            At this point, I have not found anything that makes me think we should not merge this into master.

            There may be some lag in loading GFF3 files, but it is hard to say if this is much more pronounced in this branch than in master.
            On mac it is a lag, on Windows (at least on one machine) GFF3 did not load; on another machine I think it did. GFF3 has already been targeted for this.

            I think whatever issues we subsequently find can be addressed as their own issues. As a baseline, this looks like a good change.

            Show
            ieclabau Ivory Blakley (Inactive) added a comment - At this point, I have not found anything that makes me think we should not merge this into master. There may be some lag in loading GFF3 files, but it is hard to say if this is much more pronounced in this branch than in master. On mac it is a lag, on Windows (at least on one machine) GFF3 did not load; on another machine I think it did. GFF3 has already been targeted for this. I think whatever issues we subsequently find can be addressed as their own issues. As a baseline, this looks like a good change.
            Hide
            ieclabau Ivory Blakley (Inactive) added a comment -

            Here is the google doc
            https://docs.google.com/document/d/19_EYNgiJr-GJT5zSQ99ogrLw8_aO5rl2ejXiNUYwq9M/edit
            (copy/pasted below, sorry the formatting turned nasty in the copy/paste process)

            As part of issue IGBF-1098 Kiran had to update third party libraries.

            Java file changed
            File type affected in IGB
            Test method
            Test result
            NibbleResiduesParser.java

            SAMWriter.java

            BNIB.java

            Not worry about
            NA
            BaiZoomSymLoader.java

            FastaIdx.java

            GFF.java
            .gff
            Use smoke test quickload, load this file file type.
            Look at images in File Formats to make sure it is displayed correctly.
            Does this mean GTF? I can’t see any GFF file!
            GFF3.java
            .gff3
            Use smoke test quickload, load this file file type.
            Look at images in File Formats to make sure it is displayed correctly.
            Does not load.
            Did not load for a particular region too on Windows.

            GFF3 from smoke testing QL does load on mac, but even a small region takes forever (about a full minute for chr1:61,009,789-61,561,938) takes a minute
            IndexZoomSymLoader.java

            LineProcessor.java

            Doesn’t affect
            Looks Good!
            PSL.java

            Use smoke test quickload, load this file file type.
            Look at images in File Formats to make sure it is displayed correctly.
            (need to add psl and pslx to quickload)
            PSL file is not loaded in both 9.0.1 and 9.0.0
            SymLoaderTabix.java

            TwoBitNew.java

            UnindexedSymLoader.java

            VCF.java

            Use smoke test quickload, load this file file type.
            Look at images in File Formats to make sure it is displayed correctly.
            Compared 9.0.0 VCF with current branch. Tallied Selection info. Looks good!
            Wiggle.java

            Use smoke test quickload, load this file file type.
            Look at images in File Formats to make sure it is displayed correctly.
            Looks good!
            BAMSym.java

            Use smoke test quickload, load this file file type.
            Look at images in File Formats to make sure it is displayed correctly.
            Looks good!
            GeneralUtils.java

            getURILength is never called and it has dependency imports of new library.(SeekableStream and SeekableStreamFactory) We need eliminate them and if method is not used then we need to delete it..
            No major changes in file regarding new file.
            LocalUrlCacher.java

            No major code change. Only library updation in imports. Checked methods in API documentation of the library and the usage of them in the codebase.
            Looks good!
            SeekableBufferedStream.java

            Only BAM.java uses SeekableBufferedStream of library. Everywhere else in the codebase, We have used customized SeekableBufferedStream file. Why?
            Import change only.
            SeekableFTPStream.java

            Only import change. Doesn’t affect.
            Looks Good!
            core/shared-lib-wrapper/pom.xml

            HttpDirectoryInputSeekableStream.java

            Import change. Doesn’t affect the code
            Looks Good!
            ExonConnectorGlyph.java

            One import change. Doesn’t affect.
            Looks good!
            TranscriptIsoformEvidenceVisualizationManager.java

            Only import change. No major change.
            Ivory please take a look.

            main/pom.xml

            plugins/bam/pom.xml

            BAM.java

            Use smoke test quickload, load this file file type.
            Look at images in File Formats to make sure it is displayed correctly.
            A doubt?
            Looks Good!
            Tested according to File Formats.
            SAM.java

            Use smoke test quickload, load this file file type.
            Look at images in File Formats to make sure it is displayed correctly.

            XAM.java

            plugins/bed/pom.xml

            BedSymloader.java
            .bed file
            Tested with Fileformat test page for .bed file. Compared Selection info of 9.0 to 9.0.1. Both matched. Import change in code file. No major change.
            Looks Good!
            plugins/external-sort/pom.xml

            ExternalMergeSort.java

            Import changed. Looked at the code. Does affect anything.
            Looks Good!
            pom.xml

            Show
            ieclabau Ivory Blakley (Inactive) added a comment - Here is the google doc https://docs.google.com/document/d/19_EYNgiJr-GJT5zSQ99ogrLw8_aO5rl2ejXiNUYwq9M/edit (copy/pasted below, sorry the formatting turned nasty in the copy/paste process) As part of issue IGBF-1098 Kiran had to update third party libraries. Java file changed File type affected in IGB Test method Test result NibbleResiduesParser.java SAMWriter.java BNIB.java Not worry about NA BaiZoomSymLoader.java FastaIdx.java GFF.java .gff Use smoke test quickload, load this file file type. Look at images in File Formats to make sure it is displayed correctly. Does this mean GTF? I can’t see any GFF file! GFF3.java .gff3 Use smoke test quickload, load this file file type. Look at images in File Formats to make sure it is displayed correctly. Does not load. Did not load for a particular region too on Windows. GFF3 from smoke testing QL does load on mac, but even a small region takes forever (about a full minute for chr1:61,009,789-61,561,938) takes a minute IndexZoomSymLoader.java LineProcessor.java Doesn’t affect Looks Good! PSL.java Use smoke test quickload, load this file file type. Look at images in File Formats to make sure it is displayed correctly. (need to add psl and pslx to quickload) PSL file is not loaded in both 9.0.1 and 9.0.0 SymLoaderTabix.java TwoBitNew.java UnindexedSymLoader.java VCF.java Use smoke test quickload, load this file file type. Look at images in File Formats to make sure it is displayed correctly. Compared 9.0.0 VCF with current branch. Tallied Selection info. Looks good! Wiggle.java Use smoke test quickload, load this file file type. Look at images in File Formats to make sure it is displayed correctly. Looks good! BAMSym.java Use smoke test quickload, load this file file type. Look at images in File Formats to make sure it is displayed correctly. Looks good! GeneralUtils.java getURILength is never called and it has dependency imports of new library.(SeekableStream and SeekableStreamFactory) We need eliminate them and if method is not used then we need to delete it.. No major changes in file regarding new file. LocalUrlCacher.java No major code change. Only library updation in imports. Checked methods in API documentation of the library and the usage of them in the codebase. Looks good! SeekableBufferedStream.java Only BAM.java uses SeekableBufferedStream of library. Everywhere else in the codebase, We have used customized SeekableBufferedStream file. Why? Import change only. SeekableFTPStream.java Only import change. Doesn’t affect. Looks Good! core/shared-lib-wrapper/pom.xml HttpDirectoryInputSeekableStream.java Import change. Doesn’t affect the code Looks Good! ExonConnectorGlyph.java One import change. Doesn’t affect. Looks good! TranscriptIsoformEvidenceVisualizationManager.java Only import change. No major change. Ivory please take a look. main/pom.xml plugins/bam/pom.xml BAM.java Use smoke test quickload, load this file file type. Look at images in File Formats to make sure it is displayed correctly. A doubt? Looks Good! Tested according to File Formats. SAM.java Use smoke test quickload, load this file file type. Look at images in File Formats to make sure it is displayed correctly. XAM.java plugins/bed/pom.xml BedSymloader.java .bed file Tested with Fileformat test page for .bed file. Compared Selection info of 9.0 to 9.0.1. Both matched. Import change in code file. No major change. Looks Good! plugins/external-sort/pom.xml ExternalMergeSort.java Import changed. Looked at the code. Does affect anything. Looks Good! pom.xml

              People

              • Assignee:
                ieclabau Ivory Blakley (Inactive)
                Reporter:
                ieclabau Ivory Blakley (Inactive)
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: