Uploaded image for project: 'IGB'
  1. IGB
  2. IGBF-1425

Investigate & fix error when opening SAM files from Quickload, URL

    Details

    • Story Points:
      1
    • Sprint:
      Fall 2018 Sprint 3, Fall 2018 Sprint 4, Fall 2018 Sprint 5

      Description

      Noticed by Kiran Korey:

      Attempts to open SAM files from Internet fail:

      http://igbquickload.org/smokeTestingQuickload/H_sapiens_Dec_2013/Sam/chr1.p22.3_noHeader.sam
      http://igbquickload.org/smokeTestingQuickload/H_sapiens_Dec_2013/Sam/chr1.p22.3_wHeader.sam

      However, if you download "wHeader" file and try to open it locally, it can be done.

      The "noHeader" file gives an error, however, even if the corresponding human genome is already opened and on display.

        Attachments

          Activity

          ann.loraine Ann Loraine created issue -
          ann.loraine Ann Loraine made changes -
          Field Original Value New Value
          Rank Ranked higher
          kkorey Kiran Korey (Inactive) made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          kkorey Kiran Korey (Inactive) made changes -
          Assignee Srishti Tiwari [ stiwari8 ] Kiran Korey [ kkorey ]
          Hide
          kkorey Kiran Korey (Inactive) added a comment -
          Show
          kkorey Kiran Korey (Inactive) added a comment - new (smaller) sam files are available here: http://igbquickload.org/smokeTestingQuickload/H_sapiens_Dec_2013/Sam Quickload site address is: http://igbquickload.org/smokeTestingQuickload
          ann.loraine Ann Loraine made changes -
          Rank Ranked higher
          ann.loraine Ann Loraine made changes -
          Sprint Fall 2018 Sprint 3 [ 53 ] Fall 2018 Sprint 3, Fall 2018 Sprint 4 [ 53, 54 ]
          ann.loraine Ann Loraine made changes -
          Rank Ranked higher
          Hide
          ann.loraine Ann Loraine added a comment - - edited

          Note:

          Error may be due to SAM files not using the correct genome assembly version.
          Nowlan noticed it.

          Show
          ann.loraine Ann Loraine added a comment - - edited Note: Error may be due to SAM files not using the correct genome assembly version. Nowlan noticed it.
          Hide
          kkorey Kiran Korey (Inactive) added a comment -

          I am able to get the SAM files with header load successfully from Local copy and Quickload.
          The issue might be with no header files, will talk to Nowlan about this.

          Show
          kkorey Kiran Korey (Inactive) added a comment - I am able to get the SAM files with header load successfully from Local copy and Quickload. The issue might be with no header files, will talk to Nowlan about this.
          Hide
          kkorey Kiran Korey (Inactive) added a comment -

          Need a first level review for the following branch

          https://bitbucket.org/kkorey/kkorey-igb/branch/IGBF-1425-stream#diff

          Show
          kkorey Kiran Korey (Inactive) added a comment - Need a first level review for the following branch https://bitbucket.org/kkorey/kkorey-igb/branch/IGBF-1425-stream#diff
          kkorey Kiran Korey (Inactive) made changes -
          Status In Progress [ 3 ] Needs 1st Level Review [ 10005 ]
          kkorey Kiran Korey (Inactive) made changes -
          Assignee Kiran Korey [ kkorey ]
          ann.loraine Ann Loraine made changes -
          Assignee Ann Loraine [ aloraine ]
          ann.loraine Ann Loraine made changes -
          Status Needs 1st Level Review [ 10005 ] Reviewing [ 10301 ]
          Hide
          ann.loraine Ann Loraine added a comment -

          Some requests:

          • Please handle FTP protocol in addition to HTTPS, HTTP and local file.
          • Check this line: "String reachable_url = LocalUrlCacher.getReachableUrl(uri.toASCIIString());" If the LocalUrlCacher fails and has to return a null value, does it already open an error dialog? What sort of logging is done when that happens? We need to check that if there is an error, it will be easy to diagnose the problem. We will be asking users to tell us what they see in the Console, so we need to make sure something informative is printed.
          • If there is an error, use the logger to report useful diagnostic information to the Console. For example, the log message should report the URI that could not be reached.
          • Please change the error message to make it more user-friendly. Proposed new user-friendly text: "The data set you requested could not be loaded. Select Help > Show Console for more information." If possible, "Help > Show Console" should be rendered in bold font.
          Show
          ann.loraine Ann Loraine added a comment - Some requests: Please handle FTP protocol in addition to HTTPS, HTTP and local file. Check this line: "String reachable_url = LocalUrlCacher.getReachableUrl(uri.toASCIIString());" If the LocalUrlCacher fails and has to return a null value, does it already open an error dialog? What sort of logging is done when that happens? We need to check that if there is an error, it will be easy to diagnose the problem. We will be asking users to tell us what they see in the Console, so we need to make sure something informative is printed. If there is an error, use the logger to report useful diagnostic information to the Console. For example, the log message should report the URI that could not be reached. Please change the error message to make it more user-friendly. Proposed new user-friendly text: "The data set you requested could not be loaded. Select Help > Show Console for more information." If possible, "Help > Show Console" should be rendered in bold font.
          ann.loraine Ann Loraine made changes -
          Status Reviewing [ 10301 ] Open [ 1 ]
          ann.loraine Ann Loraine made changes -
          Assignee Ann Loraine [ aloraine ] Kiran Korey [ kkorey ]
          Hide
          kkorey Kiran Korey (Inactive) added a comment -
          • Please handle FTP protocol in addition to HTTPS, HTTP and local file.
            • Done
          • Check this line: "String reachable_url = LocalUrlCacher.getReachableUrl(uri.toASCIIString());" If the LocalUrlCacher fails and has to return a null value, does it already open an error dialog? What sort of logging is done when that happens? We need to check that if there is an error, it will be easy to diagnose the problem. We will be asking users to tell us what they see in the Console, so we need to make sure something informative is printed.
            • No the method does not have any error dialog, but there is error logger (console) if there is any exception in the method.
            • Added a error logger showing URL and HTTP Status when the URL is not reachable.
          • If there is an error, use the logger to report useful diagnostic information to the Console. For example, the log message should report the URI that could not be reached.
            • Done.
          • Please change the error message to make it more user-friendly. Proposed new user-friendly text: "The data set you requested could not be loaded. Select Help > Show Console for more information." If possible, "Help > Show Console" should be rendered in bold font.
            • Done.
          Show
          kkorey Kiran Korey (Inactive) added a comment - Please handle FTP protocol in addition to HTTPS, HTTP and local file. Done Check this line: "String reachable_url = LocalUrlCacher.getReachableUrl(uri.toASCIIString());" If the LocalUrlCacher fails and has to return a null value, does it already open an error dialog? What sort of logging is done when that happens? We need to check that if there is an error, it will be easy to diagnose the problem. We will be asking users to tell us what they see in the Console, so we need to make sure something informative is printed. No the method does not have any error dialog, but there is error logger (console) if there is any exception in the method. Added a error logger showing URL and HTTP Status when the URL is not reachable. If there is an error, use the logger to report useful diagnostic information to the Console. For example, the log message should report the URI that could not be reached. Done. Please change the error message to make it more user-friendly. Proposed new user-friendly text: "The data set you requested could not be loaded. Select Help > Show Console for more information." If possible, "Help > Show Console" should be rendered in bold font. Done.
          kkorey Kiran Korey (Inactive) made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          kkorey Kiran Korey (Inactive) made changes -
          Status In Progress [ 3 ] Needs 1st Level Review [ 10005 ]
          kkorey Kiran Korey (Inactive) made changes -
          Assignee Kiran Korey [ kkorey ]
          Hide
          kkorey Kiran Korey (Inactive) added a comment -

          The following Scenarios needs to be tested for this issue:
          (Scenario --> expected behavior)

          Reachable valid URL --> Sam File should be loaded.

          Reachable valid URL but unauthorized --> Exception in the log saying URL not reachable and HTTP_status code (404)

          Invalid / Unreachable URL --> Exception in the log saying URL not reachable and HTTP_status code ( 401 0r 403)

          Reachable URL but invalid File --> Exception in the log saying invalid file format.

          Show
          kkorey Kiran Korey (Inactive) added a comment - The following Scenarios needs to be tested for this issue: (Scenario --> expected behavior) Reachable valid URL --> Sam File should be loaded. Reachable valid URL but unauthorized --> Exception in the log saying URL not reachable and HTTP_status code (404) Invalid / Unreachable URL --> Exception in the log saying URL not reachable and HTTP_status code ( 401 0r 403) Reachable URL but invalid File --> Exception in the log saying invalid file format.
          stiwari8 Srishti Tiwari (Inactive) made changes -
          Assignee Srishti Tiwari [ stiwari8 ]
          Hide
          stiwari8 Srishti Tiwari (Inactive) added a comment -

          For the positive scenario, with a valid reachable URL on the quickload it works. On the console it prints:
          org.lorainelab.igb.bam.SAM initTheSeqs
          SEVERE: Sam file must be sorted by coordinate.

          For a reachable but invalid File, it does not load anything. On the console it prints:
          org.lorainelab.igb.bam.XAM initTheSeqs
          WARNING: Couldn't find sequences in file

          For a reachable valid URL but unauthorized this error message is shown:
          Format not recognized for file: https://drive.google.com/open?id=14KYlPVkMnvUuSnSAwDbwe400hqC3bKFf

          Please have a look.

          Show
          stiwari8 Srishti Tiwari (Inactive) added a comment - For the positive scenario, with a valid reachable URL on the quickload it works. On the console it prints: org.lorainelab.igb.bam.SAM initTheSeqs SEVERE: Sam file must be sorted by coordinate. For a reachable but invalid File, it does not load anything. On the console it prints: org.lorainelab.igb.bam.XAM initTheSeqs WARNING: Couldn't find sequences in file For a reachable valid URL but unauthorized this error message is shown: Format not recognized for file: https://drive.google.com/open?id=14KYlPVkMnvUuSnSAwDbwe400hqC3bKFf Please have a look.
          stiwari8 Srishti Tiwari (Inactive) made changes -
          Assignee Srishti Tiwari [ stiwari8 ] Kiran Korey [ kkorey ]
          Hide
          kkorey Kiran Korey (Inactive) added a comment -

          Srishti Tiwari

          I think that is the expected behavior.
          Because when you go to the URL google redirects you to a login page which is an unrecognized format for IGB.

          Show
          kkorey Kiran Korey (Inactive) added a comment - Srishti Tiwari I think that is the expected behavior. Because when you go to the URL google redirects you to a login page which is an unrecognized format for IGB.
          Hide
          stiwari8 Srishti Tiwari (Inactive) added a comment -

          Thank you for confirming this. Marking this as ready for pull request

          Show
          stiwari8 Srishti Tiwari (Inactive) added a comment - Thank you for confirming this. Marking this as ready for pull request
          stiwari8 Srishti Tiwari (Inactive) made changes -
          Status Needs 1st Level Review [ 10005 ] Reviewing [ 10301 ]
          stiwari8 Srishti Tiwari (Inactive) made changes -
          Status Reviewing [ 10301 ] Ready for Pull Request [ 10304 ]
          kkorey Kiran Korey (Inactive) made changes -
          Status Ready for Pull Request [ 10304 ] Pull Request Submitted [ 10101 ]
          kkorey Kiran Korey (Inactive) made changes -
          Assignee Kiran Korey [ kkorey ]
          ann.loraine Ann Loraine made changes -
          Fix Version/s 9.0.2 Minor Release [ 10600 ]
          ann.loraine Ann Loraine made changes -
          Rank Ranked higher
          ann.loraine Ann Loraine made changes -
          Sprint Fall 2018 Sprint 3, Fall 2018 Sprint 4 [ 53, 54 ] Fall 2018 Sprint 3, Fall 2018 Sprint 4, Fall 2018 Sprint 5 [ 53, 54, 55 ]
          ann.loraine Ann Loraine made changes -
          Rank Ranked lower
          Hide
          ann.loraine Ann Loraine added a comment -

          Merged to master branch.
          Tagged for release with 9.1.0

          Show
          ann.loraine Ann Loraine added a comment - Merged to master branch. Tagged for release with 9.1.0
          ann.loraine Ann Loraine made changes -
          Fix Version/s 9.1.0 Major Release [ 10601 ]
          ann.loraine Ann Loraine made changes -
          Status Pull Request Submitted [ 10101 ] Needs Testing [ 10002 ]
          ann.loraine Ann Loraine made changes -
          Resolution Done [ 10000 ]
          Status Needs Testing [ 10002 ] Closed [ 6 ]
          ann.loraine Ann Loraine made changes -
          Assignee Kiran Korey [ kkorey ]
          ann.loraine Ann Loraine made changes -
          Workflow Loraine Lab Workflow [ 18131 ] Fall 2019 Workflow Update [ 19956 ]
          ann.loraine Ann Loraine made changes -
          Workflow Fall 2019 Workflow Update [ 19956 ] Revised Fall 2019 Workflow Update [ 22077 ]

            People

            • Assignee:
              kkorey Kiran Korey (Inactive)
              Reporter:
              ann.loraine Ann Loraine
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: