Uploaded image for project: 'IGB'
  1. IGB
  2. IGBF-2954

Investigate: why is loading bb (bigbed) and bigwig (bw) files so slow?

        Details

        • Type: Task
        • Status: Closed (View Workflow)
        • Priority: Major
        • Resolution: Done
        • Affects Version/s: None
        • Fix Version/s: None
        • Labels:
          None
        • Story Points:
          1.5
        • Epic Link:
        • Sprint:
          Fall 4 2021 Sep 27 - Oct 8, Fall 5 2021 Oct 11 - Oct 22

          Description

          See linked issue for track hub facade data sources URL for the "JASPAR" Track Hub with multiple bigbed files.

          Loading these data into IGB is extremely slow for some reason.

          Bigbed format files contain indexes that map chromosome positions onto file byte positions, which let client software programs (like IGB) look up exactly which part of the larger file they need and then use HTTP byte range requests to retrieve just the needed portion of the file. Therefore it is surprising that loading this file takes a long time with IGB. Retrieving and reading the index part of the file should be very fast, and requesting the bytes ought to be fast, as well.

          For this task, investigate why it is so slow and suggest some ideas for how to speed it up.

          The bb (bigbed) format is commonly used in bioinformatics for distributing and visualizing data, so it is important that we do a better job of supporting it.

            Attachments

              Issue Links

              relates to

              Task - A task that needs to be done. IGBF-2948 Attempt to convert and load each track hub featured in the table interface

              • Major - Major loss of function.
              • Closed

              Task - A task that needs to be done. IGBF-1938 Investigate: Why is bigwig loading so slow?

              • Major - Major loss of function.
              • To-Do

              Task - A task that needs to be done. IGBF-2979 Investigate: bigBed chromosome synonyms logic

              • Major - Major loss of function.
              • To-Do

              Improvement - An improvement or enhancement to an existing feature or task. IGBF-2944 Update annots.xml url validation and filetype determination for trackhubs

              • Major - Major loss of function.
              • Closed

              Improvement - An improvement or enhancement to an existing feature or task. IGBF-2978 Improve bigBed parsing

              • Major - Major loss of function.
              • Closed

              Task - A task that needs to be done. IGBF-1417 Create new artifact with bigbed and bigwig parsing code from IGV codebase

              • Major - Major loss of function.
              • Closed

              Improvement - An improvement or enhancement to an existing feature or task. IGBF-2911 Handle araTha1 synonym in track hub facade

              • Major - Major loss of function.
              • Closed

                Activity

                  People

                  • Assignee:
                    nfreese Nowlan Freese
                    Reporter:
                    ann.loraine Ann Loraine
                  • Votes:
                    0 Vote for this issue
                    Watchers:
                    2 Start watching this issue

                    Dates

                    • Created:
                      Updated:
                      Resolved: