Details
-
Type: Task
-
Status: Closed (View Workflow)
-
Priority: Major
-
Resolution: Done
-
Affects Version/s: None
-
Fix Version/s: None
-
Labels:None
-
Story Points:1.5
-
Epic Link:
-
Sprint:Fall 4 2021 Sep 27 - Oct 8, Fall 5 2021 Oct 11 - Oct 22
Description
See linked issue for track hub facade data sources URL for the "JASPAR" Track Hub with multiple bigbed files.
Loading these data into IGB is extremely slow for some reason.
Bigbed format files contain indexes that map chromosome positions onto file byte positions, which let client software programs (like IGB) look up exactly which part of the larger file they need and then use HTTP byte range requests to retrieve just the needed portion of the file. Therefore it is surprising that loading this file takes a long time with IGB. Retrieving and reading the index part of the file should be very fast, and requesting the bytes ought to be fast, as well.
For this task, investigate why it is so slow and suggest some ideas for how to speed it up.
The bb (bigbed) format is commonly used in bioinformatics for distributing and visualizing data, so it is important that we do a better job of supporting it.
Attachments
Issue Links
- relates to
-
IGBF-2948 Attempt to convert and load each track hub featured in the table interface
- Closed
-
IGBF-1938 Investigate: Why is bigwig loading so slow?
- To-Do
-
IGBF-2979 Investigate: bigBed chromosome synonyms logic
- To-Do
-
IGBF-2944 Update annots.xml url validation and filetype determination for trackhubs
- Closed
-
IGBF-2978 Improve bigBed parsing
- Closed
-
IGBF-1417 Create new artifact with bigbed and bigwig parsing code from IGV codebase
- Closed
-
IGBF-2911 Handle araTha1 synonym in track hub facade
- Closed
Downloaded the slow-loading bb file:
wget http://expdata.cmmt.ubc.ca/JASPAR/downloads/UCSC_tracks/2022/JASPAR2022_araTha1.bb
The file is more than 2 Gb in size.