Details
-
Type:
Task
-
Status: To-Do (View Workflow)
-
Priority:
Major
-
Resolution: Unresolved
-
Affects Version/s: None
-
Fix Version/s: None
-
Labels:
-
Story Points:2
-
Epic Link:
-
Sprint:Summer 2019 Sprint 12, Spring 3 : 17 Feb to 28 Feb, Spring 4 : 2 Mar to 13 Mar, Spring 5 : 16 Mar to 27 Mar, Spring 6 : 30 Mar to Apr 10, Spring 8 : 24 Apr to 8 May
Description
Watch this: https://drive.google.com/file/d/1uZHOoVkl-AgQ-jogSGxFH5zulD3wNQeS/view?usp=sharing
The bigwig format is a graph data file format that is supposed to be random access. It contains a "header" at the beginning of the file that maps locations in the file to locations in the genome.
When users open and then click "Load Data" for a small region in the genome, loading the bigwig file's data should feel very fast because IGB only needs to send a byte range request for a small part of the file.
However, this does not seem to be happening.
In fact, the same exact data converted to a different format - tabix-indexed bedgraph - loads much, much faster.
This task is to investigate what is going on. Why is bw file loading so incredibly slow?
Please report what you find as comments in this ticket.
See below comment for example data.
Attachments
Issue Links
- relates to
-
-
- Closed
-
Activity
| Field | Original Value | New Value |
|---|---|---|
| Epic Link | IGBF-1765 [ 17855 ] |
| Rank | Ranked higher |
| Assignee | Ann Loraine [ aloraine ] |
| Sprint | Summer 2019 Sprint 12 [ 71 ] | Summer 2019 Sprint 12, Fall 2019 Sprint 1 [ 71, 72 ] |
| Rank | Ranked higher |
| Rank | Ranked higher |
| Workflow | Loraine Lab Workflow [ 18662 ] | Fall 2019 Workflow Update [ 19012 ] |
| Sprint | Summer 2019 Sprint 12, Fall 2019 Sprint 1 [ 71, 72 ] | Summer 2019 Sprint 12, Fall 2019 Sprint 2 [ 71, 73 ] |
| Sprint | Summer 2019 Sprint 12, Fall 2019 Sprint 2 [ 71, 73 ] | Summer 2019 Sprint 12, Fall 2019 Sprint 8 [ 71, 78 ] |
| Workflow | Fall 2019 Workflow Update [ 19012 ] | Revised Fall 2019 Workflow Update [ 21123 ] |
| Status | Open [ 1 ] | In Progress [ 3 ] |
| Sprint | Summer 2019 Sprint 12, Fall 7 : 11 Nov to 22 Nov [ 71, 78 ] | Summer 2019 Sprint 12 [ 71 ] |
| Rank | Ranked higher |
| Status | In Progress [ 3 ] | To-Do [ 10305 ] |
| Sprint | Summer 2019 Sprint 12 [ 71 ] | Summer 2019 Sprint 12, Spring 2 : 20 Jan to 31 Jan [ 71, 85 ] |
| Rank | Ranked higher |
| Sprint | Summer 2019 Sprint 12, Spring 2 : 20 Jan to 31 Jan [ 71, 85 ] | Summer 2019 Sprint 12, Spring 3 : 3 Feb to 14 Feb [ 71, 86 ] |
| Rank | Ranked lower |
| Rank | Ranked lower |
| Sprint | Summer 2019 Sprint 12, Spring 3 : 3 Feb to 14 Feb [ 71, 86 ] | Summer 2019 Sprint 12, Spring 3 : 17 Feb to 21 Feb 2 [ 71, 87 ] |
| Rank | Ranked higher |
| Sprint | Summer 2019 Sprint 12, Spring 3 : 17 Feb to 28 Feb [ 71, 87 ] | Summer 2019 Sprint 12, Spring 3 : 17 Feb to 28 Feb, Spring 4 : 2 Mar to 13 Mar [ 71, 87, 89 ] |
| Rank | Ranked higher |
| Sprint | Summer 2019 Sprint 12, Spring 3 : 17 Feb to 28 Feb, Spring 4 : 2 Mar to 13 Mar [ 71, 87, 89 ] | Summer 2019 Sprint 12, Spring 3 : 17 Feb to 28 Feb, Spring 4 : 2 Mar to 13 Mar, Spring 5 : 16 Mar to 27 Mar [ 71, 87, 89, 90 ] |
| Rank | Ranked higher |
| Assignee | Ann Loraine [ aloraine ] |
| Status | To-Do [ 10305 ] | In Progress [ 3 ] |
| Description |
The bigwig format is a graph data file format that is supposed to be random access. It contains a "header" at the beginning of the file that maps locations in the file to locations in the genome.
When users open and then click "Load Data" for a small region in the genome, loading the bigwig file's data should feel very fast because IGB only needs to send a byte range request for a small part of the file. However, this does not seem to be happening. This task is to investigate what is going on. Why is it so slow? Please report what you find as comments in this ticket. Contact Dr. Loraine for some example data. |
The bigwig format is a graph data file format that is supposed to be random access. It contains a "header" at the beginning of the file that maps locations in the file to locations in the genome.
When users open and then click "Load Data" for a small region in the genome, loading the bigwig file's data should feel very fast because IGB only needs to send a byte range request for a small part of the file. However, this does not seem to be happening. In fact, the same exact data converted to a different format - tabix-indexed bedgraph - loads much, much faster. This task is to investigate what is going on. Why is bw file loading so incredibly slow? Please report what you find as comments in this ticket. See below comment for example data. |
| Assignee | Ann Loraine [ aloraine ] |
| Status | In Progress [ 3 ] | To-Do [ 10305 ] |
| Description |
The bigwig format is a graph data file format that is supposed to be random access. It contains a "header" at the beginning of the file that maps locations in the file to locations in the genome.
When users open and then click "Load Data" for a small region in the genome, loading the bigwig file's data should feel very fast because IGB only needs to send a byte range request for a small part of the file. However, this does not seem to be happening. In fact, the same exact data converted to a different format - tabix-indexed bedgraph - loads much, much faster. This task is to investigate what is going on. Why is bw file loading so incredibly slow? Please report what you find as comments in this ticket. See below comment for example data. |
Watch this: https://drive.google.com/file/d/1uZHOoVkl-AgQ-jogSGxFH5zulD3wNQeS/view?usp=sharing
The bigwig format is a graph data file format that is supposed to be random access. It contains a "header" at the beginning of the file that maps locations in the file to locations in the genome. When users open and then click "Load Data" for a small region in the genome, loading the bigwig file's data should feel very fast because IGB only needs to send a byte range request for a small part of the file. However, this does not seem to be happening. In fact, the same exact data converted to a different format - tabix-indexed bedgraph - loads much, much faster. This task is to investigate what is going on. Why is bw file loading so incredibly slow? Please report what you find as comments in this ticket. See below comment for example data. |
| Sprint | Summer 2019 Sprint 12, Spring 3 : 17 Feb to 28 Feb, Spring 4 : 2 Mar to 13 Mar, Spring 5 : 16 Mar to 27 Mar [ 71, 87, 89, 90 ] | Summer 2019 Sprint 12, Spring 3 : 17 Feb to 28 Feb, Spring 4 : 2 Mar to 13 Mar, Spring 5 : 16 Mar to 27 Mar, Spring 6 : 30 Mar to Apr 10 [ 71, 87, 89, 90, 91 ] |
| Rank | Ranked higher |
| Rank | Ranked higher |
| Sprint | Summer 2019 Sprint 12, Spring 3 : 17 Feb to 28 Feb, Spring 4 : 2 Mar to 13 Mar, Spring 5 : 16 Mar to 27 Mar, Spring 6 : 30 Mar to Apr 10 [ 71, 87, 89, 90, 91 ] | Summer 2019 Sprint 12, Spring 3 : 17 Feb to 28 Feb, Spring 4 : 2 Mar to 13 Mar, Spring 5 : 16 Mar to 27 Mar, Spring 6 : 30 Mar to Apr 10, Spring 7 : 13 Apr to 24 Apr [ 71, 87, 89, 90, 91, 92 ] |
| Rank | Ranked higher |
| Sprint | Summer 2019 Sprint 12, Spring 3 : 17 Feb to 28 Feb, Spring 4 : 2 Mar to 13 Mar, Spring 5 : 16 Mar to 27 Mar, Spring 6 : 30 Mar to Apr 10, Spring 7 : 13 Apr to 24 Apr [ 71, 87, 89, 90, 91, 92 ] | Summer 2019 Sprint 12, Spring 3 : 17 Feb to 28 Feb, Spring 4 : 2 Mar to 13 Mar, Spring 5 : 16 Mar to 27 Mar, Spring 6 : 30 Mar to Apr 10, Spring 8 : 24 Apr to 8 May [ 71, 87, 89, 90, 91, 93 ] |
| Rank | Ranked lower |
| Rank | Ranked higher |
| Sprint | Summer 2019 Sprint 12, Spring 3 : 17 Feb to 28 Feb, Spring 4 : 2 Mar to 13 Mar, Spring 5 : 16 Mar to 27 Mar, Spring 6 : 30 Mar to Apr 10, Spring 8 : 24 Apr to 8 May [ 71, 87, 89, 90, 91, 93 ] | Summer 2019 Sprint 12, Spring 3 : 17 Feb to 28 Feb, Spring 4 : 2 Mar to 13 Mar, Spring 5 : 16 Mar to 27 Mar, Spring 6 : 30 Mar to Apr 10, Spring 8 : 24 Apr to 8 May, Spring 8 : 11 May to 25 May [ 71, 87, 89, 90, 91, 93, 94 ] |
| Rank | Ranked higher |
| Sprint | Summer 2019 Sprint 12, Spring 3 : 17 Feb to 28 Feb, Spring 4 : 2 Mar to 13 Mar, Spring 5 : 16 Mar to 27 Mar, Spring 6 : 30 Mar to Apr 10, Spring 8 : 24 Apr to 8 May, Spring 8 : 11 May to 25 May [ 71, 87, 89, 90, 91, 93, 94 ] | Summer 2019 Sprint 12, Spring 3 : 17 Feb to 28 Feb, Spring 4 : 2 Mar to 13 Mar, Spring 5 : 16 Mar to 27 Mar, Spring 6 : 30 Mar to Apr 10, Spring 8 : 24 Apr to 8 May [ 71, 87, 89, 90, 91, 93 ] |
| Link | This issue relates to IGBF-2195 [ IGBF-2195 ] |
| Rank | Ranked higher |
| Rank | Ranked higher |