[IGBF-3660] Include schema of UCSC data in the properties filed of the dataset. - JIRA UNCC

Details

Type: Task
Status: In Progress (View Workflow)
Priority: Blocker
Resolution: Unresolved
Affects Version/s: None
Fix Version/s: None
Labels:
None

Story Points:
2
Epic Link:
Improve IGB for users
Sprint:
Spring 6, Spring 7, Spring 8

Description

Pass the schema of the UCSC data in the properties field while constructing the dataset. This enables developers to extract the schema/metadata of the data easily. This metadata can be used to provide rich set of Filter by/Color by options to IGB users. Structure the metadata in the same way as actual data, for example, [attribute 1, attribute 2, ..., props].

Attachments

Options
- Sort By Name
- Sort By Date
- Ascending
- Descending
- Thumbnails
- List
- Download All

Attachments

Screenshot_1.png
211 kB
27/Mar/24 3:34 PM
Screenshot_2.png
64 kB
27/Mar/24 3:36 PM
Screenshot_3.png
118 kB
27/Mar/24 3:36 PM
Screenshot_4.png
72 kB
27/Mar/24 3:36 PM

Issue Links

is blocked by

IGBF-3640 Add expand color by and Filter by to include the props

To-Do

Activity

Ascending order - Click to sort in descending order

Hide

Permalink

Jaya Sravani Sirigineedi added a comment - 19/Mar/24 5:10 PM - edited

As per discussion with Kaushik Gopu, will add the asked columns in the featureProps variable in DataSet objects created while getting the available datasets using the schema response from the schema API, example API: https://api.genome.ucsc.edu/list/schema?genome=galGal6;track=nestedRepeats. Below is an example showing how the featureProps look like:

Example schema response:

Unable to find source-code formatter for language: json. Available languages are: actionscript, html, java, javascript, none, sql, xhtml, xml

// https://api.genome.ucsc.edu/list/schema?genome=galGal6;track=nestedRepeats
{
  "downloadTime": "2024:03:19T17:07:25Z",
  "downloadTimeStamp": 1710868045,
  "genome": "galGal6",
  "track": "nestedRepeats",
  "dataTime": "2018-10-11T14:52:43",
  "dataTimeStamp": 1539294763,
  "columnTypes": [
    {
      "name": "bin",
      "sqlType": "smallint(6)",
      "jsonType": "number",
      "description": "Indexing field to speed chromosome range queries"
    },
    {
      "name": "chrom",
      "sqlType": "varchar(255)",
      "jsonType": "string",
      "description": "Chromosome (or contig, scaffold, etc.)"
    },
    {
      "name": "chromStart",
      "sqlType": "int(10) unsigned",
      "jsonType": "number",
      "description": "Start position in chromosome"
    },
    {
      "name": "chromEnd",
      "sqlType": "int(10) unsigned",
      "jsonType": "number",
      "description": "End position in chromosome"
    },
    {
      "name": "name",
      "sqlType": "varchar(255)",
      "jsonType": "string",
      "description": "Name of item"
    },
    {
      "name": "score",
      "sqlType": "int(10) unsigned",
      "jsonType": "number",
      "description": "Average of fragment identity scores, transformed into 0..1000 range for shading"
    },
    {
      "name": "strand",
      "sqlType": "char(1)",
      "jsonType": "string",
      "description": "+, -, or . for mixed (some fragments +, some -)"
    },
    {
      "name": "thickStart",
      "sqlType": "int(10) unsigned",
      "jsonType": "number",
      "description": "for BED compatibility -- same as chromStart"
    },
    {
      "name": "thickEnd",
      "sqlType": "int(10) unsigned",
      "jsonType": "number",
      "description": "for BED compatibility -- same as chromEnd"
    },
    {
      "name": "reserved",
      "sqlType": "int(10) unsigned",
      "jsonType": "number",
      "description": "for BED compatibility"
    },
    {
      "name": "blockCount",
      "sqlType": "int(11)",
      "jsonType": "number",
      "description": "Number of blocks"
    },
    {
      "name": "blockSizes",
      "sqlType": "longblob",
      "jsonType": "string",
      "description": "Comma separated list of block (fragment) sizes"
    },
    {
      "name": "chromStarts",
      "sqlType": "longblob",
      "jsonType": "string",
      "description": "Start positions relative to chromStart"
    },
    {
      "name": "blockStrands",
      "sqlType": "longblob",
      "jsonType": "string",
      "description": "Strand of each fragment"
    },
    {
      "name": "id",
      "sqlType": "int(10) unsigned",
      "jsonType": "number",
      "description": "RepeatMasker-assigned ID used to join fragments"
    },
    {
      "name": "repClass",
      "sqlType": "varchar(255)",
      "jsonType": "string",
      "description": "Class of repeat"
    },
    {
      "name": "repFamily",
      "sqlType": "varchar(255)",
      "jsonType": "string",
      "description": "Family of repeat"
    }
  ],
  "shortLabel": "Interrupted Rpts",
  "type": "bed 12 +",
  "longLabel": "Fragments of Interrupted Repeats Joined by RepeatMasker ID",
  "itemCount": 29762,
  "visibility": "hide",
  "group": "varRep",
  "exonNumbers": "off",
  "useScore": "1"
}

featureProps for the above schema:

Unable to find source-code formatter for language: json. Available languages are: actionscript, html, java, javascript, none, sql, xhtml, xml

{ 
  "genome": "galGal6",
  "track": "nestedRepeats",
  "props": {
       "id": "number",
       "repClass": "string",
       "repFamily": "string"
  }
}

Show

Jaya Sravani Sirigineedi added a comment - 19/Mar/24 5:10 PM - edited As per discussion with Kaushik Gopu , will add the asked columns in the featureProps variable in DataSet objects created while getting the available datasets using the schema response from the schema API, example API: https://api.genome.ucsc.edu/list/schema?genome=galGal6;track=nestedRepeats . Below is an example showing how the featureProps look like: Example schema response: Unable to find source-code formatter for language: json. Available languages are: actionscript, html, java, javascript, none, sql, xhtml, xml // https://api.genome.ucsc.edu/list/schema?genome=galGal6;track=nestedRepeats { "downloadTime" : "2024:03:19T17:07:25Z" , "downloadTimeStamp" : 1710868045, "genome" : "galGal6" , "track" : "nestedRepeats" , "dataTime" : "2018-10-11T14:52:43" , "dataTimeStamp" : 1539294763, "columnTypes" : [ { "name" : "bin" , "sqlType" : "smallint(6)" , "jsonType" : "number" , "description" : "Indexing field to speed chromosome range queries" }, { "name" : "chrom" , "sqlType" : "varchar(255)" , "jsonType" : "string" , "description" : "Chromosome (or contig, scaffold, etc.)" }, { "name" : "chromStart" , "sqlType" : " int (10) unsigned" , "jsonType" : "number" , "description" : "Start position in chromosome" }, { "name" : "chromEnd" , "sqlType" : " int (10) unsigned" , "jsonType" : "number" , "description" : "End position in chromosome" }, { "name" : "name" , "sqlType" : "varchar(255)" , "jsonType" : "string" , "description" : "Name of item" }, { "name" : "score" , "sqlType" : " int (10) unsigned" , "jsonType" : "number" , "description" : "Average of fragment identity scores, transformed into 0..1000 range for shading" }, { "name" : "strand" , "sqlType" : " char (1)" , "jsonType" : "string" , "description" : "+, -, or . for mixed (some fragments +, some -)" }, { "name" : "thickStart" , "sqlType" : " int (10) unsigned" , "jsonType" : "number" , "description" : " for BED compatibility -- same as chromStart" }, { "name" : "thickEnd" , "sqlType" : " int (10) unsigned" , "jsonType" : "number" , "description" : " for BED compatibility -- same as chromEnd" }, { "name" : "reserved" , "sqlType" : " int (10) unsigned" , "jsonType" : "number" , "description" : " for BED compatibility" }, { "name" : "blockCount" , "sqlType" : " int (11)" , "jsonType" : "number" , "description" : " Number of blocks" }, { "name" : "blockSizes" , "sqlType" : "longblob" , "jsonType" : "string" , "description" : "Comma separated list of block (fragment) sizes" }, { "name" : "chromStarts" , "sqlType" : "longblob" , "jsonType" : "string" , "description" : "Start positions relative to chromStart" }, { "name" : "blockStrands" , "sqlType" : "longblob" , "jsonType" : "string" , "description" : "Strand of each fragment" }, { "name" : "id" , "sqlType" : " int (10) unsigned" , "jsonType" : "number" , "description" : "RepeatMasker-assigned ID used to join fragments" }, { "name" : "repClass" , "sqlType" : "varchar(255)" , "jsonType" : "string" , "description" : " Class of repeat" }, { "name" : "repFamily" , "sqlType" : "varchar(255)" , "jsonType" : "string" , "description" : "Family of repeat" } ], "shortLabel" : "Interrupted Rpts" , "type" : "bed 12 +" , "longLabel" : "Fragments of Interrupted Repeats Joined by RepeatMasker ID" , "itemCount" : 29762, "visibility" : "hide" , "group" : "varRep" , "exonNumbers" : "off" , "useScore" : "1" } featureProps for the above schema: Unable to find source-code formatter for language: json. Available languages are: actionscript, html, java, javascript, none, sql, xhtml, xml { "genome" : "galGal6" , "track" : "nestedRepeats" , "props" : { "id" : "number" , "repClass" : "string" , "repFamily" : "string" } }

11 older comments

Hide

Permalink

Ann Loraine added a comment - 11/Apr/24 1:04 PM

PR is now merged. Thank you Jaya Sravani Sirigineedi for letting me know you were waiting on this!

Show

Ann Loraine added a comment - 11/Apr/24 1:04 PM PR is now merged. Thank you Jaya Sravani Sirigineedi for letting me know you were waiting on this!

Hide

Permalink

Nowlan Freese added a comment - 18/Apr/24 10:45 AM

After discussion with Sravani, we have decided to attempt to only make the schema API call when a user selects a specific data set.

For example, a user selects the galGal6 genome (no schema API calls are made). Then, under Available Data > UCSC REST (UCSC REST), when the user selects cpgIslandExt the schema API call is made for just cpgIslandExt.

Show

Nowlan Freese added a comment - 18/Apr/24 10:45 AM After discussion with Sravani, we have decided to attempt to only make the schema API call when a user selects a specific data set. For example, a user selects the galGal6 genome (no schema API calls are made). Then, under Available Data > UCSC REST (UCSC REST), when the user selects cpgIslandExt the schema API call is made for just cpgIslandExt.

Hide

Permalink

Jaya Sravani Sirigineedi added a comment - 18/Apr/24 11:24 AM

Started investigating on where to add the integration of this schema API i.e., have to find what part of code is responsible for displaying the empty track when user clicks on the track checkbox under Available Data.

Show

Jaya Sravani Sirigineedi added a comment - 18/Apr/24 11:24 AM Started investigating on where to add the integration of this schema API i.e., have to find what part of code is responsible for displaying the empty track when user clicks on the track checkbox under Available Data.

Hide

Permalink

Jaya Sravani Sirigineedi added a comment - 23/Apr/24 8:55 AM

When the user clicks on the track checkbox under Available Data, the addFeatureTier method in GeneralLoadUtils is being called which uses the addEmptyTierFor method from TrackView class to display both the forward and reverse empty tracks. Code can be added here to get the schema for the selected track as the Dataset is available in the method. But the only problem is right now in Kaushik Gopu implementation he is getting tracks from the DataProvider again, this might override the featureProps and assign it to null, have to chack whether the same dataset can be used in his implementation as this is saved in a ITrackStyleExtended instance as well.

Show

Jaya Sravani Sirigineedi added a comment - 23/Apr/24 8:55 AM When the user clicks on the track checkbox under Available Data, the addFeatureTier method in GeneralLoadUtils is being called which uses the addEmptyTierFor method from TrackView class to display both the forward and reverse empty tracks. Code can be added here to get the schema for the selected track as the Dataset is available in the method. But the only problem is right now in Kaushik Gopu implementation he is getting tracks from the DataProvider again, this might override the featureProps and assign it to null, have to chack whether the same dataset can be used in his implementation as this is saved in a ITrackStyleExtended instance as well.

Hide

Permalink

Jaya Sravani Sirigineedi added a comment - 24/Apr/24 5:25 PM

Had a discussion with Kaushik Gopu and we think the above approach might work, implementing it right now. Will update here once it's tested.

Show

Jaya Sravani Sirigineedi added a comment - 24/Apr/24 5:25 PM Had a discussion with Kaushik Gopu and we think the above approach might work, implementing it right now. Will update here once it's tested.

Include schema of UCSC data in the properties filed of the dataset.

Details

Description

Attachments

Attachments

Issue Links

Activity

People

Dates