Uploaded image for project: 'IGB'
  1. IGB
  2. IGBF-2852

Document the different requests IGB makes when using a Quickload site

    Details

    • Type: Task
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None
    • Story Points:
      1
    • Sprint:
      Spring 4 2021 May 3 - May 14, Spring 5 2021 May 17 - May 28, Spring 6 2021 May 31 - June 11, Summer 1 2021 Jun 14 - Jun 25

      Description

      When you add a new Quickload site to IGB, IGB then requests a number of files (endpoints) from the site, some of which are optional. These include:

      • contents.txt - tab-delimited file IGB uses this to determine which genome versions are supported on the site
      • species.txt - optional
      • chromosomes.txt - optional
      • synonyms.txt - optional

      Later, if a user selects one of the genomes supported by the site, IGB next requests:

      genome_version/annots.xml

      The file "annots.xml" must be located in a subdirectory of the quickload root directory, in a folder named for the genome version indicated in contents.txt which was requested before that.

      IGB then reads the annots.xml file to identify which data sets are available for visualization. IGB uses the "annots.xml" data to populate the list of available datasets from that quickload site. The "annots.xml" file is similar to the "trackdb" file in a Track Hub in that it contains a list of datasets that can be loaded.

      For this task, please review the above information and make sure it is correct by looking at the IGB logs as you add and use a quickload site.

        Attachments

          Activity

          Hide
          chirag24 Chirag Chandrahas Shetty (Inactive) added a comment - - edited

          [~aloraine] Is genome.txt file also optional ?

          Show
          chirag24 Chirag Chandrahas Shetty (Inactive) added a comment - - edited [~aloraine] Is genome.txt file also optional ?
          Hide
          ann.loraine Ann Loraine added a comment -

          I am not sure what happens if IGB cannot access this resource. I think an error will be generated, but I'm not sure.

          Show
          ann.loraine Ann Loraine added a comment - I am not sure what happens if IGB cannot access this resource. I think an error will be generated, but I'm not sure.
          Hide
          inaylor Irvin Naylor (Inactive) added a comment - - edited

          Genome.txt - There seems to be no error or reference to it one way or another in the log. I've done some testing with multiple quickload local files and have found that it will open and function the same. The only major difference is the genome length is blank on the current genome tab.

          Species.txt - confirmed optional, log "Optional species.txt could not be loaded from: http://igbquickload.org/quickload/A_thaliana_Jun_2009/species.txt"

          synonyms.txt - confirmed optional, log: Optional quickload synonyms.txt file could not be loaded from http://igbquickload.org/quickload/A_thaliana_Jun_2009/synonyms.txt

          Contents.txt is required, log states it will not be able to read from it. log: (Could not read contents.txt from: file:/C:/Users/Inayl/Desktop/Trex/contents.txt
          )

          Chromosomes.txt - seems to be an optional file as no reference about it appears in the log. Tested on local quickload files. All quickload sources were created and used with no issue without one.

          As for the annots.xml file, there haven't been any references in the log to it specifically. The only thing that comes is a reference to the species_genome being loaded and how long it took. However, it is required and its omission states "Invalid file path or annots.xml file not found." It's safe to assume in the case of a track.db, this functions as the main component to a new quickload.

          Show
          inaylor Irvin Naylor (Inactive) added a comment - - edited Genome.txt - There seems to be no error or reference to it one way or another in the log. I've done some testing with multiple quickload local files and have found that it will open and function the same. The only major difference is the genome length is blank on the current genome tab. Species.txt - confirmed optional, log "Optional species.txt could not be loaded from: http://igbquickload.org/quickload/A_thaliana_Jun_2009/species.txt " synonyms.txt - confirmed optional, log: Optional quickload synonyms.txt file could not be loaded from http://igbquickload.org/quickload/A_thaliana_Jun_2009/synonyms.txt Contents.txt is required, log states it will not be able to read from it. log: (Could not read contents.txt from: file:/C:/Users/Inayl/Desktop/Trex/contents.txt ) Chromosomes.txt - seems to be an optional file as no reference about it appears in the log. Tested on local quickload files. All quickload sources were created and used with no issue without one. As for the annots.xml file, there haven't been any references in the log to it specifically. The only thing that comes is a reference to the species_genome being loaded and how long it took. However, it is required and its omission states "Invalid file path or annots.xml file not found." It's safe to assume in the case of a track.db, this functions as the main component to a new quickload.
          Hide
          ann.loraine Ann Loraine added a comment -

          Chirag Chandrahas Shetty: Using Microsoft Network Monitor which records all requests made from an application. Shows this as a tree, with application as the root. Also tried WireShark, but the interface / data display was less useful. Only displays the request and not by application.

          Show
          ann.loraine Ann Loraine added a comment - Chirag Chandrahas Shetty : Using Microsoft Network Monitor which records all requests made from an application. Shows this as a tree, with application as the root. Also tried WireShark, but the interface / data display was less useful. Only displays the request and not by application.
          Hide
          chirag24 Chirag Chandrahas Shetty (Inactive) added a comment - - edited

          Requests made by IGB:

          On Start:

          Host: lorainelab-quickload.scidas.org
          Http: Request, GET /rnaseq/contents.txt

          Host: lorainelab-quickload.scidas.org(optional)
          Http: Request, GET /rnaseq/synonyms.txt

          Host: lorainelab-quickload.scidas.org(optional)
          Http: Request, GET /rnaseq/species.txt

          Host: lorainelab-quickload.scidas.org
          Http: Request, GET /quickload/contents.txt

          Host: lorainelab-quickload.scidas.org
          Http: Request, GET /quickload/synonyms.txt

          Host: lorainelab-quickload.scidas.org
          Http: Request, GET /quickload/species.txt

          Host: lorainelab-quickload.scidas.org
          Http: Request, GET /blueberry/contents.txt

          Host: lorainelab-quickload.scidas.org
          Http: Request, GET /blueberry/synonyms.txt (Not Found)

          Host: lorainelab-quickload.scidas.org
          Http: Request, GET /blueberry/species.txt (Not Found)

          Requests after selecting a Genome:

          Host: lorainelab-quickload.scidas.org
          Http: Request, GET /rnaseq/H_sapiens_Dec_2013/genome.txt

          Host: lorainelab-quickload.scidas.org
          Http: Request, GET /rnaseq/H_sapiens_Dec_2013/annots.xml

          Host: lorainelab-quickload.scidas.org
          Http: Request, GET /rnaseq/favicon.ico

          Requests made by IGB from annots.xml file

          Host: lorainelab-quickload.scidas.org
          Http: Request, GET /quickload/H_sapiens_Dec_2013/H_sapiens_Dec_2013_ncbiRefSeqCurated.bed.gz

          Host: lorainelab-quickload.scidas.org
          Http: Request, GET /quickload/H_sapiens_Dec_2013/cytoBand.cyt

          Host: lorainelab-quickload.scidas.org
          Http: Request, HEAD /quickload/H_sapiens_Dec_2013/H_sapiens_Dec_2013_ncbiRefSeqCurated.bed.gz

          Host: lorainelab-quickload.scidas.org
          Http: Request, GET /quickload/H_sapiens_Dec_2013/H_sapiens_Dec_2013_ncbiRefSeqCurated.bed.gz

          Host: lorainelab-quickload.scidas.org
          Http: Request, HEAD /quickload/H_sapiens_Dec_2013/H_sapiens_Dec_2013_ncbiRefSeqCurated.bed.gz.tbi

          Host: lorainelab-quickload.scidas.org
          Http: Request, GET /quickload/H_sapiens_Dec_2013/cytoBand.cyt

          Show
          chirag24 Chirag Chandrahas Shetty (Inactive) added a comment - - edited Requests made by IGB: On Start: Host: lorainelab-quickload.scidas.org Http: Request, GET /rnaseq/contents.txt Host: lorainelab-quickload.scidas.org(optional) Http: Request, GET /rnaseq/synonyms.txt Host: lorainelab-quickload.scidas.org(optional) Http: Request, GET /rnaseq/species.txt Host: lorainelab-quickload.scidas.org Http: Request, GET /quickload/contents.txt Host: lorainelab-quickload.scidas.org Http: Request, GET /quickload/synonyms.txt Host: lorainelab-quickload.scidas.org Http: Request, GET /quickload/species.txt Host: lorainelab-quickload.scidas.org Http: Request, GET /blueberry/contents.txt Host: lorainelab-quickload.scidas.org Http: Request, GET /blueberry/synonyms.txt (Not Found) Host: lorainelab-quickload.scidas.org Http: Request, GET /blueberry/species.txt (Not Found) Requests after selecting a Genome: Host: lorainelab-quickload.scidas.org Http: Request, GET /rnaseq/H_sapiens_Dec_2013/genome.txt Host: lorainelab-quickload.scidas.org Http: Request, GET /rnaseq/H_sapiens_Dec_2013/annots.xml Host: lorainelab-quickload.scidas.org Http: Request, GET /rnaseq/favicon.ico Requests made by IGB from annots.xml file Host: lorainelab-quickload.scidas.org Http: Request, GET /quickload/H_sapiens_Dec_2013/H_sapiens_Dec_2013_ncbiRefSeqCurated.bed.gz Host: lorainelab-quickload.scidas.org Http: Request, GET /quickload/H_sapiens_Dec_2013/cytoBand.cyt Host: lorainelab-quickload.scidas.org Http: Request, HEAD /quickload/H_sapiens_Dec_2013/H_sapiens_Dec_2013_ncbiRefSeqCurated.bed.gz Host: lorainelab-quickload.scidas.org Http: Request, GET /quickload/H_sapiens_Dec_2013/H_sapiens_Dec_2013_ncbiRefSeqCurated.bed.gz Host: lorainelab-quickload.scidas.org Http: Request, HEAD /quickload/H_sapiens_Dec_2013/H_sapiens_Dec_2013_ncbiRefSeqCurated.bed.gz.tbi Host: lorainelab-quickload.scidas.org Http: Request, GET /quickload/H_sapiens_Dec_2013/cytoBand.cyt
          Hide
          inaylor Irvin Naylor (Inactive) added a comment -

          Review: Looking at much of what we've compiled, it would seem at this point, many of the major requests that we need to make use of have been captured for later use. I'll move this to closed since I don't think we have much else to add on this one.

          Show
          inaylor Irvin Naylor (Inactive) added a comment - Review: Looking at much of what we've compiled, it would seem at this point, many of the major requests that we need to make use of have been captured for later use. I'll move this to closed since I don't think we have much else to add on this one.

            People

            • Assignee:
              chirag24 Chirag Chandrahas Shetty (Inactive)
              Reporter:
              ann.loraine Ann Loraine
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: