Details

    • Type: Task
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None

      Description

      Task: Add the Vanessa cardui genome and annotation to IGB.

      Link to genome on NCBI: https://www.ncbi.nlm.nih.gov/datasets/genome/GCF_905220365.1/

        Attachments

          Issue Links

            Activity

            Hide
            pkulzer Paige Kulzer (Inactive) added a comment -

            Below is an outline of the steps I followed to create this Quickload:

            1.Use wget to obtain the .2bit file from UCSC's track hub directory, then rename it

            wget https://hgdownload.soe.ucsc.edu/hubs/GCF/905/220/365/GCF_905220365.1/GCF_905220365.1.2bit
            mv GCF_905220365.1.2bit ilVanCard2.1.2bit
            

            2. Create genome.txt, then check that the chromosome's are ordered logically (i.e., numerically)

            ./twoBitInfo ilVanCard2.1.2bit genome.txt
            cat genome.txt
            

            3. Use Vanessa cardui's taxID (171605) to get the information needed from gene2accession.gz and gene_info.gz to create the BED14 file in a later step

            gunzip -c gene2accession.gz | grep '^171605\t' > 171605.gene2accession.txt
            gunzip -c gene_info.gz | grep '^171605\t' > 171605.gene_info.txt
            

            4. Download the RefSeqAll BED file from UCSC's table browser (Link: https://genome.ucsc.edu/cgi-bin/hgTables), then create the BED14 file using the following code:

            cd ~/Documents/Repos/genomesource/
            ./ucscToBedDetail.py -a ~/Downloads/171605.gene2accession.txt -g ~/Downloads/171605.gene_info.txt ~/Downloads/V_cardui_ncbiRefSeq.bed.gz ~/Downloads/V_cardui_Feb_2021_ncbiRefSeq.bed
            

            5. Sort, gzip, and tabix the BED14 file

            cd ~/Downloads/
            sort -k1,1 -k2,2n V_cardui_Feb_2021_ncbiRefSeq.bed | bgzip > V_cardui_Feb_2021_ncbiRefSeq.bed.gz
            tabix -0 -s 1 -b 2 -e 3 V_cardui_Feb_2021_ncbiRefSeq.bed.gz
            

            6. Sanity check the 2bit and BED files - Add the 2bit file as a reference, then drag/drop the BED files into IGB. Confirm that gene models are present, labeled correctly, and that no error messages are present in the Log.

            7. Create a new directory in the quickload repo, then create annots.xml

            cd ~/Documents/Repos/quickload/
            svn mkdir V_cardui_Feb_2021
            svn cp A_gambiae_Feb_2003/annots.xml V_cardui_Feb_2021
            nano V_cardui_Feb_2021/annots.xml
            

            8. Add V_cardui_Feb_2021 to contents.txt and .htaccess

            nano contents.txt
            

            V_cardui_Feb_2021 Vanessa cardui (Feb 2021) painted lady (ilVanCard2.1)

            nano .htaccess
            

            AddDescription "Vanessa cardui (Feb 2021) painted lady (ilVanCard2.1)" V_cardui_Feb_2021

            9. Create HEADER.md

            ../genomesource/writeQuickLoadHeaderUCSC.py V_cardui_Feb_2021 > V_cardui_Feb_2021/HEADER.md
            
            Show
            pkulzer Paige Kulzer (Inactive) added a comment - Below is an outline of the steps I followed to create this Quickload: 1.Use wget to obtain the .2bit file from UCSC's track hub directory, then rename it wget https: //hgdownload.soe.ucsc.edu/hubs/GCF/905/220/365/GCF_905220365.1/GCF_905220365.1.2bit mv GCF_905220365.1.2bit ilVanCard2.1.2bit 2. Create genome.txt, then check that the chromosome's are ordered logically (i.e., numerically) ./twoBitInfo ilVanCard2.1.2bit genome.txt cat genome.txt 3. Use Vanessa cardui 's taxID (171605) to get the information needed from gene2accession.gz and gene_info.gz to create the BED14 file in a later step gunzip -c gene2accession.gz | grep '^171605\t' > 171605.gene2accession.txt gunzip -c gene_info.gz | grep '^171605\t' > 171605.gene_info.txt 4. Download the RefSeqAll BED file from UCSC's table browser (Link: https://genome.ucsc.edu/cgi-bin/hgTables ), then create the BED14 file using the following code: cd ~/Documents/Repos/genomesource/ ./ucscToBedDetail.py -a ~/Downloads/171605.gene2accession.txt -g ~/Downloads/171605.gene_info.txt ~/Downloads/V_cardui_ncbiRefSeq.bed.gz ~/Downloads/V_cardui_Feb_2021_ncbiRefSeq.bed 5. Sort, gzip, and tabix the BED14 file cd ~/Downloads/ sort -k1,1 -k2,2n V_cardui_Feb_2021_ncbiRefSeq.bed | bgzip > V_cardui_Feb_2021_ncbiRefSeq.bed.gz tabix -0 -s 1 -b 2 -e 3 V_cardui_Feb_2021_ncbiRefSeq.bed.gz 6. Sanity check the 2bit and BED files - Add the 2bit file as a reference, then drag/drop the BED files into IGB. Confirm that gene models are present, labeled correctly, and that no error messages are present in the Log. 7. Create a new directory in the quickload repo, then create annots.xml cd ~/Documents/Repos/quickload/ svn mkdir V_cardui_Feb_2021 svn cp A_gambiae_Feb_2003/annots.xml V_cardui_Feb_2021 nano V_cardui_Feb_2021/annots.xml 8. Add V_cardui_Feb_2021 to contents.txt and .htaccess nano contents.txt V_cardui_Feb_2021 Vanessa cardui (Feb 2021) painted lady (ilVanCard2.1) nano .htaccess AddDescription "Vanessa cardui (Feb 2021) painted lady (ilVanCard2.1)" V_cardui_Feb_2021 9. Create HEADER.md ../genomesource/writeQuickLoadHeaderUCSC.py V_cardui_Feb_2021 > V_cardui_Feb_2021/HEADER.md
            Hide
            pkulzer Paige Kulzer (Inactive) added a comment - - edited

            For review: I have zipped up a folder containing all of the relevant files needed to create this Quickload and place that in the shared Google Drive. Please take a look at these files and load the 2bit and BED files in IGB to double check that they're working properly.

            Location of the Quickload on Google Drive:
            Path: research-big-lorainelab > IGB Project Documentation and Plans > IGB Genomes > V_cardui.zip
            Link: https://drive.google.com/drive/folders/1bFRx4PqldxNf400n7Vr9SD_dNeNmtpvk?usp=drive_link

            Show
            pkulzer Paige Kulzer (Inactive) added a comment - - edited For review: I have zipped up a folder containing all of the relevant files needed to create this Quickload and place that in the shared Google Drive. Please take a look at these files and load the 2bit and BED files in IGB to double check that they're working properly. Location of the Quickload on Google Drive: Path : research-big-lorainelab > IGB Project Documentation and Plans > IGB Genomes > V_cardui.zip Link : https://drive.google.com/drive/folders/1bFRx4PqldxNf400n7Vr9SD_dNeNmtpvk?usp=drive_link
            Hide
            pkulzer Paige Kulzer (Inactive) added a comment -

            I've edited the header to make it specific to NCBI rather than UCSC, added species.txt and synonyms.txt to the zip file, cut down contents.txt to only include this genome, and edited annots.txt to replace UCSC's "refGene" with NCBI's "refSeq".

            Let me know if I've updated all of these files correctly!

            Show
            pkulzer Paige Kulzer (Inactive) added a comment - I've edited the header to make it specific to NCBI rather than UCSC, added species.txt and synonyms.txt to the zip file, cut down contents.txt to only include this genome, and edited annots.txt to replace UCSC's "refGene" with NCBI's "refSeq". Let me know if I've updated all of these files correctly!
            Hide
            nfreese Nowlan Freese added a comment -

            Testing: Everything looks good

            Show
            nfreese Nowlan Freese added a comment - Testing: Everything looks good
            Hide
            pkulzer Paige Kulzer (Inactive) added a comment -

            The Vanessa cardui genome has been pushed to the SVN repo.

            Ready for final review!

            Show
            pkulzer Paige Kulzer (Inactive) added a comment - The Vanessa cardui genome has been pushed to the SVN repo. Ready for final review!
            Hide
            ann.loraine Ann Loraine added a comment -

            I have deployed the latest copy of quickload repository to:

            RENCI hosting - http://igbquickload-main.bioviz.org/quickload/ (primary)
            UNC Charlotte hosting - http://igbquickload.org/quickload/ (backup)
            To test:

            • launch IGB and visit each new genome version (see above)
            • visit the subdirectories for each genome (by following the links above) and check that there is text describing the genome and datasets visible in IGB itself
            • within IGB Available Data section, click any "linkout" icons and make sure a Web page opens and that it goes to a place that describes the dataset somehow
            • check that when the datasets load, they look OK - gene models should be boxes with lines connecting them, for instance, and the track labels should be readable and should make sense ("making sense" is a subjective of course! mainly we're looking for problems that could trip up a user and cause confusion.)
            Show
            ann.loraine Ann Loraine added a comment - I have deployed the latest copy of quickload repository to: RENCI hosting - http://igbquickload-main.bioviz.org/quickload/ (primary) UNC Charlotte hosting - http://igbquickload.org/quickload/ (backup) To test: launch IGB and visit each new genome version (see above) visit the subdirectories for each genome (by following the links above) and check that there is text describing the genome and datasets visible in IGB itself within IGB Available Data section, click any "linkout" icons and make sure a Web page opens and that it goes to a place that describes the dataset somehow check that when the datasets load, they look OK - gene models should be boxes with lines connecting them, for instance, and the track labels should be readable and should make sense ("making sense" is a subjective of course! mainly we're looking for problems that could trip up a user and cause confusion.)
            Hide
            nfreese Nowlan Freese added a comment -

            Paige Kulzer - the 2bit file has a weird name and isn't loading correctly, can you update it to V_cardui_Feb_2021.2bit and push it to the SVN repo then ask Dr. Loraine to deploy it?

            Show
            nfreese Nowlan Freese added a comment - Paige Kulzer - the 2bit file has a weird name and isn't loading correctly, can you update it to V_cardui_Feb_2021.2bit and push it to the SVN repo then ask Dr. Loraine to deploy it?
            Hide
            pkulzer Paige Kulzer (Inactive) added a comment -

            Ann Loraine, as per Nowlan's comment above, I've updated the 2bit files name to "V_cardui_Feb_2021.2bit" and have pushed it to the SVN repo. It's ready for you to deploy!

            Show
            pkulzer Paige Kulzer (Inactive) added a comment - Ann Loraine , as per Nowlan's comment above, I've updated the 2bit files name to "V_cardui_Feb_2021.2bit" and have pushed it to the SVN repo. It's ready for you to deploy!
            Hide
            ann.loraine Ann Loraine added a comment -

            Update:

            • New file is deployed and old one is deleted - I updated quickloads hosted at RENCI and UNC Charlotte
            Show
            ann.loraine Ann Loraine added a comment - Update: New file is deployed and old one is deleted - I updated quickloads hosted at RENCI and UNC Charlotte
            Hide
            ann.loraine Ann Loraine added a comment - - edited

            Testing:

            It looks like the documentation describing the data files needs to be improved.

            The description of the files and where they came from in the comments above don't match the documentation. Specifically, the origin of the files is different. It looks like we got the files from the UCSC Genome Informations Web site and then used our tools to convert them to our preferred formats.

            Suggestion:

            Since we are doing a pretty good job of describing our process in the comments above (thanks Paige Kulzer) could we simply link to this ticket from our header file?

            We could provide some very generic – but accurate – descriptions of where the files came from and how we processed them. We could also then provide a direct link to this Jira ticket, which should be publicly accessible.

            I recommend we maybe make "Improve documentation for painted lady genome assembly quickload" as a new ticket.

            Once that documentation is more complete, we could then get in touch with researchers working with this assembly and get their input. Also, we should make sure to link to articles for this particular genome assembly so that users will be able to get more information.

            Show
            ann.loraine Ann Loraine added a comment - - edited Testing: It looks like the documentation describing the data files needs to be improved. The description of the files and where they came from in the comments above don't match the documentation. Specifically, the origin of the files is different. It looks like we got the files from the UCSC Genome Informations Web site and then used our tools to convert them to our preferred formats. Suggestion: Since we are doing a pretty good job of describing our process in the comments above (thanks Paige Kulzer ) could we simply link to this ticket from our header file? We could provide some very generic – but accurate – descriptions of where the files came from and how we processed them. We could also then provide a direct link to this Jira ticket, which should be publicly accessible. I recommend we maybe make "Improve documentation for painted lady genome assembly quickload" as a new ticket. Once that documentation is more complete, we could then get in touch with researchers working with this assembly and get their input. Also, we should make sure to link to articles for this particular genome assembly so that users will be able to get more information.
            Hide
            ann.loraine Ann Loraine added a comment -

            I made the recommend ticket for improving the genome assembly documentation. Closing this one now.

            Show
            ann.loraine Ann Loraine added a comment - I made the recommend ticket for improving the genome assembly documentation. Closing this one now.

              People

              • Assignee:
                pkulzer Paige Kulzer (Inactive)
                Reporter:
                pkulzer Paige Kulzer (Inactive)
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: