Uploaded image for project: 'IGB'
  1. IGB
  2. IGBF-3264

Set up new Quickload infrastructure on UNCC hosting

    Details

    • Type: Task
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None
    • Story Points:
      5
    • Sprint:
      Spring 4 2023 Feb 21, Spring 9 2023 May 1, Summer 1 2023 May 15, Summer 2 2023 May 29, Summer 3 2023 June 12, Summer 4 2023 June 26, Summer 5 2023 July 10, Summer 6 2023 July 24, Summer 7 2023 Aug 7, Summer 8 2023 Aug 21

      Description

      Ann requested more space and fix to "igbquickload.org" resolution on-campus.

      See attached email explaining details and work that was done on it by UNCC IT professional Michael Cowan:

      Tasks:

      • migrate data to the new location and test access
      • create new git code repository for managing Integrated Genome Browser Quickload sites for the project
      • create Web page with Javascript button users can click to add the new quickload site to IGB

      References:

        Attachments

          Issue Links

            Activity

            Hide
            ann.loraine Ann Loraine added a comment - - edited

            Something has changed. I now can log into the new VM with:

            • ssh aloraine@10.16.57.232

            No password required as my key is already installed in my account's "authorized hosts" file.

            Commencing to replicate igbquickload.org content on the host.

            Required directories:

            • chipseq - DONE
            • dnaseq - DONE
            • quickload - DONE
            • rnaseq - in progress

            Interesting. File transfer rate is much faster. Now it's >150MB/s versus 20MB/s yesterday.

            Show
            ann.loraine Ann Loraine added a comment - - edited Something has changed. I now can log into the new VM with: ssh aloraine@10.16.57.232 No password required as my key is already installed in my account's "authorized hosts" file. Commencing to replicate igbquickload.org content on the host. Required directories: chipseq - DONE dnaseq - DONE quickload - DONE rnaseq - in progress Interesting. File transfer rate is much faster. Now it's >150MB/s versus 20MB/s yesterday.
            Hide
            ann.loraine Ann Loraine added a comment -

            "rnaseq" is nearly finished. Decided to switch igbquickload.org to 152.15.236.217.

            Show
            ann.loraine Ann Loraine added a comment - "rnaseq" is nearly finished. Decided to switch igbquickload.org to 152.15.236.217.
            Hide
            ann.loraine Ann Loraine added a comment - - edited

            "rnaseq" is finished with the following report:

            sent 45.73M bytes received 275.21G bytes 18.61M bytes/sec
            total size is 572.68G speedup is 2.08

            This is a little confusing because when I changed into the source directory on the RENCI host (/projects/igbquickload/lorainelab/www/main/htdocs/rnaseq) and executed "du -h", the total size of the current working directory (".") was reported as 655G. However, when I ran the same command in the target directory on the new VM (/mnt/igbdata/rnaseq), its size was reported as 534G. Why the different sizes?

            Looking into it a bit more...

            Show
            ann.loraine Ann Loraine added a comment - - edited "rnaseq" is finished with the following report: sent 45.73M bytes received 275.21G bytes 18.61M bytes/sec total size is 572.68G speedup is 2.08 This is a little confusing because when I changed into the source directory on the RENCI host (/projects/igbquickload/lorainelab/www/main/htdocs/rnaseq) and executed "du -h", the total size of the current working directory (".") was reported as 655G. However, when I ran the same command in the target directory on the new VM (/mnt/igbdata/rnaseq), its size was reported as 534G. Why the different sizes? Looking into it a bit more...
            Hide
            ann.loraine Ann Loraine added a comment -

            Set up SSL.

            Closing as this is now done.

            Show
            ann.loraine Ann Loraine added a comment - Set up SSL. Closing as this is now done.
            Hide
            ann.loraine Ann Loraine added a comment - - edited

            Copied blueberry data with:

            data.bioviz.org root$ rsync -avzhP -e "ssh -J aloraine@hop.renci.org" aloraine@lorainelab-quickload.scidas.org:/projects/igbquickload/lorainelab/www/main/htdocs/blueberry/ blueberry 
            

            Copied soyseq data with:

            rsync -avzhP -e "ssh -J aloraine@hop.renci.org" aloraine@lorainelab-quickload.scidas.org:/projects/igbquickload/lorainelab/www/main/htdocs/soyseq/ soyseq
            
            Show
            ann.loraine Ann Loraine added a comment - - edited Copied blueberry data with: data.bioviz.org root$ rsync -avzhP -e "ssh -J aloraine@hop.renci.org" aloraine@lorainelab-quickload.scidas.org:/projects/igbquickload/lorainelab/www/main/htdocs/blueberry/ blueberry Copied soyseq data with: rsync -avzhP -e "ssh -J aloraine@hop.renci.org" aloraine@lorainelab-quickload.scidas.org:/projects/igbquickload/lorainelab/www/main/htdocs/soyseq/ soyseq

              People

              • Assignee:
                ann.loraine Ann Loraine
                Reporter:
                ann.loraine Ann Loraine
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: