[IGBF-3264] Set up new Quickload infrastructure on UNCC hosting - JIRA UNCC

Details

Type: Task
Status: Closed (View Workflow)
Priority: Major
Resolution: Done
Affects Version/s: None
Fix Version/s: None
Labels:
None

Story Points:
5
Epic Link:
Deploy and maintain infrastructure
Sprint:
Spring 4 2023 Feb 21, Spring 9 2023 May 1, Summer 1 2023 May 15, Summer 2 2023 May 29, Summer 3 2023 June 12, Summer 4 2023 June 26, Summer 5 2023 July 10, Summer 6 2023 July 24, Summer 7 2023 Aug 7, Summer 8 2023 Aug 21

Description

Ann requested more space and fix to "igbquickload.org" resolution on-campus.

See attached email explaining details and work that was done on it by UNCC IT professional Michael Cowan:

Tasks:

migrate data to the new location and test access
create new git code repository for managing Integrated Genome Browser Quickload sites for the project
create Web page with Javascript button users can click to add the new quickload site to IGB

References:

Add drive to a VM w/o rebooting it: https://www.cyberciti.biz/tips/vmware-add-a-new-hard-disk-without-rebooting-guest.html
Find out if your host is a VM: https://kb.vmware.com/s/article/1005436
Add new disk to Linux (not necessarily a VM): https://www.tecmint.com/add-new-disk-to-an-existing-linux/
Info about Red Hat distributions: https://access.redhat.com/articles/3078#RHEL9

Attachments

Options
- Sort By Name
- Sort By Date
- Ascending
- Descending
- Thumbnails
- List
- Download All

Attachments

email.pdf
21/Feb/23 10:33 AM
175 kB
Ann Loraine

Issue Links

relates to

IGBF-3395 Update readme for quickload site

Closed

Activity

Ascending order - Click to sort in descending order

Hide

Permalink

Ann Loraine added a comment - 22/Feb/23 11:56 AM

Check the new disk with:

lsblk

Check linux release:

uname -a
Linux cci-vm12 2.6.32-431.23.3.el6.x86_64 #1 SMP Wed Jul 16 06:12:23 EDT 2014 x86_64 x86_64 x86_64 GNU/Linux

Check linux distribution:

cat /etc/system-release
Red Hat Enterprise Linux Server release 6.5 (Santiago)

Show

Ann Loraine added a comment - 22/Feb/23 11:56 AM Check the new disk with: lsblk Check linux release: uname -a Linux cci-vm12 2.6.32-431.23.3.el6.x86_64 #1 SMP Wed Jul 16 06:12:23 EDT 2014 x86_64 x86_64 x86_64 GNU/Linux Check linux distribution: cat /etc/system-release Red Hat Enterprise Linux Server release 6.5 (Santiago)

21 older comments

Hide

Permalink

Ann Loraine added a comment - 26/Jul/23 6:23 PM - edited

Something has changed. I now can log into the new VM with:

ssh aloraine@10.16.57.232

No password required as my key is already installed in my account's "authorized hosts" file.

Commencing to replicate igbquickload.org content on the host.

Required directories:

chipseq - DONE
dnaseq - DONE
quickload - DONE
rnaseq - in progress

Interesting. File transfer rate is much faster. Now it's >150MB/s versus 20MB/s yesterday.

Show

Ann Loraine added a comment - 26/Jul/23 6:23 PM - edited Something has changed. I now can log into the new VM with: ssh aloraine@10.16.57.232 No password required as my key is already installed in my account's "authorized hosts" file. Commencing to replicate igbquickload.org content on the host. Required directories: chipseq - DONE dnaseq - DONE quickload - DONE rnaseq - in progress Interesting. File transfer rate is much faster. Now it's >150MB/s versus 20MB/s yesterday.

Hide

Permalink

Ann Loraine added a comment - 26/Jul/23 10:05 PM

"rnaseq" is nearly finished. Decided to switch igbquickload.org to 152.15.236.217.

Show

Ann Loraine added a comment - 26/Jul/23 10:05 PM "rnaseq" is nearly finished. Decided to switch igbquickload.org to 152.15.236.217.

Hide

Permalink

Ann Loraine added a comment - 27/Jul/23 12:36 PM - edited

"rnaseq" is finished with the following report:

sent 45.73M bytes received 275.21G bytes 18.61M bytes/sec
total size is 572.68G speedup is 2.08

This is a little confusing because when I changed into the source directory on the RENCI host (/projects/igbquickload/lorainelab/www/main/htdocs/rnaseq) and executed "du -h", the total size of the current working directory (".") was reported as 655G. However, when I ran the same command in the target directory on the new VM (/mnt/igbdata/rnaseq), its size was reported as 534G. Why the different sizes?

Looking into it a bit more...

Show

Ann Loraine added a comment - 27/Jul/23 12:36 PM - edited "rnaseq" is finished with the following report: sent 45.73M bytes received 275.21G bytes 18.61M bytes/sec total size is 572.68G speedup is 2.08 This is a little confusing because when I changed into the source directory on the RENCI host (/projects/igbquickload/lorainelab/www/main/htdocs/rnaseq) and executed "du -h", the total size of the current working directory (".") was reported as 655G. However, when I ran the same command in the target directory on the new VM (/mnt/igbdata/rnaseq), its size was reported as 534G. Why the different sizes? Looking into it a bit more...

Hide

Permalink

Ann Loraine added a comment - 04/Sep/23 6:49 PM

Set up SSL.

Closing as this is now done.

Show

Ann Loraine added a comment - 04/Sep/23 6:49 PM Set up SSL. Closing as this is now done.

Hide

Permalink

Ann Loraine added a comment - 23/Feb/24 4:36 PM - edited

Copied blueberry data with:

data.bioviz.org root$ rsync -avzhP -e "ssh -J aloraine@hop.renci.org" aloraine@lorainelab-quickload.scidas.org:/projects/igbquickload/lorainelab/www/main/htdocs/blueberry/ blueberry

Copied soyseq data with:

rsync -avzhP -e "ssh -J aloraine@hop.renci.org" aloraine@lorainelab-quickload.scidas.org:/projects/igbquickload/lorainelab/www/main/htdocs/soyseq/ soyseq

Show

Ann Loraine added a comment - 23/Feb/24 4:36 PM - edited Copied blueberry data with: data.bioviz.org root$ rsync -avzhP -e "ssh -J aloraine@hop.renci.org" aloraine@lorainelab-quickload.scidas.org:/projects/igbquickload/lorainelab/www/main/htdocs/blueberry/ blueberry Copied soyseq data with: rsync -avzhP -e "ssh -J aloraine@hop.renci.org" aloraine@lorainelab-quickload.scidas.org:/projects/igbquickload/lorainelab/www/main/htdocs/soyseq/ soyseq

Set up new Quickload infrastructure on UNCC hosting

Details

Description

Attachments

Attachments

Issue Links

Activity

People

Dates