Difference between revisions of "Globus"

From Carl R. Woese Institute for Genomic Biology - University of Illinois Urbana-Champaign
Jump to navigation Jump to search
(Create Account)
(Collections)
 
(33 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
__TOC__
 
__TOC__
  
Globus is service that allows the transferring of very large files with auto-resume function and transfer verification.
+
*Globus is service that allows the transferring of very large files (100s of GBs and TBs of data) with auto-resume function and transfer verification.
 +
* Connecting to Globus is a two step process.  First you login into the Globus service itself.  Then you can login to a specific collections. 
 +
* Most people will have a Globus account already as you can login with your University of Illinois Credentials.
  
== IGB Managed Endpoints ==
+
= IGB Managed Collections =
 +
* Below is a list of IGB Mananged Globus Collections
 
{| class="wikitable" style="width: 800px" border="1" cellpadding="1" cellspacing="1"
 
{| class="wikitable" style="width: 800px" border="1" cellpadding="1" cellspacing="1"
 
|-
 
|-
! scope="col" | Endpoint Name<br/>
+
! scope="col" | Collection Name<br/>
 
! scope="col" | Description<br/>
 
! scope="col" | Description<br/>
 
|-
 
|-
| biotech#ftp.biotech.illinois.edu<br/>
+
| biocluster.igb.illinois.edu<br/>
| Roy J Carver Biotech Center FTP Server<br/>
 
|-
 
| igb#biocluster.igb.illinois.edu<br/>
 
 
| IGB Biocluster - [http://biocluster.igb.illinois.edu http://biocluster.igb.illinois.edu]<br/>
 
| IGB Biocluster - [http://biocluster.igb.illinois.edu http://biocluster.igb.illinois.edu]<br/>
 
|-
 
|-
| igb#compgen.igb.illinois.edu<br/>
 
| Compgen Cluster - [http://compgen.igb.illinois.edu http://compgen.igb.illinois.edu]<br/>
 
|-
 
| igb#virtualmatt.igb.illinois.edu<br/>
 
| virtualmatt.igb.illinois.edu Server<br/>
 
 
|}
 
|}
  
== Create Account ==
+
= Login to Globus =
===Within an Organization===
+
== Within an Existing Organization ==
 
*Open a web browser (Firefox, Chrome, etc.)
 
*Open a web browser (Firefox, Chrome, etc.)
 
*In the address bar, please go to [https://www.globus.org https://www.globus.org]
 
*In the address bar, please go to [https://www.globus.org https://www.globus.org]
 
*Top right corner of the website should have a link button, '''Log In'''. Please click here.
 
*Top right corner of the website should have a link button, '''Log In'''. Please click here.
[[File:Globus_home.jpg|800px|Globus_home.jpg]]
+
[[File:Globus_home.jpg|450px]]
*Select the organization you belong to.  Click '''Continue'''.
+
*Select the organization you belong to.  Click '''Continue'''.  For most people, this will be '''University of Illinois at Urbana-Champaign'''.
[[File:Globus_organization.JPG|800px|Globus_organization.JPG]]
+
[[File:Globus_organization.JPG|450px]]
*If you selected '''University of Illinois at Urbana-Champaign''', then proceed to login with your netID and your Active Directory Password.
+
*If you selected '''University of Illinois at Urbana-Champaign''', then proceed to login with your netID, campus password, and 2 Factor Authentication.
[[File:Globus_ad.jpg|800px|Globus_ad.jpg]]
+
[[File:Globus_ad.jpg|450px]]
 
*If you previously had a Globus Account, then select '''Link to an existing account'''.  If not select '''No thanks, continue'''.
 
*If you previously had a Globus Account, then select '''Link to an existing account'''.  If not select '''No thanks, continue'''.
[[File:Globus_link.jpg|800px|Globus_link.jpg]]
+
[[File:Globus_link.jpg|450px]]
 
*Follow the next couple of screens to confirm your information
 
*Follow the next couple of screens to confirm your information
[[File:Globus_confirm.png|800px|Globus_confirm.png]]
+
[[File:Globus_confirm.png|450px]]
  
===Generic Globus Account===
+
==Generic Globus Account==
 
*If your organization is not listed, you can signup with a Globus ID instead.
 
*If your organization is not listed, you can signup with a Globus ID instead.
 
*Go to [https://www.globusid.org https://www.globusid.org] and select '''Create a Globus ID'''.
 
*Go to [https://www.globusid.org https://www.globusid.org] and select '''Create a Globus ID'''.
[[File:Globus_create_1.jpg|800px|Globus_create_1.jpg]]
+
[[File:Globus_create_1.jpg|450px|Globus_create_1.jpg]]
 
*Enter the necessary information in its respective field. This account is not affiliated with the University of Illinois or IGB, so the username will be specific for Globus only.
 
*Enter the necessary information in its respective field. This account is not affiliated with the University of Illinois or IGB, so the username will be specific for Globus only.
[[File:Globus2.png|800px|Globus2.png]]
+
[[File:Globus2.png|450px|Globus2.png]]
 
*Click '''Create ID''' and assuming your information is verified and accepted, Globus will send a verification e-mail to the e-mail you used in signing up for the account. Please check your e-mail for the Globus Verification e-mail and verify your e-mail address.
 
*Click '''Create ID''' and assuming your information is verified and accepted, Globus will send a verification e-mail to the e-mail you used in signing up for the account. Please check your e-mail for the Globus Verification e-mail and verify your e-mail address.
 
*You've successfully created your Globus Account.
 
*You've successfully created your Globus Account.
  
== Create Personal Endpoint ==
+
= Collections =
 
+
* Now you need to login to a globus collection. We will be using '''biocluster.igb.illinois.edu''' as an example.
*Now that your account is created, you can download the end-user client application. This application will be necessary to download files from the server to your local computer.
+
* Search for your collection in the '''Collection''' Search Box. For '''biocluster.igb.illinois.edu''', it should find it like below.
 
+
[[File:Globus collection.jpg|700px]]
*Please log in to the Globus Homepage, [https://www.globus.org https://www.globus.org].
+
* Select '''Continue''' to start the process to login to the biocluster with your IGB credentials
*Please click on the button, "Sign In".  Proceed to login with your organization's login or your Globus ID.
+
[[File:Globus-igb-1.jpg|700px]]
 
+
* Select '''Link an identity from Globus IGB (globus.igb.illinois.edu)'''
[[File:Globus_home.jpg|800px|Globus_home.jpg]]
+
[[File:Globus-igb-2.jpg|500px]]
 
+
* Login with your '''netID''' and '''IGB''' password.
*Near the top right of the window should consist of four link buttons; please click on the "Manage Endpoints" button.
+
[[File:Globus-igb-3.jpg|500px]]
 
+
* Click '''Allow''' to link your Biocluster account with your Globus account
[[File:Globus4.png|800px|Globus4.png]]
+
[[File:Globus-igb-4.jpg|500px]]
 
+
* You should now be able to browse your files on the biocluster
*Please click on "+ add Globus Connect Personal".
 
 
 
[[File:Globus5.png|800px|Globus5.png]]
 
 
 
*In the page seen below, please give your endpoint a name that you can recognize. Refer to the example below, the endpoint name will appear as the following, "johndoe1#homepc". Click "Generate Setup Key" after you've typed in the endpoint name.
 
*PLEASE MAKE A COPY OF THIS GENERATED KEY AS YOU'LL NEED THIS KEY TO INSTALL THE CLIENT APPLICATION. THE SETUP KEY SHOULD BE IN THE FORM OF "xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx".
 
 
 
[[File:Globus6.png|800px|Globus6.png]]
 
 
 
*When the program has finished downloading, please run the install file. Towards the end of the installation process, the client will ask for the Generated Setup Key. Please copy and paste that key into the install window. Once the server has detected the endpoint, Globus will now be able to send files directly to your computer.
 
  
== Transferring Files ==
+
= Create Personal Endpoint =
*Globus can be used to transfer files from your personal computer to another server.
+
* A Personal Endpoint will allow you to transfer files from your computer to another globus endpoint.
*Globus also can be used to transfer files between different server endpoints such as the Biotech FTP Server (biotech#ftp.biotech.illinois.edu) to the IGB Biocluster (igb#biocluster.igb.illinois.edu).
+
* Select '''File Manager'''->'''Your Collections'''. Then select '''Get Globus Connect Personal'''
*To start using Globus for file transfers, please click on “Manage Data”, then “Transfer Files”.
+
[[File:Globus-personal-1.jpg|500px]]
*For the first endpoint,
+
* Select '''Download Globus Connect Personal for Windows'''.  You can select other operating systems alsoFollow the installation prompts to install the software.
**Enter the endpoint name (ie '''biotech#ftp.biotech.illinois.edu''').  
+
[[File:Globus-personal-2.jpg|500px]]
**Then type in your username and password.
+
* After Installation, a web browser will open to provide a label name for your personal collection and to '''Allow''' the collection permission.
**Select the advance tab and enter '''168''' for '''Credential Lifetime'''.  This defaults to 1 day (24 hours)  This will save your username and password up to 7 days.  Thus no transfer can exceed 7 daysThis is the maximum value that can be set.
+
[[File:Globus-personal-3.jpg|500px]]
*For the second endpoint
+
* In the final stage of installing the '''Globus Connect Personal''', re-input your personal collection name and a description.  Then select '''Save'''
**Please type in the second endpoint such as (ie '''igb#biocluster.biotech.illinois.edu''') or your personal endpoint.  
+
[[File:Globus-personal-4.jpg|500px]]
**Then type in your username and password to authenticate.
 
**Select the advance tab and enter '''168''' for '''Credential Lifetime'''.
 
[[File:Globus_transfer.jpg|800px|Globus_transfer.jpg]]
 
*Please select the file from endpoint 1 and select the destination in endpoint 2. Select the arrow that reflects the direction of transfer (left to right, or right to left).
 
*A transfer request will be submitted and issued a Task ID if successful.
 
*Please check the designated destination folder and verify that the requested download successfully downloaded.
 
*Congratulations, you now have Globus Access and can transfer files.
 
  
== References ==
+
= Transferring Files =
 +
*Globus can be used to transfer files from one collection to another collection.
 +
* Select the split panel in the upper right corner to show both collections you are trying to transfer from and to.
 +
[[File:Globus transfer.jpg|700px]]
 +
* For the first collection, enter the collection name.  An example would be '''biocluster.igb.illinois.edu'''
 +
* For the second collection, enter another collection name.  This could be your personal collection or another HPC with Globus installed
 +
* Select Files you want to transfer from one collection to another.
  
 +
= References =
 
*[https://www.globus.org/ https://www.globus.org/]
 
*[https://www.globus.org/ https://www.globus.org/]
*[https://support.globus.org/entries/23602336-Sharing-Data-using-Globus https://support.globus.org/entries/23602336-Sharing-Data-using-Globus]
+
*[https://docs.globus.org/how-to/get-started/ https://docs.globus.org/how-to/get-started/]

Latest revision as of 19:55, 21 August 2023

  • Globus is service that allows the transferring of very large files (100s of GBs and TBs of data) with auto-resume function and transfer verification.
  • Connecting to Globus is a two step process. First you login into the Globus service itself. Then you can login to a specific collections.
  • Most people will have a Globus account already as you can login with your University of Illinois Credentials.

IGB Managed Collections[edit]

  • Below is a list of IGB Mananged Globus Collections
Collection Name
Description
biocluster.igb.illinois.edu
IGB Biocluster - http://biocluster.igb.illinois.edu

Login to Globus[edit]

Within an Existing Organization[edit]

  • Open a web browser (Firefox, Chrome, etc.)
  • In the address bar, please go to https://www.globus.org
  • Top right corner of the website should have a link button, Log In. Please click here.

Globus home.jpg

  • Select the organization you belong to. Click Continue. For most people, this will be University of Illinois at Urbana-Champaign.

Globus organization.JPG

  • If you selected University of Illinois at Urbana-Champaign, then proceed to login with your netID, campus password, and 2 Factor Authentication.

Globus ad.jpg

  • If you previously had a Globus Account, then select Link to an existing account. If not select No thanks, continue.

Globus link.jpg

  • Follow the next couple of screens to confirm your information

Globus confirm.png

Generic Globus Account[edit]

  • If your organization is not listed, you can signup with a Globus ID instead.
  • Go to https://www.globusid.org and select Create a Globus ID.

Globus_create_1.jpg

  • Enter the necessary information in its respective field. This account is not affiliated with the University of Illinois or IGB, so the username will be specific for Globus only.

Globus2.png

  • Click Create ID and assuming your information is verified and accepted, Globus will send a verification e-mail to the e-mail you used in signing up for the account. Please check your e-mail for the Globus Verification e-mail and verify your e-mail address.
  • You've successfully created your Globus Account.

Collections[edit]

  • Now you need to login to a globus collection. We will be using biocluster.igb.illinois.edu as an example.
  • Search for your collection in the Collection Search Box. For biocluster.igb.illinois.edu, it should find it like below.

Globus collection.jpg

  • Select Continue to start the process to login to the biocluster with your IGB credentials

Globus-igb-1.jpg

  • Select Link an identity from Globus IGB (globus.igb.illinois.edu)

Globus-igb-2.jpg

  • Login with your netID and IGB password.

Globus-igb-3.jpg

  • Click Allow to link your Biocluster account with your Globus account

Globus-igb-4.jpg

  • You should now be able to browse your files on the biocluster

Create Personal Endpoint[edit]

  • A Personal Endpoint will allow you to transfer files from your computer to another globus endpoint.
  • Select File Manager->Your Collections. Then select Get Globus Connect Personal

Globus-personal-1.jpg

  • Select Download Globus Connect Personal for Windows. You can select other operating systems also. Follow the installation prompts to install the software.

Globus-personal-2.jpg

  • After Installation, a web browser will open to provide a label name for your personal collection and to Allow the collection permission.

Globus-personal-3.jpg

  • In the final stage of installing the Globus Connect Personal, re-input your personal collection name and a description. Then select Save

Globus-personal-4.jpg

Transferring Files[edit]

  • Globus can be used to transfer files from one collection to another collection.
  • Select the split panel in the upper right corner to show both collections you are trying to transfer from and to.

Globus transfer.jpg

  • For the first collection, enter the collection name. An example would be biocluster.igb.illinois.edu
  • For the second collection, enter another collection name. This could be your personal collection or another HPC with Globus installed
  • Select Files you want to transfer from one collection to another.

References[edit]