Globus

Globus is a file transfer service that is efficient and easy to use. It has several advantages:

  • Globus can transfer large files, and large collections of files
  • Files can be transferred between your computer and the clusters
  • Files can be transferred between Yale and other sites
  • Transfer is secure and robust
  • There is a simple web interface for starting and monitoring transfers, as well as a command line interface.
  • Transfers are parallelized for increased speed
  • Credentials are cached, so fewer logins are required
  • Transfer errors are caught and transfers retried See www.globus.org for more information.
  • You can provide access to specific files or directories to non-Yale people in a secure way, using their globus id.

We have set up Globus endpoints on most of the Yale clusters. Globus uses gridftp to perform transfers in parallel. Globus works a bit differently than other transfer services such as ftp or rsync. With Globus, files are always transferred between two "endpoints". One endpoint is always a Globus server, such as the ones we've set up on the clusters. The other endpoint can be a second server, or a Globus connect personal endpoint, which is a desktop application.

Transfers are managed by a master server at globusonline, which manages credentials, sets up, and monitors transfers. However, data streams directly between the two endpoints.

​To get started with Globus

  1. In a browser, go to www.globus.org.
  2. Click on "Login".
  3. Use the pulldown to select Yale in the list of organizations and click "Continue".
  4. If you are not already logged into CAS, you will be asked for netid and password.
  5. You'll see a transfer panel with dual panes. Enter an endpoint name in the left endpoint box, e.g. yale#transfer-louise. A dialog box will open to help complete the name.
  6. The file browser will show you the directories in the root directory that globus is exporting, normally /
  7. Browse to any directory you can normally access, e.g. your home directory.
  8. Enter an endpoint name in the right endpoint box, and browse to a different directory.
  9. Select one or more files in either the left or right box, and click the < or > button to transfer the files in that direction.

For more information, see the globus documentation.

Cluster Endpoints

We currently support endpoints for all the clusters

  • Ruddle: yale#ruddle
  • Farnam: yale#farnam
  • Omega: yale#transfer-omega
  • Grace: yale#grace
  • HEP: yale#transfer-hep

All of these endpoints provide access to all files you normally have access to, except on Ruddle. There, you have access to your home, project, and scratch60 directories.

Setting up an endpoint on your own computer

You can set up your own endpoint for transferring data to and from your own computer: linux, windows, or mac. This is called Globus Connect, and you can find info here.

Access by non-Yale people

Among Globus' greatest features is the ability to allow non-Yale people access to specific files or directories. This is done by creating a "share" and setting the permissions on that share. Here are the steps:

  1. Create a directory somewhere on a cluster in one of your directories, e.g. $ mkdir /fastscratch/rdb9/ToBob
  2. In the globus webpage, click on "endpoints" and select the endpoint for the cluster hosting that directory.
  3. Click on the myshares tab
  4. Click on "Add Shared Endpoint"
  5. Fill in the form, using the browse button to select the directory you want to share. Give it a name and description.
  6. On the next form, you'll provide the identity of the user or group you want to give permission to, and the permission set. You can use the search to help find the person. Then click "Add Permission"
  7. Click on "View link for sharing". You'll see a url that you can give to the user, which will allow them access to that directory.