Transfer Files To or From a Cluster

Transferring files can be done using several methods. The most common are described below.

Globus

For larger transfers both within Yale and to external collaborators, we recommend using Globus. See our guide to using Globus here.

Transfer Files on OSX/Linux

Unix, Linux and Apple OS X users can use rsync to transfer files to/from a cluster. You will need the hostname of the cluster login nodes to transfer files.

Note that you must have your ssh keys properly configured to use the commands outlined below. See the SSH Guide for more info.

scp and sftp are both used from a Terminal window. The basic syntax of scp is

scp [from] [to]

The “from” portion can be a filename or a directory/folder. The “to” portion will contain your netid, the hostname of the cluster login node, and the destination directory/folder.

Transfer a file from your computer to a cluster

Assuming the user’s netid is “ra359″:

scp myfile.txt ra359@grace.hpc.yale.edu:/home/fas/admins/ra359/test

In this example, “myfile.txt’ is copied to the directory “/home/fas/admins/ra359/test: on Grace. This example assumes that “myfile.txt” is in your current directory. You may also specify the full path of “myfile.txt”:

scp /home/xyz/myfile.txt ra359@grace.hpc.yale.edu:/home/fas/admins/ra359/test

Transfer a directory and its contents to a cluster

scp -r -p mydirectory ra359@grace.hpc.yale.edu:/home/fas/admins/ra359/test

In this example, the contents of “mydirectory” are transferred. The “-r” indicates that the copy is recursive. The “-p” preserves dates/times/permissions of the files.

Transfer files from the cluster to your computer

Assuming you would like the files copied to your current directory:

scp ra359@grace.hpc.yale.edu:/home/fas/admins/ra359/myfile.txt .

Note that “.” represents your current working directory.
To specify the destination, simply replace the “.” with the full path:

scp ra359@grace.hpc.yale.edu:/home/fas/admins/ra359/myfile.txt /path/myfolder

Via FTP Client

You can also transfer files between your local computer and a cluster using an FTP client, such as Cyberduck (OSX) or FileZilla (Linux). You will need to configure the client with your netid as the username, the cluster login node as the hostname and your private key as the authentication method. An example configuration of Cyberduck is shown below.

If you are connecting to either Farnam or Ruddle, which require multi-factor authentication, there are a couple additional steps.

You will need to change the Cyberduck preferences to "Use browser connection" instead of "open multiple connections". This can be found under Cyberduck > Preferences > Transfers > General.

Then once you establish your connection, you will prompted with a "Partial authentication success" window. In the password field, type in "push" to receive a push approval notification to the Duo app on your phone. Alternate multi-factor authentications can be used by enter the following words in the password field:

  • "push" to receive a push notification to your smart phone (requires the Duo mobile app)
  • "sms" to receive a verification passcode via text message
  • "phone" to receive a phone call

Transfer Files from Windows

On Microsoft Windows, we recommend using WinSCP to securely transfer files to/from a cluster. Download and execute the WinSCP installer.

Accept the license and the defaults, including the following:

At the end of the installation you may see the following prompt:

Select "OK" and then "OK" again to import the sessions you have defined in PuTTY as shown below

At this point you can launch WinSCP. You should see something like the following:

If you double-click a host on the left you should be prompted for your ssh passphrase and then logged into the cluster. You will see the following screen the first time you log in:

Select Yes to accept the key.

A screen will appear showing your local desktop on the left and the cluster you are logged into on the right:

Simply drag files from on side of the screen to the other to transfer data.

If you need to set-up a new connection to another cluster, start WinSCP and select New Site on the top left. Fill-in the Host name and User name as well:

Select Advanced and then SSH -> Auth. Use the Browse button to select your private key file (netid.ppk):

Select Ok and then Save to save the session.