It's always a good idea to backup your files. This guide will help you transfer files from the DSMLP cluster to your desktop or git.
Create a repository on the git host of host of your choice, e.g. GitHub. Then commit and push your files.
Launch a Visual Studio Code pod and copy/paste the files from VS Code to your desktop.
Data may be copied to/from the cluster using the "SCP" or "SFTP" file transfer protocol from a Mac or Linux terminal window, or on Windows using a freely downloadable utility. We recommend this option for most users.
Example using the Mac/Linux 'sftp' command line program:
slithy:Downloads agt$ sftp <username>@dsmlp-login.ucsd.edu pod agt-4049 up and running; starting sftp Connected to ieng6.ucsd.edu sftp> put 2017-11-29-raspbian-stretch-lite.img Uploading 2017-11-29-raspbian-stretch-lite.img to /datasets/home/08/108/agt/2017-11-29-raspbian-stretch-lite.img 2017-11-29-raspbian-stretch-lite.img 100% 1772MB 76.6MB/s 00:23 sftp> quit sftp complete; deleting pod agt-4049 slithy:Downloads agt$
On Windows, we recommend the WinSCP utility.
'rsync' also may be used from a Mac or Linux terminal window to synchronize data sets:
slithy:ME198 agt$ rsync -avr tub_1_17-11-18 <username>@dsmlp-login.ucsd.edu pod agt-9924 up and running; starting rsync building file list ... done rsync complete; deleting pod agt-9924 sent 557671 bytes received 20 bytes 53113.43 bytes/sec size is 41144035 speedup is 73.78 slithy:ME198 agt$
As of summer 2022, all courses now have workspaces, which provide two options for file transfer.
The first option is to download the files from dsmlp-login as described above. However, the location of the files will be under /dsmlp/workspaces-fs01/COURSE/home/USERNAME, /dsmlp/workspaces-fs02/COURSE/home/USERNAME , or /dsmlp/workspaces-fs03/COURSE/home/USERNAME .
The second option is to use Visual Studio Code. The ProxyCommand line in ssh_config should include the -W (workspace) option with the ID of the workspace, i.e. the course id.
ProxyCommand ssh -i path/to/privatekey firstname.lastname@example.org /opt/launch-sh/bin/launch-datascience.sh -p normal -W COURSEID -H -N vscode-dsmlp
The files can be copy/pasted from VS Code to your desktop.
Sometimes you may need to download a dataset onto the cluster. First, consider its size and the number of users. If it is large and used by multiple people please send a request to email@example.com and we can put it into the /datasets folder so it can be shared.
If it's small you can use wget or curl to download it. First SSH to dsmlp-login and then invoke wget or curl.
For more information, check the FAQ.
Your instructor or TA will be your best resource for course-specific questions.