17. How do I use a remote server more effectively?#

Important

Since I’ve been sick, this is not fully filled in yet. Use a previous semester’s equivalent sessions as a reference for more detail for now. Create a PR that inserts an explanation of a few steps for a community badge.

Previous notes:

ls
example
github-in-class-brownsarahm-1
kwl
seawulf
test
testobj.md
tiny-book
ssh -l brownsarahm seawulf.uri.edu
brownsarahm@seawulf.uri.edu's password: 
Last failed login: Thu Mar 30 12:53:35 EDT 2023 from pool-96-238-44-82.prvdri.fios.verizon.net on ssh:notty
There was 1 failed login attempt since the last successful login.
Last login: Thu Mar 30 11:54:40 2023 from pool-72-87-118-171.prvdri.fios.verizon.net
[brownsarahm@seawulf ~]$ pwd
/home/brownsarahm
[brownsarahm@seawulf ~]$ ls
bash-lesson.tar.gz                           SRR307024_2.fastq
bash-lesson.tar.gz.1                         SRR307025_1.fastq
demo.sh                                      SRR307025_2.fastq
dmel-all-r6.19.gtf                           SRR307026_1.fastq
dmel_unique_protein_isoforms_fb_2016_01.tsv  SRR307026_2.fastq
gene_association.fb                          SRR307027_1.fastq
linecounts.txt                               SRR307027_2.fastq
my_job.sh                                    SRR307028_1.fastq
results                                      SRR307028_2.fastq
slurm-23950.out                              SRR307029_1.fastq
SRR307023_1.fastq                            SRR307029_2.fastq
SRR307023_2.fastq                            SRR307030_1.fastq
SRR307024_1.fastq                            SRR307030_2.fastq
[brownsarahm@seawulf ~]$ mkdir example
[brownsarahm@seawulf ~]$ cd example/
[brownsarahm@seawulf example]$ pwd
/home/brownsarahm/example
[brownsarahm@seawulf example]$ mkdir ex2
[brownsarahm@seawulf example]$ cd ex2/
[brownsarahm@seawulf ex2]$ pwd
/home/brownsarahm/example/ex2
[brownsarahm@seawulf ex2]$ cd 
[brownsarahm@seawulf ~]$ ls -l
total 150704
-rw-r--r--. 1 brownsarahm spring2022-csc392 12534006 Apr 18  2021 bash-lesson.tar.gz
-rw-r--r--. 1 brownsarahm spring2022-csc392 12534006 Apr 18  2021 bash-lesson.tar.gz.1
-rwxr-xr-x. 1 brownsarahm spring2022-csc392       20 Oct 26 17:11 demo.sh
-rw-r--r--. 1 brownsarahm spring2022-csc392 77426528 Jan 16  2018 dmel-all-r6.19.gtf
-rw-r--r--. 1 brownsarahm spring2022-csc392   721242 Jan 25  2016 dmel_unique_protein_isoforms_fb_2016_01.tsv
drwxr-xr-x. 3 brownsarahm spring2022-csc392       24 Mar 30 12:59 example
-rw-r--r--. 1 brownsarahm spring2022-csc392 25056938 Jan 25  2016 gene_association.fb
-rw-r--r--. 1 brownsarahm spring2022-csc392      447 Mar  8  2022 linecounts.txt
-rw-r--r--. 1 brownsarahm spring2022-csc392       84 Mar  8  2022 my_job.sh
drwxr-xr-x. 2 brownsarahm spring2022-csc392       10 Mar  8  2022 results
-rw-r--r--. 1 brownsarahm spring2022-csc392       89 Mar  8  2022 slurm-23950.out
-rw-r--r--. 1 brownsarahm spring2022-csc392  1625262 Jan 25  2016 SRR307023_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392  1625262 Jan 25  2016 SRR307023_2.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392  1625376 Jan 25  2016 SRR307024_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392  1625376 Jan 25  2016 SRR307024_2.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392  1625286 Jan 25  2016 SRR307025_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392  1625286 Jan 25  2016 SRR307025_2.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392  1625302 Jan 25  2016 SRR307026_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392  1625302 Jan 25  2016 SRR307026_2.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392  1625312 Jan 25  2016 SRR307027_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392  1625312 Jan 25  2016 SRR307027_2.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392  1625338 Jan 25  2016 SRR307028_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392  1625338 Jan 25  2016 SRR307028_2.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392  1625390 Jan 25  2016 SRR307029_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392  1625390 Jan 25  2016 SRR307029_2.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392  1625318 Jan 25  2016 SRR307030_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392  1625318 Jan 25  2016 SRR307030_2.fastq
[brownsarahm@seawulf ~]$ ls -l --block-size=M
total 148M
-rw-r--r--. 1 brownsarahm spring2022-csc392 12M Apr 18  2021 bash-lesson.tar.gz
-rw-r--r--. 1 brownsarahm spring2022-csc392 12M Apr 18  2021 bash-lesson.tar.gz.1
-rwxr-xr-x. 1 brownsarahm spring2022-csc392  1M Oct 26 17:11 demo.sh
-rw-r--r--. 1 brownsarahm spring2022-csc392 74M Jan 16  2018 dmel-all-r6.19.gtf
-rw-r--r--. 1 brownsarahm spring2022-csc392  1M Jan 25  2016 dmel_unique_protein_isoforms_fb_2016_01.tsv
drwxr-xr-x. 3 brownsarahm spring2022-csc392  1M Mar 30 12:59 example
-rw-r--r--. 1 brownsarahm spring2022-csc392 24M Jan 25  2016 gene_association.fb
-rw-r--r--. 1 brownsarahm spring2022-csc392  1M Mar  8  2022 linecounts.txt
-rw-r--r--. 1 brownsarahm spring2022-csc392  1M Mar  8  2022 my_job.sh
drwxr-xr-x. 2 brownsarahm spring2022-csc392  1M Mar  8  2022 results
-rw-r--r--. 1 brownsarahm spring2022-csc392  1M Mar  8  2022 slurm-23950.out
-rw-r--r--. 1 brownsarahm spring2022-csc392  2M Jan 25  2016 SRR307023_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392  2M Jan 25  2016 SRR307023_2.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392  2M Jan 25  2016 SRR307024_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392  2M Jan 25  2016 SRR307024_2.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392  2M Jan 25  2016 SRR307025_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392  2M Jan 25  2016 SRR307025_2.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392  2M Jan 25  2016 SRR307026_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392  2M Jan 25  2016 SRR307026_2.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392  2M Jan 25  2016 SRR307027_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392  2M Jan 25  2016 SRR307027_2.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392  2M Jan 25  2016 SRR307028_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392  2M Jan 25  2016 SRR307028_2.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392  2M Jan 25  2016 SRR307029_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392  2M Jan 25  2016 SRR307029_2.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392  2M Jan 25  2016 SRR307030_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392  2M Jan 25  2016 SRR307030_2.fastq
[brownsarahm@seawulf ~]$ exit
logout
Connection to seawulf.uri.edu closed.

17.1. Creating SSH Keys#

ssh-keygen -f ~/seawulf -t rsa -b 1024
Generating public/private rsa key pair.
/Users/brownsarahm/seawulf already exists.
Overwrite (y/n)? y
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /Users/brownsarahm/seawulf.
Your public key has been saved in /Users/brownsarahm/seawulf.pub.
The key fingerprint is:
SHA256:NK9gJGqWW4E2B5AI6Alb7+34JGYQ2QS7BpCM081WQfU brownsarahm@Sarahs-MBP
The key's randomart image is:
+---[RSA 1024]----+
|X=o+.o+o.        |
|X.oB+    .       |
|++Oo= . o E      |
|.= B.+ . o       |
|  O...o S .      |
| + +.... .       |
|  . +o. .        |
|   o.o.          |
|     ..          |
+----[SHA256]-----+
ssh-copy-id -i ~/seawulf brownsarahm@seawulf.uri.edu
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/Users/brownsarahm/seawulf.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
brownsarahm@seawulf.uri.edu's password: 

Number of key(s) added:        1

Now try logging into the machine, with:   "ssh 'brownsarahm@seawulf.uri.edu'"
and check to make sure that only the key(s) you wanted were added.

17.2. Logging in#

ssh brownsarahm@seawulf.uri.edu
brownsarahm@seawulf.uri.edu's password: 
Last login: Thu Mar 30 13:18:24 2023 from pool-72-87-118-171.prvdri.fios.verizon.net
[brownsarahm@seawulf ~]$ pwd
/home/brownsarahm
[brownsarahm@seawulf ~]$ lshw
WARNING: you should run this program as super-user.
seawulf.uri.edu             
    description: Computer
    width: 64 bits
    capabilities: smp vsyscall32
  *-core
       description: Motherboard
       physical id: 0
     *-memory
          description: System memory
          physical id: 0
          size: 32GiB
     *-cpu:0
          product: Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz
          vendor: Intel Corp.
          vendor_id: GenuineIntel
          physical id: 1
          bus info: cpu@0
          version: 6.79.1
          size: 1200MHz
          capacity: 2900MHz
          width: 64 bits
          capabilities: fpu fpu_exception wp vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp x86-64 constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch epb cat_l3 cdp_l3 invpcid_single intel_ppin intel_pt ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts spec_ctrl intel_stibp cpufreq

Truncated output

[brownsarahm@seawulf ~]$ interactive
salloc: Granted job allocation 26471
salloc: Waiting for resource configuration
salloc: Nodes n005 are ready for job
[brownsarahm@n005 ~]$ ls
bash-lesson.tar.gz                           SRR307024_2.fastq
bash-lesson.tar.gz.1                         SRR307025_1.fastq
demo.sh                                      SRR307025_2.fastq
dmel-all-r6.19.gtf                           SRR307026_1.fastq
dmel_unique_protein_isoforms_fb_2016_01.tsv  SRR307026_2.fastq
example                                      SRR307027_1.fastq
gene_association.fb                          SRR307027_2.fastq
linecounts.txt                               SRR307028_1.fastq
my_job.sh                                    SRR307028_2.fastq
results                                      SRR307029_1.fastq
slurm-23950.out                              SRR307029_2.fastq
SRR307023_1.fastq                            SRR307030_1.fastq
SRR307023_2.fastq                            SRR307030_2.fastq
SRR307024_1.fastq
[brownsarahm@n005 ~]$ cat demo.sh 
echo 'script works'
[brownsarahm@n005 ~]$ ./demo.sh 
script works
[brownsarahm@n005 ~]$ ls my_job.sh 
my_job.sh
[brownsarahm@n005 ~]$ cat my_job.sh 
#!/bin/bash
#SBATCH -t 1:00:00
#SBATCH --nodes=1 --ntasks-per-node=1
./single_job
[brownsarahm@n005 ~]$ exit
logout
salloc: Relinquishing job allocation 26471
[brownsarahm@seawulf ~]$ interactive -t 0:03:00
Usage: interactive [-c] [-p] [-J] [-w]

Optional arguments:
    -c: number of CPU cores to request (default: 1)
    -p: partition to run job in (default: general)
    -J: job name (default: interactive)
    -w: node name

NB: interactive jobs have a time limit of 8 hours.

Written by: Alan Orth <a.orth@cgiar.org>
[brownsarahm@seawulf ~]$ interactive -c 4 
salloc: Granted job allocation 26490
salloc: Waiting for resource configuration
salloc: Nodes n005 are ready for job
[brownsarahm@n005 ~]$ exit
logout
salloc: Relinquishing job allocation 26490
[brownsarahm@seawulf ~]$ interactive
salloc: Granted job allocation 26499
salloc: Waiting for resource configuration
salloc: Nodes n005 are ready for job
[brownsarahm@n005 ~]$ python3
Python 3.6.8 (default, Nov 16 2020, 16:55:22) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-44)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 4+3
7
>>> exit
Use exit() or Ctrl-D (i.e. EOF) to exit
>>> exit()(
... )
[brownsarahm@n005 ~]$ exit
logout
salloc: Relinquishing job allocation 26499
[brownsarahm@seawulf ~]$ exit
logout
Connection to seawulf.uri.edu closed.

Important

No badges today

17.3. Experience Report Evidence#

Create ssh keys

17.4. Questions After Today’s Class#

Post via PR

Submit a PR to the course website with your question as a level 3 heading for one community badge and with an attempted answer (I’ll check it before merging it) for two community badges