17. How do I use a remote server more effectively?#
Important
Since I’ve been sick, this is not fully filled in yet. Use a previous semester’s equivalent sessions as a reference for more detail for now. Create a PR that inserts an explanation of a few steps for a community badge.
Previous notes:
ls
example
github-in-class-brownsarahm-1
kwl
seawulf
test
testobj.md
tiny-book
ssh -l brownsarahm seawulf.uri.edu
brownsarahm@seawulf.uri.edu's password:
Last failed login: Thu Mar 30 12:53:35 EDT 2023 from pool-96-238-44-82.prvdri.fios.verizon.net on ssh:notty
There was 1 failed login attempt since the last successful login.
Last login: Thu Mar 30 11:54:40 2023 from pool-72-87-118-171.prvdri.fios.verizon.net
[brownsarahm@seawulf ~]$ pwd
/home/brownsarahm
[brownsarahm@seawulf ~]$ ls
bash-lesson.tar.gz SRR307024_2.fastq
bash-lesson.tar.gz.1 SRR307025_1.fastq
demo.sh SRR307025_2.fastq
dmel-all-r6.19.gtf SRR307026_1.fastq
dmel_unique_protein_isoforms_fb_2016_01.tsv SRR307026_2.fastq
gene_association.fb SRR307027_1.fastq
linecounts.txt SRR307027_2.fastq
my_job.sh SRR307028_1.fastq
results SRR307028_2.fastq
slurm-23950.out SRR307029_1.fastq
SRR307023_1.fastq SRR307029_2.fastq
SRR307023_2.fastq SRR307030_1.fastq
SRR307024_1.fastq SRR307030_2.fastq
[brownsarahm@seawulf ~]$ mkdir example
[brownsarahm@seawulf ~]$ cd example/
[brownsarahm@seawulf example]$ pwd
/home/brownsarahm/example
[brownsarahm@seawulf example]$ mkdir ex2
[brownsarahm@seawulf example]$ cd ex2/
[brownsarahm@seawulf ex2]$ pwd
/home/brownsarahm/example/ex2
[brownsarahm@seawulf ex2]$ cd
[brownsarahm@seawulf ~]$ ls -l
total 150704
-rw-r--r--. 1 brownsarahm spring2022-csc392 12534006 Apr 18 2021 bash-lesson.tar.gz
-rw-r--r--. 1 brownsarahm spring2022-csc392 12534006 Apr 18 2021 bash-lesson.tar.gz.1
-rwxr-xr-x. 1 brownsarahm spring2022-csc392 20 Oct 26 17:11 demo.sh
-rw-r--r--. 1 brownsarahm spring2022-csc392 77426528 Jan 16 2018 dmel-all-r6.19.gtf
-rw-r--r--. 1 brownsarahm spring2022-csc392 721242 Jan 25 2016 dmel_unique_protein_isoforms_fb_2016_01.tsv
drwxr-xr-x. 3 brownsarahm spring2022-csc392 24 Mar 30 12:59 example
-rw-r--r--. 1 brownsarahm spring2022-csc392 25056938 Jan 25 2016 gene_association.fb
-rw-r--r--. 1 brownsarahm spring2022-csc392 447 Mar 8 2022 linecounts.txt
-rw-r--r--. 1 brownsarahm spring2022-csc392 84 Mar 8 2022 my_job.sh
drwxr-xr-x. 2 brownsarahm spring2022-csc392 10 Mar 8 2022 results
-rw-r--r--. 1 brownsarahm spring2022-csc392 89 Mar 8 2022 slurm-23950.out
-rw-r--r--. 1 brownsarahm spring2022-csc392 1625262 Jan 25 2016 SRR307023_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392 1625262 Jan 25 2016 SRR307023_2.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392 1625376 Jan 25 2016 SRR307024_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392 1625376 Jan 25 2016 SRR307024_2.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392 1625286 Jan 25 2016 SRR307025_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392 1625286 Jan 25 2016 SRR307025_2.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392 1625302 Jan 25 2016 SRR307026_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392 1625302 Jan 25 2016 SRR307026_2.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392 1625312 Jan 25 2016 SRR307027_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392 1625312 Jan 25 2016 SRR307027_2.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392 1625338 Jan 25 2016 SRR307028_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392 1625338 Jan 25 2016 SRR307028_2.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392 1625390 Jan 25 2016 SRR307029_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392 1625390 Jan 25 2016 SRR307029_2.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392 1625318 Jan 25 2016 SRR307030_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392 1625318 Jan 25 2016 SRR307030_2.fastq
[brownsarahm@seawulf ~]$ ls -l --block-size=M
total 148M
-rw-r--r--. 1 brownsarahm spring2022-csc392 12M Apr 18 2021 bash-lesson.tar.gz
-rw-r--r--. 1 brownsarahm spring2022-csc392 12M Apr 18 2021 bash-lesson.tar.gz.1
-rwxr-xr-x. 1 brownsarahm spring2022-csc392 1M Oct 26 17:11 demo.sh
-rw-r--r--. 1 brownsarahm spring2022-csc392 74M Jan 16 2018 dmel-all-r6.19.gtf
-rw-r--r--. 1 brownsarahm spring2022-csc392 1M Jan 25 2016 dmel_unique_protein_isoforms_fb_2016_01.tsv
drwxr-xr-x. 3 brownsarahm spring2022-csc392 1M Mar 30 12:59 example
-rw-r--r--. 1 brownsarahm spring2022-csc392 24M Jan 25 2016 gene_association.fb
-rw-r--r--. 1 brownsarahm spring2022-csc392 1M Mar 8 2022 linecounts.txt
-rw-r--r--. 1 brownsarahm spring2022-csc392 1M Mar 8 2022 my_job.sh
drwxr-xr-x. 2 brownsarahm spring2022-csc392 1M Mar 8 2022 results
-rw-r--r--. 1 brownsarahm spring2022-csc392 1M Mar 8 2022 slurm-23950.out
-rw-r--r--. 1 brownsarahm spring2022-csc392 2M Jan 25 2016 SRR307023_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392 2M Jan 25 2016 SRR307023_2.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392 2M Jan 25 2016 SRR307024_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392 2M Jan 25 2016 SRR307024_2.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392 2M Jan 25 2016 SRR307025_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392 2M Jan 25 2016 SRR307025_2.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392 2M Jan 25 2016 SRR307026_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392 2M Jan 25 2016 SRR307026_2.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392 2M Jan 25 2016 SRR307027_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392 2M Jan 25 2016 SRR307027_2.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392 2M Jan 25 2016 SRR307028_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392 2M Jan 25 2016 SRR307028_2.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392 2M Jan 25 2016 SRR307029_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392 2M Jan 25 2016 SRR307029_2.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392 2M Jan 25 2016 SRR307030_1.fastq
-rw-r--r--. 1 brownsarahm spring2022-csc392 2M Jan 25 2016 SRR307030_2.fastq
[brownsarahm@seawulf ~]$ exit
logout
Connection to seawulf.uri.edu closed.
17.1. Creating SSH Keys#
ssh-keygen -f ~/seawulf -t rsa -b 1024
Generating public/private rsa key pair.
/Users/brownsarahm/seawulf already exists.
Overwrite (y/n)? y
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /Users/brownsarahm/seawulf.
Your public key has been saved in /Users/brownsarahm/seawulf.pub.
The key fingerprint is:
SHA256:NK9gJGqWW4E2B5AI6Alb7+34JGYQ2QS7BpCM081WQfU brownsarahm@Sarahs-MBP
The key's randomart image is:
+---[RSA 1024]----+
|X=o+.o+o. |
|X.oB+ . |
|++Oo= . o E |
|.= B.+ . o |
| O...o S . |
| + +.... . |
| . +o. . |
| o.o. |
| .. |
+----[SHA256]-----+
ssh-copy-id -i ~/seawulf brownsarahm@seawulf.uri.edu
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/Users/brownsarahm/seawulf.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
brownsarahm@seawulf.uri.edu's password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh 'brownsarahm@seawulf.uri.edu'"
and check to make sure that only the key(s) you wanted were added.
17.2. Logging in#
ssh brownsarahm@seawulf.uri.edu
brownsarahm@seawulf.uri.edu's password:
Last login: Thu Mar 30 13:18:24 2023 from pool-72-87-118-171.prvdri.fios.verizon.net
[brownsarahm@seawulf ~]$ pwd
/home/brownsarahm
[brownsarahm@seawulf ~]$ lshw
WARNING: you should run this program as super-user.
seawulf.uri.edu
description: Computer
width: 64 bits
capabilities: smp vsyscall32
*-core
description: Motherboard
physical id: 0
*-memory
description: System memory
physical id: 0
size: 32GiB
*-cpu:0
product: Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz
vendor: Intel Corp.
vendor_id: GenuineIntel
physical id: 1
bus info: cpu@0
version: 6.79.1
size: 1200MHz
capacity: 2900MHz
width: 64 bits
capabilities: fpu fpu_exception wp vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp x86-64 constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch epb cat_l3 cdp_l3 invpcid_single intel_ppin intel_pt ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts spec_ctrl intel_stibp cpufreq
Truncated output
[brownsarahm@seawulf ~]$ interactive
salloc: Granted job allocation 26471
salloc: Waiting for resource configuration
salloc: Nodes n005 are ready for job
[brownsarahm@n005 ~]$ ls
bash-lesson.tar.gz SRR307024_2.fastq
bash-lesson.tar.gz.1 SRR307025_1.fastq
demo.sh SRR307025_2.fastq
dmel-all-r6.19.gtf SRR307026_1.fastq
dmel_unique_protein_isoforms_fb_2016_01.tsv SRR307026_2.fastq
example SRR307027_1.fastq
gene_association.fb SRR307027_2.fastq
linecounts.txt SRR307028_1.fastq
my_job.sh SRR307028_2.fastq
results SRR307029_1.fastq
slurm-23950.out SRR307029_2.fastq
SRR307023_1.fastq SRR307030_1.fastq
SRR307023_2.fastq SRR307030_2.fastq
SRR307024_1.fastq
[brownsarahm@n005 ~]$ cat demo.sh
echo 'script works'
[brownsarahm@n005 ~]$ ./demo.sh
script works
[brownsarahm@n005 ~]$ ls my_job.sh
my_job.sh
[brownsarahm@n005 ~]$ cat my_job.sh
#!/bin/bash
#SBATCH -t 1:00:00
#SBATCH --nodes=1 --ntasks-per-node=1
./single_job
[brownsarahm@n005 ~]$ exit
logout
salloc: Relinquishing job allocation 26471
[brownsarahm@seawulf ~]$ interactive -t 0:03:00
Usage: interactive [-c] [-p] [-J] [-w]
Optional arguments:
-c: number of CPU cores to request (default: 1)
-p: partition to run job in (default: general)
-J: job name (default: interactive)
-w: node name
NB: interactive jobs have a time limit of 8 hours.
Written by: Alan Orth <a.orth@cgiar.org>
[brownsarahm@seawulf ~]$ interactive -c 4
salloc: Granted job allocation 26490
salloc: Waiting for resource configuration
salloc: Nodes n005 are ready for job
[brownsarahm@n005 ~]$ exit
logout
salloc: Relinquishing job allocation 26490
[brownsarahm@seawulf ~]$ interactive
salloc: Granted job allocation 26499
salloc: Waiting for resource configuration
salloc: Nodes n005 are ready for job
[brownsarahm@n005 ~]$ python3
Python 3.6.8 (default, Nov 16 2020, 16:55:22)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-44)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 4+3
7
>>> exit
Use exit() or Ctrl-D (i.e. EOF) to exit
>>> exit()(
... )
[brownsarahm@n005 ~]$ exit
logout
salloc: Relinquishing job allocation 26499
[brownsarahm@seawulf ~]$ exit
logout
Connection to seawulf.uri.edu closed.
Important
No badges today
17.3. Experience Report Evidence#
Create ssh keys
17.4. Questions After Today’s Class#
Post via PR
Submit a PR to the course website with your question as a level 3 heading for one community badge and with an attempted answer (I’ll check it before merging it) for two community badges