4. Why Do I Need to Use a terminal?

1. replication/automation

2. more universal

3. cloud computing/ HPC often only have a terminal

4. some more powerful actions are only available on a terminal

We will go back to the same repository we worked with on Friday, for me that was

cd Documents/inclass/systems/github-in-class-brownsarahm/

4.1. Review

We use git status to check on the state of a git repo.

If we get an error that says this is not a git repo, it means there is no .git directory in our current working directory or its parents.

ls
README.md	about.md

ls -a
.		.git		README.md
..		.github		about.md

cd .git
(base) brownsarahm@.git $ ls
COMMIT_EDITMSG	description	info		packed-refs
HEAD		hooks		logs		refs
config		index		objects
(base) brownsarahm@.git $ cd ..

Parents mean working to the left in the path and children are to the right.

pwd

So systems is a child of inclass and the parent directory of github-inclass-brownsarahm

/Users/brownsarahm/Documents/inclass/systems/github-inclass-brownsarahm

4.2. Scenario

Note

A common question is about how to organize projects. While our main focus in this class session is the bash commands to do it, the task that we are going to do is to organize a hypothetical python project

touch abstract_base_class.py helper_functions.py important_classes.py alternative_classes.py README.md LICENSE.md CONTRIBUTING.md setup.py tests_abc.py test_help.py test_imp.py test_alt.py overview.md API.md _config.yml

touch _toc.yml philosophy.md example.md Untitled.ipynb Untitled01.ipynb Untitled02.ipynb

ls

Now we have all of these files, named in abstract ways to signal hypothetical contents and suggest how to organize them.

API.md			_toc.yml		philosophy.md
CONTRIBUTING.md		about.md		setup.py
LICENSE.md		abstract_base_class.py	test_alt.py
README.md		alternative_classes.py	test_help.py
Untitled.ipynb		example.md		test_imp.py
Untitled01.ipynb	helper_functions.py	tests_abc.py
Untitled02.ipynb	important_classes.py
_config.yml		overview.md

cat README.md
# github-inclass-brownsarahm
github-inclass-brownsarahm created by GitHub Classroom

We can add contents to files with echo and >>

echo "Sarah Brown" >> README.md

cat README.md
# github-inclass-brownsarahm
github-inclass-brownsarahm created by GitHub Classroom
Sarah Brown

4.2.1. one versus two >>

cat about.md

echo "a sample project" >> about.md

 echo "testing one >" > about.md

One writes a file and two appends

cat about.md
testing one >

 echo "|file | contents |
> | ------| ------- |
> | abstract_base_class.py | core abstract classes for the project |
> | helper_functions.py | utitly funtions that are called by many classes |
> | important_classes.py | classes that inherit from the abc |
> | alternative_classes.py | classes that inherit from the abc |
> | LICENSE.md | the info on how the code can be reused|
> | CONTRIBUTING.md | instructions for how people can contribute to the project|
> | setup.py | file with function with instructions for pip |
> | tests_abc.py | tests for constructors and methods in abstract_base_class.py|
> | tests_helpers.py | tests for constructors and methods in helper_functions.py|
> | tests_imp.py | tests for constructors and methods in important_classes.py|
> | tests_alt.py | tests for constructors and methods in alternative_classes.py|
> | API.md | jupyterbook file to generate api documentation |
> | _config.yml | jupyterbook config for documentation |
> | _toc.yml | jupyter book toc file for documentation |
> | philosophy.md | overview of how the code is organized for docs |
> | example.md | myst notebook example of using the code |
> | scratch.ipynb | jupyter notebook from dev |" >> README.md

this explains each file a little bit more than the name of it does. We see there are sort of 5 groups of files:

• about the project/repository

• code that defines a python module

• test code

• documentation

• extra files that “we know” we can delete.

cat README.md

# github-inclass-brownsarahm
github-inclass-brownsarahm created by GitHub Classroom
Sarah Brown
|file | contents |
| ------| ------- |
| abstract_base_class.py | core abstract classes for the project |
| helper_functions.py | utitly funtions that are called by many classes |
| important_classes.py | classes that inherit from the abc |
| alternative_classes.py | classes that inherit from the abc |
| LICENSE.md | the info on how the code can be reused|
| CONTRIBUTING.md | instructions for how people can contribute to the project|
| setup.py | file with function with instructions for pip |
| tests_abc.py | tests for constructors and methods in abstract_base_class.py|
| tests_helpers.py | tests for constructors and methods in helper_functions.py|
| tests_imp.py | tests for constructors and methods in important_classes.py|
| tests_alt.py | tests for constructors and methods in alternative_classes.py|
| API.md | jupyterbook file to generate api documentation |
| _config.yml | jupyterbook config for documentation |
| _toc.yml | jupyter book toc file for documentation |
| philosophy.md | overview of how the code is organized for docs |
| example.md | myst notebook example of using the code |
| scratch.ipynb | jupyter notebook from dev |

Note

using the open quote " then you stay inside that until you close it. when you press enter the command does not run until after you close the quotes

 echo " kasdlkfjsdljf
> kjsdfksdj
> sdfjdsl
> fsklsdjf
> "
 kasdlkfjsdljf
kjsdfksdj
sdfjdsl
fsklsdjf

You can use the up arrow to repeat a command

 echo " kasdlkfjsdljf
kjsdfksdj
sdfjdsl
fsklsdjf
" >> junk

and we can see that it keeps the new line characters from the terminal in the file.

 cat junk
 kasdlkfjsdljf
kjsdfksdj
sdfjdsl
fsklsdjf

rm junk

4.3. Making Directories

First we will make directories. We saw mkdir on Friday

mkdir docs

This doesn’t return anything, but we can see the effect with ls

ls

API.md			_toc.yml		overview.md
CONTRIBUTING.md		about.md		philosophy.md
LICENSE.md		abstract_base_class.py	setup.py
README.md		alternative_classes.py	test_alt.py
Untitled.ipynb		docs			test_help.py
Untitled01.ipynb	example.md		test_imp.py
Untitled02.ipynb	helper_functions.py	tests_abc.py
_config.yml		important_classes.py

We might not want to make them all one at a time. Like with touch we can pass multiple names to mkdir with spaces between to make multiple at once.

mkdir tests mymodule

ls

and again check what happened

API.md			about.md		philosophy.md
CONTRIBUTING.md		abstract_base_class.py	setup.py
LICENSE.md		alternative_classes.py	test_alt.py
README.md		docs			test_help.py
Untitled.ipynb		example.md		test_imp.py
Untitled01.ipynb	helper_functions.py	tests
Untitled02.ipynb	important_classes.py	tests_abc.py
_config.yml		mymodule
_toc.yml		overview.md

4.4. Moving files

we can move files with mv. We’ll first move the philosophy.md file into docs and check that it worked.

mv philosophy.md docs/

mv example.md docs/

4.4.1. Getting help in bash

To learn more about the mv command, we can use the man(ual) file.

For GitBash:

mv --help

for *nix (including macos)

Important

press q to exit the program that starts when you do this.

man mv

use enter/return or arrows to scroll and q to quit

If we type something wrong, the error message also provides some help

mv ls
usage: mv [-f | -i | -n] [-v] source target
       mv [-f | -i | -n] [-v] source ... directory

We can use man on any bash command to see the options so we do not need to remember them all, or go to the internet every time we need help. We have high quality help for the details right in the shell, if we remember the basics.

4.4.2. Moving multiple files with patterns

let’s look at the list of files again.

ls

API.md			_toc.yml		overview.md
CONTRIBUTING.md		about.md		setup.py
LICENSE.md		abstract_base_class.py	test_alt.py
README.md		alternative_classes.py	test_help.py
Untitled.ipynb		docs			test_imp.py
Untitled01.ipynb	helper_functions.py	tests
Untitled02.ipynb	important_classes.py	tests_abc.py
_config.yml		mymodule

cat README.md

# github-inclass-brownsarahm
github-inclass-brownsarahm created by GitHub Classroom
Sarah Brown
|file | contents |
| ------| ------- |
| abstract_base_class.py | core abstract classes for the project |
| helper_functions.py | utitly funtions that are called by many classes |
| important_classes.py | classes that inherit from the abc |
| alternative_classes.py | classes that inherit from the abc |
| LICENSE.md | the info on how the code can be reused|
| CONTRIBUTING.md | instructions for how people can contribute to the project|
| setup.py | file with function with instructions for pip |
| tests_abc.py | tests for constructors and methods in abstract_base_class.py|
| tests_helpers.py | tests for constructors and methods in helper_functions.py|
| tests_imp.py | tests for constructors and methods in important_classes.py|
| tests_alt.py | tests for constructors and methods in alternative_classes.py|
| API.md | jupyterbook file to generate api documentation |
| _config.yml | jupyterbook config for documentation |
| _toc.yml | jupyter book toc file for documentation |
| philosophy.md | overview of how the code is organized for docs |
| example.md | myst notebook example of using the code |
| scratch.ipynb | jupyter notebook from dev |

Note

this is why good file naming is important even if you have not organized the whole project yet, you can use the good conventions to help yourself later.

We see that the ones with similar purposes have similar names.

We can use * as a wildcard operator and then move will match files to that pattern and move them all. We’ll start with the two yml (yaml) files that are both for the documentation.

mv *.yml docs/

 ls
API.md			about.md		overview.md
CONTRIBUTING.md		abstract_base_class.py	setup.py
LICENSE.md		alternative_classes.py	test_alt.py
README.md		docs			test_help.py
Untitled.ipynb		helper_functions.py	test_imp.py
Untitled01.ipynb	important_classes.py	tests
Untitled02.ipynb	mymodule		tests_abc.py

4.4.3. Renaming a single file with mv

We see that most of the test files start with test_ but one starts with tests_. We could use the pattern test*.py to move them all without conflicting with the directory tests/ but we also want consistent names.

We can use mv to change the name as well. This is because “moving” a file and is really about changing its path, not actually copying it from one location to another and the file name is a part of the path.

mv tests_abc.py test_abc.py
ls

now that it’s fixed

API.md			abstract_base_class.py	setup.py
CONTRIBUTING.md		alternative_classes.py	test_abc.py
LICENSE.md		docs			test_alt.py
README.md		example.md		test_help.py
Untitled.ipynb		helper_functions.py	test_imp.py
Untitled01.ipynb	important_classes.py	tests
Untitled02.ipynb	mymodule
about.md		overview.md

mv test_* tests/

Note

In class I did the mv tests* before renaming and we did not have to rename. This way is cleaner, but that way worked, albiet with an error

ls

API.md			Untitled02.ipynb	important_classes.py
CONTRIBUTING.md		about.md		mymodule
LICENSE.md		abstract_base_class.py	overview.md
README.md		alternative_classes.py	setup.py
Untitled.ipynb		docs			tests
Untitled01.ipynb	helper_functions.py

ls tests/
test_alt.py	test_help.py	test_imp.py	test_abc.py

ls

API.md			Untitled02.ipynb	important_classes.py
CONTRIBUTING.md		about.md		mymodule
LICENSE.md		abstract_base_class.py	overview.md
README.md		alternative_classes.py	setup.py
Untitled.ipynb		docs			tests
Untitled01.ipynb	helper_functions.py

Now we can move all of the other .py files to the module

mv *.py mymodule/
ls

4.5. Working with relative paths

Let’s review our info again

...
|file | contents |
| ------| ------- |
| abstract_base_class.py | core abstract classes for the project |
| helper_functions.py | utitly funtions that are called by many classes |
| important_classes.py | classes that inherit from the abc |
| alternative_classes.py | classes that inherit from the abc |
| LICENSE.md | the info on how the code can be reused|
| CONTRIBUTING.md | instructions for how people can contribute to the project|
| setup.py | file with function with instructions for pip |
| tests_abc.py | tests for constructors and methods in abstract_base_class.py|
| tests_helpers.py | tests for constructors and methods in helper_functions.py|
| tests_imp.py | tests for constructors and methods in important_classes.py|
| tests_alt.py | tests for constructors and methods in alternative_classes.py|
| API.md | jupyterbook file to generate api documentation |
| _config.yml | jupyterbook config for documentation |
| _toc.yml | jupyter book toc file for documentation |
| philosophy.md | overview of how the code is organized for docs |
| example.md | myst notebook example of using the code |
| scratch.ipynb | jupyter notebook from dev |

We’ve made a mistake, setup.py is actually instructions that need to be at the top level, not inside the module’s sub directory.

We can get it back using the relative path to the file and then using . to move it to where we “are” sicne we are in the top level directory still.

mv mymodule/setup.py .
ls

API.md			Untitled01.ipynb	overview.md
CONTRIBUTING.md		Untitled02.ipynb	setup.py
LICENSE.md		about.md		tests
README.md		docs
Untitled.ipynb		mymodule

Or, if we put it back temporarily

mv setup.py mymodule/

We can cd to where we put it

cd mymodule/
ls

abstract_base_class.py helper_functions.py setup.py alternative_classes.py important_classes.py

and move it up a level using `..`
```bash
mv setup.py ..

and then go back

cd ..

Now we’ll move the last few docs files.

mv API.md docs/
mv example.md docs/

ls
API.md			Untitled01.ipynb	overview.md
CONTRIBUTING.md		Untitled02.ipynb	setup.py
LICENSE.md		about.md		tests
README.md		docs
Untitled.ipynb		mymodule

4.6. More relative paths

We need a __init__.py in the mymodule directory but we are in the docs directory currently. No problem!

touch mymodule/__init__.py

ls mymodule/
__init__.py		alternative_classes.py	important_classes.py
abstract_base_class.py	helper_functions.py

4.7. Copying

The typical contents of the README we would also want in the documentation website. We might add to the file later, but that’s a good start. We can do that by copying.

When we copy we designate the file to copy and a path/name for the copy we want to make.

cp README.md docs/overview.md

cp about.md docs/

ls docs/
_config.yml	about.md	overview.md
_toc.yml	example.md	philosophy.md

cat about.md
testing one >

cat docs/about.md
testing one >

4.8. Removing files

We still have to deal with the untitled files that we know we don’t need any more.

Tab completion

If we type

rm U

then it canot tab complete the whole file name we get only

rm Untitled

because there are multiple. If we prest tab twice in a row, it will return the list of possible options

Untitled.ipynb    Untitled01.ipynb  Untitled02.ipynb  

and re-seed the input with

rm Untitled

so we can type whatever else we want to add to pick the one we want.

we can delete them with rm and use * to delete them all.

rm Untitled*

 ls
API.md		LICENSE.md	about.md	mymodule	setup.py
CONTRIBUTING.md	README.md	docs		overview.md	tests

 mv API.md docs/

4.9. Git Order of operations

Important

We did not push in class, but you are supposed to so I added some tips here

To create and switch to a new branch at once you can use

git checkout -b newbranch

where newbranch is the name of your new branch

git status

Then you can see which branch you are on with git status

On branch newbranch
...

4.10. Recap

Why do I need a terminal

1. replication/automation

2. it’s always there and doesn’t change

3. it’s faster one you know it (also see above)

So, is the shell the feature that interacts with the operating system and then the terminal is the gui that interacts with the shell?

Important

if your push gets rejected, read the hints, it probably has the answer. We will come back to that error though

4.11. Review today’s class

1. Review the notes

2. Reorganize a folder on your computer ( good candidate may be desktop or downloads folder), using only a terminal to make new directories, move files, check what’s inside them, etc. Answer reflection questions (will be in notes) in a new file, terminal.md in your kwl repo.

3. Make a PR that adds a glossary entry to your team repo to define a term we have learned so far. Create an issue and tag yourself to “claim” a term and check the issues and PRs that are open before your.

4. Review past classes activities (eg via the activities section on the left) and catchup if appropriate

Start with a file explorer open, but then try to close it and use only command line tools to explore and make your choices

### Terminal File moving reflection
1. Did this get easier toward the end?
1. Use the history to see which commands you used and how many times each, make a table below.
1. Did you have to look up how to do anything we had not done in class?
1. When do you think that using the terminal will be better than using your GUI file explorer?
1. What questions/challenges/ relfections do you have after this?
1. What kinds of things might you want to write a bash script for given what you know in bash so far? come up with 1-2 scenarios

4.12. Prepare for Next Class

Note

This is what is required, before the next class and will be checked or if you don’t do it you will have trouble participating in class

1. Examine a large project you have done or by finding an open source project on GitHub. Answer the reflection questions in software.md in your kwl repo. (will be in notes)

2. install h/w simulator

3. map out how you think about data moving through a small program and bring it with you to class (no need to submit)

## Software Reflection

1. link to public repo if applicable or title of your project
1. What types of files are there that are not code?
1. What different types of code files are in the project? Do they serve different goals?
1. Is it all in one language or are there multiple?
1. Try to figure out (remember) how the project works. What types of things, without running the code can you look at at a high level?

4.13. More Practice

1. Once your PRs in your kwl are merged, add a Table of Contents to the README with relative links to each file

2. Add cheatsheet entry to your team repo to do something of interest with git or shell. Make an issue for it first and assign yourself so that your team mates know you are working on that topic.

4.14. Questions After Class

4.14.1. What’s the difference in creating a file using touch vs echo?

touch creates an empty file echo writes content to a place. We used with with > and >> to redirect that output to a file for today, but we will learn more about that later.

4.14.2. I thought you could restore the state of the system using snapshots? Say if you remove something or something gets corrupted

You can restore if a snapshot was created, not all computers have that, and most recovery snapshots are not created very frequently (like once a day or week).

4.14.3. what are other things I can do with the terminal?

There are lots of small programs and you can pipe them all together. We will come back to the shell in the coming weeks.

4.14.4. What are examples of what we are learning used in a professional setting?

Most companies use git to track their code and most dev jobs you will need to log into a server that only has a shell at some point.

4.14.5. if you commit something offline and then close the terminal, can you push it next time you log on or is it lost?

Yes, the file is saved in your computer. You do not actually even need to ever push it to a remote to be using git as version control.

4.14.6. How can we edit files on the terminal?

To edit files in more detailed ways, you use a text editor, some are built for the command line. We will see one later.

4.14.7. Where am I supposed to add the “terminal.md” file?

To your KWL repo

4.14.8. Show should the team repo be laid out and should we each determine our responsibilities for them? What should be expected to be inside the team repo?

The repo has some outline in it. Remember it is not a team project where you are graded together. You will have various activities assigned there throughout the semester in order to practice collaborating. Every student will need to do every role, so that you get to learn them all.

4.14.9. When will grading start?

Next week, we’ll start your grading contracts on Wednesday.