.ipynb .md .pdf

repository open issue suggest edit

Binder

Contents

4. Why Do I Need to Use a terminal?

We will go back to the same repository we worked with on Tuesday, for me that was

cd Documents/teaching/sysinclass/github-in-class-brownsarahm/

We can use touch that we saw Tuesday to create many files at once:

touch abstract_base_class.py helper_functions.py important_classes.py alternative_classes.py README.md LICENSE.md CONTRIBUTING.md setup.py tests_abc.py test_help.py test_imp.py test_alt.py overview.md API.md _config.yml
_toc.yml philosophy.md example.md Untitled.ipynb Untitled01.ipynb Untitled02.ipynb

we got an error from this:

-bash: _toc.yml: command not found

we can intermet this, bash thought _toc.yml was a command. That means there was a hard to see accidental line break in the text above.

If we didn’t know what that meant, we could also investigate further using ls to list.

ls

we see we have most of the files actually created,

API.md			abstract_base_class.py	test_alt.py
CONTRIBUTING.md		alternative_classes.py	test_help.py
LICENSE.md		helper_functions.py	test_imp.py
README.md		important_classes.py	tests_abc.py
_config.yml		overview.md
about.md		setup.py

we can use the up arrow key to get back the last line.

_toc.yml philosophy.md example.md Untitled.ipynb Untitled01.ipynb Untitled02.ipynb

and add touch at the start of it to create those last few files.

touch _toc.yml philosophy.md example.md Untitled.ipynb Untitled01.ipynb Untitled02.ipynb

and confirm

ls

4.1. Scenario

Note

a few of you asked about learning how to organize projects. While our main focus in this class session is the bash commands to do it, the task that we are going to do is to organize a hypothetical python project

Now we have all of these files, named in abstract ways to signal hypothecial contents and suggest how to organize them.

API.md			_toc.yml		philosophy.md
CONTRIBUTING.md		about.md		setup.py
LICENSE.md		abstract_base_class.py	test_alt.py
README.md		alternative_classes.py	test_help.py
Untitled.ipynb		example.md		test_imp.py
Untitled01.ipynb	helper_functions.py	tests_abc.py
Untitled02.ipynb	important_classes.py
_config.yml		overview.md

First we’re goign to paste some contents (shared from prismia, view below) in to the readme with nano

nano README.md

We can view the contents of the file using cat to print the contents to the terminal output.

cat README.md

and we see:

# GitHub Practice

Name: sarah
|file | contents |
| ------| ------- |
| abstract_base_class.py | core abstract classes for the project |
| helper_functions.py | utitly funtions that are called by many classes |
| important_classes.py | classes that inherit from the abc |
| alternative_classes.py | classes that inherit from the abc |
| LICENSE.md | the info on how the code can be reused|
| CONTRIBUTING.md | instructions for how people can contribute to the project|
| setup.py | file with function with instructions for pip |
| tests_abc.py | tests for constructors and methods in abstract_base_class.py|
| tests_helpers.py | tests for constructors and methods in helper_functions.py|
| tests_imp.py | tests for constructors and methods in important_classes.py|
| tests_alt.py | tests for constructors and methods in alternative_classes.py|
| API.md | jupyterbook file to generate api documentation |
| _config.yml | jupyterbook config for documentation |
| _toc.yml | jupyter book toc file for documentation |
| philosophy.md | overview of how the code is organized for docs |
| example.md | myst notebook example of using the code |
| Untitled*.ipynb | jupyter notebook from dev, not important to keep |

this explains each file a little bit more than the name of it does. We see there are sort of 5 groups of files:

• about the project/repository

• code that defines a python module

• test code

• documentation

• extra files that “we know” we can delete.

4.2. Making Directories

First we will make directories. We saw mkdir on Tuesday

mkdir docs/

This doesn’t return anything, but we can see the effect with ls

ls

API.md			_toc.yml		overview.md
CONTRIBUTING.md		about.md		philosophy.md
LICENSE.md		abstract_base_class.py	setup.py
README.md		alternative_classes.py	test_alt.py
Untitled.ipynb		docs			test_help.py
Untitled01.ipynb	example.md		test_imp.py
Untitled02.ipynb	helper_functions.py	tests_abc.py
_config.yml		important_classes.py

We might not want to make them all one at a time. Like with touch we can pass multiple names to mkdir with spaces between to make multiple at once.

mkdir tests mymodule

and again use ls to see the output

API.md			about.md		philosophy.md
CONTRIBUTING.md		abstract_base_class.py	setup.py
LICENSE.md		alternative_classes.py	test_alt.py
README.md		docs			test_help.py
Untitled.ipynb		example.md		test_imp.py
Untitled01.ipynb	helper_functions.py	tests
Untitled02.ipynb	important_classes.py	tests_abc.py
_config.yml		mymodule
_toc.yml		overview.md

4.3. Moving files

we can move files with mv. We’ll first move the philosophy.md file into docs and check that it worked.

mv philosophy.md docs/
ls

API.md			_toc.yml		mymodule
CONTRIBUTING.md		about.md		overview.md
LICENSE.md		abstract_base_class.py	setup.py
README.md		alternative_classes.py	test_alt.py
Untitled.ipynb		docs			test_help.py
Untitled01.ipynb	example.md		test_imp.py
Untitled02.ipynb	helper_functions.py	tests
_config.yml		important_classes.py	tests_abc.py

4.3.1. Getting help in bash

To learn more about the mv command, we can use the man(ual) file.

man mv

use enter/return or arrows to scroll and q to quit

If we type something wrong, the error message also provides some help

mv ls
usage: mv [-f | -i | -n] [-v] source target
       mv [-f | -i | -n] [-v] source ... directory

We can use man on any bash command to see the options so we do not need to remember them all, or go to the internet every time we need help. We have high quality help for the details right in the shell, if we remember the basics.

man ls

ls -hl

total 16
-rw-r--r--  1 brownsarahm  staff     0B Feb  3 12:51 API.md
-rw-r--r--  1 brownsarahm  staff     0B Feb  3 12:51 CONTRIBUTING.md
-rw-r--r--  1 brownsarahm  staff     0B Feb  3 12:51 LICENSE.md
-rw-r--r--  1 brownsarahm  staff   1.2K Feb  3 12:56 README.md
-rw-r--r--  1 brownsarahm  staff     0B Feb  3 12:52 Untitled.ipynb
-rw-r--r--  1 brownsarahm  staff     0B Feb  3 12:52 Untitled01.ipynb
-rw-r--r--  1 brownsarahm  staff     0B Feb  3 12:52 Untitled02.ipynb
-rw-r--r--  1 brownsarahm  staff     0B Feb  3 12:51 _config.yml
-rw-r--r--  1 brownsarahm  staff     0B Feb  3 12:52 _toc.yml
-rw-r--r--  1 brownsarahm  staff    14B Feb  1 13:23 about.md
-rw-r--r--  1 brownsarahm  staff     0B Feb  3 12:51 abstract_base_class.py
-rw-r--r--  1 brownsarahm  staff     0B Feb  3 12:51 alternative_classes.py
drwxr-xr-x  3 brownsarahm  staff    96B Feb  3 13:04 docs
-rw-r--r--  1 brownsarahm  staff     0B Feb  3 12:52 example.md
-rw-r--r--  1 brownsarahm  staff     0B Feb  3 12:51 helper_functions.py
-rw-r--r--  1 brownsarahm  staff     0B Feb  3 12:51 important_classes.py
drwxr-xr-x  2 brownsarahm  staff    64B Feb  3 13:01 mymodule
-rw-r--r--  1 brownsarahm  staff     0B Feb  3 12:51 overview.md
-rw-r--r--  1 brownsarahm  staff     0B Feb  3 12:51 setup.py
-rw-r--r--  1 brownsarahm  staff     0B Feb  3 12:51 test_alt.py
-rw-r--r--  1 brownsarahm  staff     0B Feb  3 12:51 test_help.py
-rw-r--r--  1 brownsarahm  staff     0B Feb  3 12:51 test_imp.py
drwxr-xr-x  2 brownsarahm  staff    64B Feb  3 13:01 tests
-rw-r--r--  1 brownsarahm  staff     0B Feb  3 12:51 tests_abc.py

In some versions of bash we can use:

mv --help

4.3.2. Moving multiple files with patterns

let’s look at the list of files again.

cat README.md

# GitHub Practice

Name: sarah
|file | contents |
| ------| ------- |
| abstract_base_class.py | core abstract classes for the project |
| helper_functions.py | utitly funtions that are called by many classes |
| important_classes.py | classes that inherit from the abc |
| alternative_classes.py | classes that inherit from the abc |
| LICENSE.md | the info on how the code can be reused|
| CONTRIBUTING.md | instructions for how people can contribute to the project|
| setup.py | file with function with instructions for pip |
| test_abc.py | tests for constructors and methods in abstract_base_class.py|
| test_helpers.py | tests for constructors and methods in helper_functions.py|
| test_imp.py | tests for constructors and methods in important_classes.py|
| tests_alt.py | tests for constructors and methods in alternative_classes.py|
| API.md | jupyterbook file to generate api documentation |
| _config.yml | jupyterbook config for documentation |
| _toc.yml | jupyter book toc file for documentation |
| philosophy.md | overview of how the code is organized for docs |
| example.md | myst notebook example of using the code |
| scratch.ipynb | jupyter notebook from dev |

Note

this is why good file naming is important even if you have not organized the whole project yet, you can use the good conventions to help yourself later.

We see that the ones with similar purposes have similar names.

We can use * as a wildcard operator and then move will match files to that pattern and move them all. We’ll start with the two yml (yaml) files that are both for the documentation.

mv *.yml docs/

4.3.3. Renaming a single file with mv

We see that most of the test files start with test_ but one starts with tests_. We could use the pattern test*.py to move them all without conflicting with the directory tests/ but we also want consistent names.

We can use mv to change the name as well. This is because “moving” a file and is really about changing its path, not actually copying it from one location to another and the file name is a part of the path.

mv tests_abc.py test_abc.py
ls

now that it’s fixed

API.md			abstract_base_class.py	setup.py
CONTRIBUTING.md		alternative_classes.py	test_abc.py
LICENSE.md		docs			test_alt.py
README.md		example.md		test_help.py
Untitled.ipynb		helper_functions.py	test_imp.py
Untitled01.ipynb	important_classes.py	tests
Untitled02.ipynb	mymodule
about.md		overview.md

We can use the pattern test_* to move them all.

mv test_* tests/
ls

API.md			Untitled02.ipynb	helper_functions.py
CONTRIBUTING.md		about.md		important_classes.py
LICENSE.md		abstract_base_class.py	mymodule
README.md		alternative_classes.py	overview.md
Untitled.ipynb		docs			setup.py
Untitled01.ipynb	example.md		tests

Now we can move all of the other .py files to the module

mv *.py mymodule/
ls

API.md			Untitled01.ipynb	mymodule
CONTRIBUTING.md		Untitled02.ipynb	overview.md
LICENSE.md		about.md		tests
README.md		docs
Untitled.ipynb		example.md

4.4. Working with relative paths

Let’s review our info again

cat README.md
# GitHub Practice

Name: sarah
|file | contents |
| ------| ------- |
| abstract_base_class.py | core abstract classes for the project |
| helper_functions.py | utitly funtions that are called by many classes |
| important_classes.py | classes that inherit from the abc |
| alternative_classes.py | classes that inherit from the abc |
| LICENSE.md | the info on how the code can be reused|
| CONTRIBUTING.md | instructions for how people can contribute to the project|
| setup.py | file with function with instructions for pip |
| tests_abc.py | tests for constructors and methods in abstract_base_class.py|
| tests_helpers.py | tests for constructors and methods in helper_functions.py|
| tests_imp.py | tests for constructors and methods in important_classes.py|
| tests_alt.py | tests for constructors and methods in alternative_classes.py|
| API.md | jupyterbook file to generate api documentation |
| _config.yml | jupyterbook config for documentation |
| _toc.yml | jupyter book toc file for documentation |
| philosophy.md | overview of how the code is organized for docs |
| example.md | myst notebook example of using the code |
| scratch.ipynb | jupyter notebook from dev |

We’ve made a mistake, setup.py is actually instructions that need to be at the top level, not inside the module’s sub directory.

We can get it back using the relative path to the file and then using . to move it to where we “are” sicne we are in the top level directory still.

mv mymodule/setup.py .
ls

API.md			Untitled01.ipynb	mymodule
CONTRIBUTING.md		Untitled02.ipynb	overview.md
LICENSE.md		about.md		setup.py
README.md		docs			tests
Untitled.ipynb		example.md

Or, if we put it back temporarily

mv setup.py mymodule/

We can cd to where we put it

cd mymodule/
ls

abstract_base_class.py	helper_functions.py	setup.py
alternative_classes.py	important_classes.py

and move it up a level using ..

mv setup.py ..
ls

abstract_base_class.py	helper_functions.py
alternative_classes.py	important_classes.py

then the .. to go up a level gets us back to where we were.

cd ..
ls

API.md			Untitled01.ipynb	mymodule
CONTRIBUTING.md		Untitled02.ipynb	overview.md
LICENSE.md		about.md		setup.py
README.md		docs			tests
Untitled.ipynb		example.md

Now we’ll move the last few docs files.

mv API.md docs/
mv example.md docs/
mv overview.md docs/
ls

4.5. Removing files

We still have to deal with the untitled files that we know we don’t need any more.

CONTRIBUTING.md		Untitled01.ipynb	mymodule
LICENSE.md		Untitled02.ipynb	setup.py
README.md		about.md		tests
Untitled.ipynb		docs

Tab completion

If we type

rm U

then it canot tab complete the whole file name we get only

rm Untitled

because there are multiple. If we prest tab twice in a row, it will return the list of possible options

Untitled.ipynb    Untitled01.ipynb  Untitled02.ipynb  

and re-seed the input with

rm Untitled

so we can type whatever else we want to add to pick the one we want.

we can delete them with rm and use * to delet them all.

rm Untitled*

ls

now we have a nice clean repository.

CONTRIBUTING.md	README.md	docs		setup.py
LICENSE.md	about.md	mymodule	tests

4.6. Copying

The typical contents of the README we would also want in the documentation website. We might add to the file later, but that’s a good start. We can do that by copying.

When we copy we designate the file to copy and a path/name for the copy we want to make.

cp README.md docs/index.md
cd docs/
ls

API.md		_toc.yml	index.md	philosophy.md
_config.yml	example.md	overview.md

we can check the contents of the file too:

cat index.md

# GitHub Practice

Name: sarah
|file | contents |
| ------| ------- |
| abstract_base_class.py | core abstract classes for the project |
| helper_functions.py | utitly funtions that are called by many classes |
| important_classes.py | classes that inherit from the abc |
| alternative_classes.py | classes that inherit from the abc |
| LICENSE.md | the info on how the code can be reused|
| CONTRIBUTING.md | instructions for how people can contribute to the project|
| setup.py | file with function with instructions for pip |
| tests_abc.py | tests for constructors and methods in abstract_base_class.py|
| tests_helpers.py | tests for constructors and methods in helper_functions.py|
| tests_imp.py | tests for constructors and methods in important_classes.py|
| tests_alt.py | tests for constructors and methods in alternative_classes.py|
| API.md | jupyterbook file to generate api documentation |
| _config.yml | jupyterbook config for documentation |
| _toc.yml | jupyter book toc file for documentation |
| philosophy.md | overview of how the code is organized for docs |
| example.md | myst notebook example of using the code |
| scratch.ipynb | jupyter notebook from dev |

it matches .

4.7. More relative paths

We need a __init__.py in the mymodule directory but we are in the docs directory currently. No problem!

touch ../mymodule/__init__.py

git status

On branch main
Your branch is up to date with 'origin/main'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   ../README.md

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	../CONTRIBUTING.md
	../LICENSE.md
	./
	../mymodule/
	../setup.py
	../tests/

no changes added to commit (use "git add" and/or "git commit -a")

note we have both changes and untracked files… but wherre is the docs folder?

we’re in there so all of the files are listed relative to there, so it’s the ./ line to say that we are currently in an untracked directory. Git status doesn’t look inside directories it doens’t know if it should track or not.

if we go back to the top level

cd ..

git status

Tip

Compare these two outputs carefully. Getting used to noticing these details will help you get yourself unstuck!

On branch main
Your branch is up to date with 'origin/main'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   README.md

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	CONTRIBUTING.md
	LICENSE.md
	docs/
	mymodule/
	setup.py
	tests/

no changes added to commit (use "git add" and/or "git commit -a")

Then we can add the changes and push

git add .
git commit -m 'insclass 2-3'

[main c15cf43] insclass 2-3
 20 files changed, 41 insertions(+)
 create mode 100644 CONTRIBUTING.md
 create mode 100644 LICENSE.md
 create mode 100644 docs/API.md
 create mode 100644 docs/_config.yml
 create mode 100644 docs/_toc.yml
 create mode 100644 docs/example.md
 create mode 100644 docs/index.md
 create mode 100644 docs/overview.md
 create mode 100644 docs/philosophy.md
 create mode 100644 mymodule/__init__.py
 create mode 100644 mymodule/abstract_base_class.py
 create mode 100644 mymodule/alternative_classes.py
 create mode 100644 mymodule/helper_functions.py
 create mode 100644 mymodule/important_classes.py
 create mode 100644 setup.py
 create mode 100644 tests/test_abc.py
 create mode 100644 tests/test_alt.py
 create mode 100644 tests/test_help.py
 create mode 100644 tests/test_imp.py

git push

Enumerating objects: 9, done.
Counting objects: 100% (9/9), done.
Delta compression using up to 8 threads
Compressing objects: 100% (6/6), done.
Writing objects: 100% (7/7), 1.22 KiB | 1.22 MiB/s, done.
Total 7 (delta 0), reused 1 (delta 0), pack-reused 0
To https://github.com/introcompsys/github-in-class-brownsarahm.git
   17320fc..c15cf43  main -> main

Important

if your push gets rejected, read the hints, it probably has the answer. We will come back to that error though

4.8. Git order of operations

above since we didn’t make a branch we pushed to main.

git status

On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean

We could make the file changes first and then make the branch we want to commit them too as well. it’s best to make the branch first so you don’t forget, but it is an option

touch test_file.md

git status

On branch main
Your branch is up to date with 'origin/main'.

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	test_file.md

nothing added to commit but untracked files present (use "git add" to track)

git checkout -b test

git status

On branch test
Untracked files:
  (use "git add <file>..." to include in what will be committed)
	test_file.md

nothing added to commit but untracked files present (use "git add" to track)

4.9. Recap

Why do I need a terminal

1. replication/automation

2. it’s always there and doesn’t change

3. it’s faster one you know it (also see above)

So, is the shell the feature that interacts with the operating system and then the terminal is the gui that interacts with the shell?

This week we saw two really important tools. Next week we’re going to take a sort of archealogical look at computer systems, first software then hardware to wrap up our overivew and exploration of what all these topics we’re going to cover in class are and how they relate.

4.10. Prepare for the next class

1. Review the notes

2. Reorganize a folder on your computer ( good candidate may be desktop or downloads folder), using only a terminal to make new directories, move files, check what's inside them, etc. Answer reflection questions (will be in notes) in a new file, terminal.md in your kwl repo.

3. Add a glossary to the site to define a term or cheatsheet entry to describe a command that we have used so far.

4. Examine a large project you have done or by finding an open source project on GitHub. Answer the reflection questions in software.md in your kwl repo. (will be in notes)

4.10.1. Terminal File moving reflection

Start with a file explorer open, but then try to close it and use only command line tools to explore and make your choices

1. Did this get easier toward the end?
1. Use the history to see which commands you used and how many times each, make a table below.
1. Did you have to look up how to do anything we had not done in class?
1. When do you think that using the terminal will be better than using your GUI file explorer?
1. What questions/challenges/ relfections do you have after this?
1. What kinds of things might you want to write a bash script for given what you know in bash so far? come up with 1-2 scenarios

4.10.2. Software Reflection

1. link to public repo if applicable or title of your project
1. What types of files are there that are not code?
1. What different types of code files are in the project? Do they serve different goals?
1. Is it all in one language or are there multiple?
1. Try to figure out (remember) how the project works. What types of things, without running the code can you look at at a high level?

4.11. More Practice

1. Try to do as many things as possible on the terminal for a whole week.

2. Make yourself a bash cheatsheet (and/or contribute to one on the course site)

3. Read through part 1 of the programmer's brain and try the exercises, especially in chapter 4.

4.12. Questions at the end of class

4.12.1. what makes a file be able to stage? In other words, what does “staging” actually mean?

We stage changes on a file by file basis, not the files themselves. Remember, git’s goal is version control to it wants to keep track of the different versions of your project. It does this by looking at what has changed. The .git record is like a playback of what you did to the files in the repository, not copies of them at each time. We’ll come back to this in more detail soon, too.

4.12.2. how do you make a branch on the GitHub site?

You can create a new branch in browser on GitHub by committing a change. You can switch to other branches from the main code tab of your repository, and then add more changes there if you want to, for example make changes to multiple files in a single pull request without downloading the code.

4.12.3. how do we write a bash script?

We will do this soon. The short answer is that it is just like programming in any other language.

4.12.4. Where can I learn more about the GitHub flow?

The GitHub Flow is a way to work on a git repository using branches. It has become pretty standard. The GitHub docs give a good overview of it.

4.12.5. How do we link a project we already have made to a new git repository

So far, I have only had you create repositories through some GitHub helpers provided by GitHub classroom. You can create a new repository from scratch using the plus sign menu on the top right of any page on GitHub. GitHub will give you the instructions to either start blank so that you can add files you already have locally or with a README and other starter files for a brand new project.

4.12.6. what can happen if I moved a file but I had another file pointing to the old address

mv can be used to rename, so by the same thing it will overwrite and it will do so without warning you. Manipulating files reqiures care.

4.12.7. how in depth will our bash scripting go in this class? clearly it can be used for a lot of different things

We will do some common most helpful things in class time. You can explore it more in depth outside of class time as a part of the “extended labs” or project you do for a B or an A.

4.12.8. How might you go about re-instantiating a repo? I.e. starting back from whatever the origin is

We used checkout for switching branches so far, but it can also be used to switch back to a different commit.

4.13. Resources

Bash Cheat Sheet

Alternative Shells

previous

3. How do I use git offline

next

5. Review and Abstraction