8. How do programmers communicate about code?#
8.1. Wrap up#
git status
On branch examplebranch
You have unmerged paths.
(fix conflicts and run "git commit")
(use "git merge --abort" to abort the merge)
Unmerged paths:
(use "git add <file>..." to mark resolution)
both modified: important_classes.py
Untracked files:
(use "git add <file>..." to include in what will be committed)
greeting.md
no changes added to commit (use "git add" and/or "git commit -a")
cat important_classes.py \
>
new code that interates bug fix and new feature
git add important_classes.py
git status
On branch examplebranch
All conflicts fixed but you are still merging.
(use "git commit" to conclude merge)
Changes to be committed:
modified: important_classes.py
Untracked files:
(use "git add <file>..." to include in what will be committed)
greeting.md
git commit -m 'resolve conflict with bug fix and feature'
[examplebranch 12bbbad] resolve conflict with bug fix and feature
git status
On branch examplebranch
Untracked files:
(use "git add <file>..." to include in what will be committed)
greeting.md
nothing added to commit but untracked files present (use "git add" to track)
git checkout main
Switched to branch 'main'
Your branch is ahead of 'origin/main' by 2 commits.
(use "git push" to publish your local commits)
cd ..
git status
fatal: not a git repository (or any of the parent directories): .git
8.2. Why Documentation#
Today we will talk about documentation, there are several reasons this is important:
using official documentation is the best way to get better at the tools
understanding how documentation is designed and built will help you use it better
writing and maintaining documentation is really important part of working on a team
documentation building tools are a type of developer tool (and these are generally good software design)
In particular documentation tools are really good examples of:
pattern matching
modularity
automation
building
8.3. conceptual topics#
By the end of today’s class you will be able to:
describe different types of documentation
find different information in a code repo
generate documentation as html
ignore content from a repo
create a repo locally and push to GitHub
practical skills
8.4. What is documentation#
from ethnography of docuemtnation data science
8.4.1. Why is documentation so important?#
we should probably spend more time on it
8.5. So, how do we do it?#
linux kernel uses sphinx and here is why and how it works
8.6. Jupyterbook#
Jupyterbook wraps sphinx and uses markdown instead of restructured text. We’re goign to use this.
jupyter-book create tiny-book
===============================================================================
Your book template can be found at
tiny-book/
===============================================================================
You can make it with any name:
jupyter-book create example
===============================================================================
Your book template can be found at
example/
===============================================================================
ls
Each one makes a directory
example tiny-book
github-in-class-brownsarahm-1
cd tiny-book/
ls -a
. intro.md notebooks.ipynb
.. logo.png references.bib
_config.yml markdown-notebooks.md requirements.txt
_toc.yml markdown.md
8.6.1. starting a git repo locally#
git init .
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint: git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint: git branch -m <name>
Initialized empty Git repository in /Users/brownsarahm/Documents/inclass/systems/tiny-book/.git/
Here we are faced with a social aspect of computing that is also a good reminder about how git actually works
8.6.2. Retiring racist language#
Historically the default branch was called master.
derived from a master/slave analogy which is not even how git works, but was adopted terminology from other projects
literally the person who chose the names “master” and “origin” regrets that choice the name main is a more accurate and not harmful term and the current convention.
we’ll change our default branch to main
git branch -m main
git status
On branch main
No commits yet
Untracked files:
(use "git add <file>..." to include in what will be committed)
_config.yml
_toc.yml
intro.md
logo.png
markdown-notebooks.md
markdown.md
notebooks.ipynb
references.bib
requirements.txt
nothing added to commit but untracked files present (use "git add" to track)
and we will commit the template.
git add .
git commit -m 'init jupyterbook'
[main (root-commit) 267e9ae] init jupyterbook
9 files changed, 342 insertions(+)
create mode 100644 _config.yml
create mode 100644 _toc.yml
create mode 100644 intro.md
create mode 100644 logo.png
create mode 100644 markdown-notebooks.md
create mode 100644 markdown.md
create mode 100644 notebooks.ipynb
create mode 100644 references.bib
create mode 100644 requirements.txt
git commit --help
8.6.3. Structure of a Jupyter book#
ls
_config.yml logo.png notebooks.ipynb
_toc.yml markdown-notebooks.md references.bib
intro.md markdown.md requirements.txt
A jupyter book has two required files (_config.yml and _toc.yml)
Some files contain the content. The other files are optional, but common. Requirements.txt is the format for pip to install python depndencies. There are different standards in other languages for how
the extention (.yml) is yaml, which stands for “YAML Ain’t Markup Language”. It consists of key, value pairs and is deigned to be a human-friendly way to encode data for use in any programming language.
Further Reading
bibliographies are generated with bibtex which takes structured information from the references in a bibtex file with help from sphinxcontrib-bibtex
For general reference, reference managers like zotero and mendeley can track all of your sources and output the references in bibtex format that you can use anywhere or sync with tools like MS Word or Google Docs.
cat _config.yml
# Book settings
# Learn more at https://jupyterbook.org/customize/config.html
title: My sample book
author: The Jupyter Book Community
logo: logo.png
# Force re-execution of notebooks on each build.
# See https://jupyterbook.org/content/execute.html
execute:
execute_notebooks: force
# Define the name of the latex output file for PDF builds
latex:
latex_documents:
targetname: book.tex
# Add a bibtex file so that we can create citations
bibtex_bibfiles:
- references.bib
# Information about where the book exists on the web
repository:
url: https://github.com/executablebooks/jupyter-book # Online location of your book
path_to_book: docs # Optional path to your book, relative to the repository root
branch: master # Which branch of the repository should be used when creating links (optional)
# Add GitHub buttons to your book
# See https://jupyterbook.org/customize/config.html#add-a-link-to-your-repository
html:
use_issues_button: true
use_repository_button: true
The configuration file, tells it basic iformation about the book, it provides all of the settings that jupyterbook and sphinx need to render the content as whatever output format we want.
The table of contents file describe how to put the other files in order.
cat _toc.yml
# Table of contents
# Learn more at https://jupyterbook.org/customize/toc.html
format: jb-book
root: intro
chapters:
- file: markdown
- file: notebooks
- file: markdown-notebooks
jupyter-book build .
Running Jupyter-Book v0.13.1
Source Folder: /Users/brownsarahm/Documents/inclass/systems/tiny-book
Config Path: /Users/brownsarahm/Documents/inclass/systems/tiny-book/_config.yml
Output Path: /Users/brownsarahm/Documents/inclass/systems/tiny-book/_build/html
Running Sphinx v4.5.0
making output directory... done
[etoc] Changing master_doc to 'intro'
checking for /Users/brownsarahm/Documents/inclass/systems/tiny-book/references.bib in bibtex cache... not found
parsing bibtex file /Users/brownsarahm/Documents/inclass/systems/tiny-book/references.bib... parsed 5 entries
myst v0.15.2: MdParserConfig(renderer='sphinx', commonmark_only=False, enable_extensions=['colon_fence', 'dollarmath', 'linkify', 'substitution', 'tasklist'], dmath_allow_labels=True, dmath_allow_space=True, dmath_allow_digits=True, dmath_double_inline=False, update_mathjax=True, mathjax_classes='tex2jax_process|mathjax_process|math|output_area', disable_syntax=[], url_schemes=['mailto', 'http', 'https'], heading_anchors=None, heading_slug_func=None, html_meta=[], footnote_transition=True, substitutions=[], sub_delimiters=['{', '}'], words_per_minute=200)
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 4 source files that are out of date
updating environment: [new config] 4 added, 0 changed, 0 removed
Executing: markdown-notebooks in: /Users/brownsarahm/Documents/inclass/systems/tiny-book
Executing: notebooks in: /Users/brownsarahm/Documents/inclass/systems/tiny-book
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] notebooks
generating indices... genindex done
writing additional pages... search done
copying static files... done
copying extra files... done
dumping search index in English (code: en)... done
dumping object inventory... done
build succeeded.
The HTML pages are in _build/html.
[etoc] missing index.html written as redirect to 'intro.html'
===============================================================================
Finished generating HTML for book.
Your book's HTML pages are here:
_build/html/
You can look at your book by opening this file in a browser:
_build/html/index.html
Or paste this line directly into your browser bar:
file:///Users/brownsarahm/Documents/inclass/systems/tiny-book/_build/html/index.html
===============================================================================
Try it yourself
Which files created by the template are not included in the rendered output? How could you tell?
ls
_build logo.png references.bib
_config.yml markdown-notebooks.md requirements.txt
_toc.yml markdown.md
intro.md notebooks.ipynb
ls _build/
html jupyter_execute
ls _build/html/
_sources index.html notebooks.html
_sphinx_design_static intro.html objects.inv
_static markdown-notebooks.html search.html
genindex.html markdown.html searchindex.js
We didn’t have to write any html and we got a responsive site!
If you wanted to change the styling with sphinx you can use built in
themes which tell sphinx to put different
files in the _static folder when it builds your site, but you don’t have to change any of your content! If you like working on front end things (which is great! it’s just not alwasy the goal) you can even
build your own theme that can work with sphinx.
8.7. Ignoring Built files#
git status
On branch main
Untracked files:
(use "git add <file>..." to include in what will be committed)
_build/
nothing added to commit but untracked files present (use "git add" to track)
We do not want to keep track of changes for the built files since they are generated from the source files. It’s redundant and makes it less clear where someone should update content.
Git helps us with this with the .gitignore
echo "_build" >> .gitignore
git status
On branch main
Untracked files:
(use "git add <file>..." to include in what will be committed)
.gitignore
nothing added to commit but untracked files present (use "git add" to track)
now that’s the only new file as far as git is concerned, so we will track this,
git add .
git commit -m "ignore build"
[main be4ae7c] ignore build
1 file changed, 1 insertion(+)
create mode 100644 .gitignore
8.8. How do I push a repo that I made locally to GitHub#
For today, create an empty github repo shared with me.
More generally, you can create a repo
That default page for an empty repo if you do not initiate it with any files will give you the instructions for what remote to add.
git remote add origin https://github.com/introcompsys/tiny-book-brownsarahm-1.git
We can see what it did
git remote
origin
git status
On branch main
nothing to commit, working tree clean
git push -u origin main
Enumerating objects: 14, done.
Counting objects: 100% (14/14), done.
Delta compression using up to 8 threads
Compressing objects: 100% (12/12), done.
Writing objects: 100% (14/14), 16.47 KiB | 5.49 MiB/s, done.
Total 14 (delta 1), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (1/1), done.
To https://github.com/introcompsys/tiny-book-brownsarahm-1.git
* [new branch] main -> main
branch 'main' set up to track 'origin/main'.
git push --help
ls
_build logo.png references.bib
_config.yml markdown-notebooks.md requirements.txt
_toc.yml markdown.md
intro.md notebooks.ipynb
Glossary Links for community badge
See the issue I created and more generally, any issue that I tag community is a chance
8.9. Review today’s class#
Make your kwl repo into a jupyter book. Review the notes carefully for what files are required to make
jupyter-book buildrun. Ignore your build directory.Add
docs.mdto your KWL repo and explain the most important things to know about documentation in your own words using other programming concepts you have learned so far. Include in a markdown (same as HTML<!-- comment -->) comment the list of CSC courses you have taken for context while we give you feedback.Learn about the documentation ecosystem in a language that you know besides Python. In
docs.mdinclude a summary of your findings and compare and contrast it to jupyter book/sphinx. Include a bibliography of the sources you used. You can use this generator for informal sources and google scholar for formal sources.
8.10. Prepare for Next Class#
Try exploring your a repo manually and bring more questions
Make sure that you have submitted and gotten feedback on your plan for the course. (Feb 7 prepare for class)
8.11. More Practice#
Make your kwl repo into a jupyter book. Review the notes carefully for what files are required to make
jupyter-book buildrun. Ignore your build directory.Add one of the following features to your kwl repo:
a glossary both the terms and linknig to their use
Learn about the documentation ecosystem in another language that you know. In
docs.mdinclude a summary of your findings and compare and contrast it to jupyter book/sphinx. Include a bibliography of the sources you used. You can use this generator for informal sources and google scholar for formal sources.
8.12. Experience Report Evidence#
Link to your tinybook repo.
8.13. Questions After Today’s Class#
8.13.1. How does the jupyter-book automatically translate html?#
It runs a parser on the markdown and adds those into templates based on the settings.
Markdown is designed to be translated to HTML. Working with templates is like at the
file level, but similar to using string formatting in programming(C++, Python,
javascript, Rust).
more in depth exploration here is a good explore badge
8.13.2. what is a .yml file?#
Be sure to read the questions after every class, this has been asked and answered. I also added yml to the glossary.
8.13.3. Is it a good habit to pretty much git status every other line just to keep track of where I am at?#
Until you are confident at keeping track of it in your head, yes. I will continue to do it a little bit more than I do when I’m working in class, but even when I am working, I use it a lot. It is also good to get confirmation that git is where you think it is.
8.13.4. Is there a cheatsheet for jupyter-book or just the documentation?#
The documentation’s RESOURCES section has a cheatsheet for Myst-Markdown and a configuration reference which are what I use a lot.
8.13.5. What is the difference between Jupyter Book, Lab, and Notebook?#
Jupyter Notebook is a single stream of computational analysis. Jupyter Lab is a more IDE like interface for doing compuational analyses. Both are part of project jupyter and on GitHub
Jupyter book is for publishing book like documents as websites and to other forms designed to be compatible with jupyter notebooks, but is a part of a separate executable books project. It is specialized for cases where there is computation in the code. See their gallery for examples. I use it for CSC310 that has code and plots in the notes
8.13.6. Is documentation a requirement when creating a programming language?#
If you want people to be able to use your language then pretty much. That said, not all languages are open source and have easily accessbile, official documentation. Python, Rust, Asteroid (developed here at URI), Ruby and Stan among many others do. On the other hand the C++ language does not have any official documentation. There is a C++ standard that you can purchase, but no official documentation.
8.13.7. Can jupyterbook convert other languages to html or only python?#
It doesn’t convert plain python to html, it can run jupyter notebooks. Jupyter notebooks can run many different kernels. Jupyter-book is an opinionated distribution of sphinx which can also be used to document other languages like C++
8.13.8. this seemed like a tedious way of creating a repository is there maybe a shorter way?#
The only thing that was for making the repository was the git init . step and then to link to GitHub,
git remote add.
8.13.9. Should I use Jupyterbook for creating websites showcasing some of my programming projects?#
You totally can. You could also use sphinx which is more customizable. A sphinx gallery might be of interest.
Build Badge
You could build a profile website using a tool like this or other jamstack tool that showcases projects for a build badge.
8.13.10. What happends to files in .gitignore once the repo is pushed?#
Absolutely nothing. They exist in your working directory but they are not in the .git directory. These files are not tracked by git locally and not backed up by being copied to a server.
8.13.11. does jupyterbook have a hosting service or should we use something else like github pages?#
Jupyterbook only provides the builder, but they do provide instructions for hosting with mulitple services.
8.13.12. What if later I do not want to ignore some files anymore?#
The .gitignore file is a plain text file (try cat .gitignore). You can edit it anytime.