From my previous post, sphinx appears to be just another static site
generator. But there’s much more to explore and do. Automation is where it’s at. A robust documentation pipeline is the goal of this delve into some of the powerful features in sphinx. Let’s go.
Code Documentation#
For these experiments I’m using code from a simple django-ninja CRUD project. I’ve documented the code and it will provide the necessary base for our testing.
I use the coc-pydocstring extension for coc.nvim in vim to generate “Google Docstring”. Full details of the plugin are available at the yaegassy / coc-pydocstring repository. The PEP 257 Docstring Conventions can be found on the python pep 257 webpage.
Getting the Project Setup#
For this post I’m doing a clean install. My basic setup is as follows:
# from my projects directory
# setup workspace
mkdir sphinx-next-level
cd sphinx-next-level
#setup virtual environment
python3.10 -m venv venv && source venv/bin/activate
source venv/bin/activate
python3 -m pip install --upgrade pip
# setup git
git init
# see note below
git add .gitignore
git commit -m"Add .gitignore in Initial Commit"
For a .gitignore for python/django/jupyter notebooks visit the toptotal website.
For this project I’m using a requirements.txt to manage the project dependencies. The key here is the dependencies. I’ll be adding them as the project develops. At the start it looks like this.
sphinx~=4.2.0
To install sphinx I run:
pip install -r requirements.txt
We know have access to the sphinx-quickbuild command.
sphinx-quickstart docs
A quick start interactive starts up. I choose to separate source and build directories in the root path.
- Add your project name
- Add your own name and email
- Project release 0.1.0
- Project language
Now is a good time to test.
cd docs
make html
No errors and we’re good to go. Let’s serve the pages and have a look.
cd build/html/
python http.server
All’s working. Now’s a good time to commit. (GTTC)
reStructuredText is one of the oldest and fully featured text markup standards. However markdown has grown in popularity and is used across many platforms. So I use markdown in Sphinx.
Hello .md Goodbye .rst#
I convert from .rst to markdown using rst-to-myst.
pip install rst-to-myst
rst2myst convert docs/**/*.rst # convert every file under docs
# It's possible to run the convert command with -R flag to
# replace the .rst file as opposed to creating .md files
# beside the original .rst files. But, caution.
I then delete the index.rst and edit conf.py. The myst_enable_extensions enable sphinx definition lists.
extensions = [
"myst_parser",
]
myst_enable_extensions = [
"deflist",
]
And install myst-parser.
pip install myst-parser
# test
cd docs/build/html/
python http.server
We are up and running and our files are in markdown.
GTTC.
Live Reload / Auto build#
Running and stoping the http.server over and over is tiresome.
Sphinx has a solution for that.
pip install sphinx-autobuild
This allows us to start the http server and let it run. When I make changes to the .md files it will automatically rebuild the site and refresh the browser. It’s similar to live reload in other apps. The server doesn’t update if you make changes to the .py files.
To start autobuild:
# from the root of your project
# command /inputDirectory /outputDirectory/
sphinx-autobuild docs/source/ docs/build/html/
Now when I modify the documentation the documentation site updates automatically.
I then remove the lines below from conf.py. These are used for the general index, the Python module index, and a standalone search page. I don’t need them for this project.
- {ref}`genindex`
- {ref}`modindex`
- {ref}`search`
Include a File#
To include the existing README.md of your project add the following to index.md. We use the include directive. Notice the opening and closing three back-tics.```
These open and close the directive. {“directive name”}
```{include} ../../README.md
:relative-images:
```
Add a Warning#
To add a warning, we use the warning directive.
```{warning}
This site is under heavy delvelopment.
```
For a list of directives the sphinx documentation pages are the place to go.
GTTC.
Are we there yet?#
To get a readout of the duration of your build add “sphinx.ext.duration” to conf.py extensions.
extensions = [
...
"sphinx.ext.duration",
]
Look and Feel#
There are a lot of great themes to choose from. The Sphinx Themes Gallery has a large selection. I’m torn between furo and documatt. It depends on the project.
Install the theme:
pip install furo
# or
pip install sphinx_documatt_theme
Activate in conf.py by changing the selected theme.
html_theme = "furo"
# or
html_theme = "sphinx_documatt_theme"
GTTC.
Adding Usage Section#
Create a usage.md file in the source folder.
# Usage
## Installation
Installation instructions to go here #TODO
pip install <libraryname>
If I build now, I will get a warning that usage.md is not in any toctree directive. Edit index.md.
```{toctree}
:caption: 'Contents:'
:maxdepth: 2
usage
```
Add Relative Reference Links#
To add a domain independent or relative link to another document use:
For more information read {doc}`usage`
In this example I added it to index.md
To add a relative reference to a specific heading in a document there is a required extension.
In conf.py add:
extensions = [
...
"sphinx.ext.autosectionlabel",
]
And in our index.md:
For more installation instructions read {ref}`Installation`
Note: The reference heading should be unique.
GTTC.
Integrating Jupyter Notebooks#
Add the nbsphnix extension to conf.py
extensions = [
...
"nbsphinx",
]
If you don’t have pandoc installed you will receive an error when you rebuild.
To install pandoc the full instructions are on the pandoc official website.
For me on my MacOS, I use:
brew install pandoc
GTTC.
Documenting Code Manually#
To manually include documentation for a particular piece of code we use the {eval-rst} directive in our markdown.
```{eval-rst}
.. function:: example.get_attendee_names(kind=None)
Return a list of attendees
:param kind: Optional "kind" argument
:type kind: list[str] or None
:rtype: list[str]
```
This outputs the following:
example.get_attendee_names(kind=None)
Return a list of optional ingredients
Parameters
kind (list_[str] or_ None) – Optional “kind” argument
Return type
list[str]
Automate Code Documentation Import#
To automatically detect and create the documentation page for our code we need to add sphinx.ext.autodoc to the extensions list in conf.py.
extensions = [
...
"sphinx.ext.autodoc",
]
We then add a reference use autoformat to our markdown. In my case I added
it to index.md.
```{eval-rst}
.. autofunction:: example.get_attendee_name
```
Every time the site documentation is built, sphinx reads the code docstring and includes it in the documentation. The documentation is automatically kept up to date with changes in the code documentation.
I document my code and autodoc semi-automatically does the rest. But there’s
a better way. Which we’ll see soon.
Linking to the Specific Code docstring#
I use the {py:func} directive in my markdown to create a link to the source code documentation of a particular function.
{py:func}`example.get_attendee_name`
The {py:func} directive generates a relative link to the specific function on the documentation page. There are similar directives for classes {py:class}, modules {py:mod}, and methods {py:meth}. For a full list of cross-referencing python objects visit the sphinx documentation website.
Ok. That’s a lot.
It’s a good time to commit. (GTTC)
In the next post, I want to start answering some of my more burning questions.
How do I manage google docstring code comments?
How do I generate the docs for a complete code base?
How to I handle todo items in my code base docstring?
How do I add some flavour to my docs?
#source
Document Your Scientific Project With Markdown, Sphinx, and Read the Docs | PyData Global 2021