.. -*- coding: utf-8 -*- .. role:: sref(numref) .. role:: xref(numref) .. Copyright (C) 2019, Wolfgang Scherer, .. .. This file is part of Documentation Standard. .. .. Permission is granted to copy, distribute and/or modify this document .. under the terms of the GNU Free Documentation License, Version 1.3 .. or any later version published by the Free Software Foundation; .. with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. .. A copy of the license is included in the section entitled "GNU .. Free Documentation License". .. inline comments (with du_comment_role) .. role:: rem(comment) .. role:: html(raw) :format: html .. role:: shx(code) :language: sh .. rst-class:: xnarrow xmedium large xhuge xultra .. _`sec:Sphinx Doc`: ################################################## :rem:`|||:sec:|||`\ Sphinx Doc ################################################## .. >>CODD See `the components of a doctoral dissertation and their order `_ .. >>CODD Dedication .. >>CODD Epigraph .. >>CODD Abstract .. compound:: Sphinx Doc is a collection of scripts and templates to allow for simple setup, administration and output formatting of `reStructuredText`_ documents. The focus is on providing a simple standalone README article and a book with chapters made from single README files. It employs snippets(1) and the python package PyJsMo. .. \|:here:| .. >>CODD Introduction .. >>CODD Chapter ================================================== :rem:`|||:sec:|||`\ Documentation workflow ================================================== :numref:`fig:documentation workflow` shows the definitive state diagram for the documentation workflow. Some state transitions are triggered by a list of mandatory shell commands. For alternatives, e.g., .. math:: \left\{\begin{array}{l} \mbox{touch file}\\ \mbox{sda automodule module}\\ \mbox{sda chapter add 'Chapter title'}\\ \end{array}\right. at least one command or an equivalent must be excuted. **There is no exception to this requirement**. .. _`fig:documentation workflow`: .. uml:: _static/doc-workflow.puml :caption: Documentation workflow :scale: 70% ====================================================== :rem:`|||:sec:|||`\ Initialize documentation ====================================================== -------------------------------------------------- :rem:`||:sec:||`\ New article -------------------------------------------------- The command .. code-block:: sh sda chapter new 'Chapter title' is roughly equivalent to .. code-block:: sh snn --mode rst --key title --value 'Chapter title' README-chapter-title.txt -------------------------------------------------- :rem:`||:sec:||`\ New book -------------------------------------------------- - Make book directory - `cd` to book directory - Execute .. code-block:: sh sda init "Book Title" This creates an empty `README.txt` and a `doc` subdirectory for building the full version of a documentation book with chapters. ================================================== :rem:`|||:sec:|||`\ Documentation output formats ================================================== -------------------------------------------------- :rem:`||:sec:||`\ Article format -------------------------------------------------- .. code-block:: sh sda readme README.txt # => README.html sda readme --format pdf README.txt # => README.pdf -------------------------------------------------- :rem:`||:sec:||`\ Book format -------------------------------------------------- .. code-block:: sh sda make html # => doc/_build/html/ sda make latexpdf # => doc/_build/latex/ sda view html # xdg-open 'doc/_build/html/index.html' sda view pdf # xdg-open "$( echo doc/_build/latex/*.aux | sed 's,\.aux$,\.pdf,;1q' )" =========================================================== :rem:`|||:sec:|||`\ Update documentation template structure =========================================================== |:todo:| add to glossary .. |book directory| replace:: **book directory** #. The |book directory| must not have any uncommitted changes. #. The |book directory| must be synchronized with master repository. #. The book at the |book directory| should build without warnings. If there are warnings, record them to be able to distinguish between old warnings and new warnings. .. code-block:: sh sda make clean sda make html sda make pdf #. Update template framework (snippets) at the |book directory| .. code-block:: sh sda update This creates a new sub-directory :file:`BOOK-DIR-update` with a new documentation project using the title from the main :file:`README` (see :numref:`fig:sphinxdoc update template structure`). .. _`fig:sphinxdoc update template structure`: .. uml:: _static/sphinx-doc-update.sh-a0.puml :caption: SphinxDoc - update template structure :scale: 50% Any generated files in BOOK-DIR-update, that are not present in BOOK-DIR updated, are moved to the corresponding location. Files that are identical (ignoring copyright lines) are deleted. (see :numref:`fig:sphinxdoc compare updated files with original tree`). .. _`fig:sphinxdoc compare updated files with original tree`: .. uml:: _static/sphinx-doc-update.sh-a1.puml :caption: SphinxDoc - compare updated files with original tree :scale: 50% #. Files in BOOK-DIR-update that differ from files in BOOK-DIR must be merged manually with :kbd:`M-x ediff-directories` or :program:`kdiff3`\ (1). .. code-block:: elisp (let* ((project (file-name-base (directory-file-name default-directory))) (update (concat default-directory project "-update"))) (ediff-directories update default-directory "")) #. Run a fresh build to detect any errors. #. Commit changes using the phrase `template framework update` (don't forget adding new files). #. Push commits to master repository. #. Publish repository. #. Remove update directory. .. \|||||:here:||||| .. _`document administration`: ================================================== :rem:`|||:sec:|||`\ Document administration ================================================== Document administration is performed with :program:`sda`, a shortcut link to :program:`sphinx-doc-admin.sh`. .. uml:: _static/sphinx-doc-admin.sh-a0.puml :scale: 50% -------------------------------------------------- :rem:`||:sec:||`\ Automodule files -------------------------------------------------- Add automodule documentation files for python modules to the :file:`doc` directory. .. code-block:: sh sda automodule MODULE-OR-FILE ... .. uml:: _static/sphinx-doc-admin.sh-a2.puml :scale: 50% -------------------------------------------------- :rem:`||:sec:||`\ Chapters -------------------------------------------------- Latest versions of the documentation have **|:**\ chapter\ **:|** symbol tags, which can be easily found with :kbd:`M-x symbol-tag-grep-find` (usually defined as :kbd:`M-g`). Chapter administration entails a number of use cases: .. uml:: _static/sphinx-doc-admin.sh-u0.puml :scale: 50% .. code-block:: sh sda chapter SUB-COMMAND .. uml:: _static/sphinx-doc-admin.sh-a1.puml :scale: 50% ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ :rem:`|:sec:|`\ Adding a chapter ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The following steps are automatically performed with: .. code-block:: sh sda chapter add topic - Create a new :file:`README-topic.txt`: .. code-block:: sh snn --mode rst README-topic.txt - Add a new entry to the variable :makevar:`CHAPTERS` in :file:`doc/Makefile`: .. code-block:: make CHAPTERS += ../README-topic.txt - Add the chapter file base :file:`topic` to the :rst:dir:`toctree` directive in :file:`doc/index-contents.snip`: .. code-block:: rest .. toctree:: :maxdepth: 1 topic Add a rule to :file:`.hgignore`:: ^doc/topic\.rst\.auto$ Define a document reference in :file:`doc/doc_defs_standalone.inc`:: .. |chapter-topic| replace:: document :file:`README-topic` Define a chapter reference in :file:`doc/doc_defs_combined.inc`:: .. |chapter-topic| replace:: chapter :doc:`topic` -------------------------------------------------- :rem:`||:sec:||`\ View processed documents -------------------------------------------------- .. code-block:: sh sda view FORMAT-OR-FILE sda view html sda view pdf .. uml:: _static/sphinx-doc-admin.sh-a4.puml :scale: 50% -------------------------------------------------- :rem:`||:sec:||`\ Command handlers for backends -------------------------------------------------- .. uml:: _static/sphinx-doc-admin.sh-a3.puml :scale: 50% .. \|||||:here:||||| ================================================== :rem:`|||:sec:|||`\ Structural specification ================================================== - `reStructuredText`_ is a markup language specification in the class of wiki markup languages. (Examples for other markup languages are HTML, TeX/LaTex, NROFF). `reStructuredText`_ is the official Python documentation markup. - `UML`_ is used for documenting programs. It is created from textual descriptions with `PlantUML`_. See |chapter-uml| for details. - `Docutils`_ is the reference library for translating `reStructuredText`_ markup into other formats like HTML/PDF/EPUB. - The document generator `Sphinx`_ extends `Docutils`_ and the `reStructuredText`_ specification with roles and directives. There are also various extensions to integrate other markup specifications (e.g. `PlantUML`_, Pygments for highlighting). See |chapter-sphinx| for further details . The program :program:`sphinx-build`\ (1) generates HTML/PDF/EPUB documentation from `reStructuredText`_ documents and doc strings of python(1) modules. It is also used in the command :command:`sda readme` (:program:`sphinx-readme.sh`) to generate standalone HTML/PDF/EPUB from a single :file:`README` file. .. uml:: @startuml ' |:here:| hide circle hide members skinparam padding 1 !definelong GENERATORS +HTML() +PDF() +EPUB() !enddefinelong package "Specifications" as PSPEC { /' |:here:||:sec:| '/ class "UML" as UML class "UML Diagram" as UMLDIA class "reStructuredText" as RST UML <|.. UMLDIA : <> } package "Documents" as PDOC { /' |:sec:| '/ class "Python __doc__" as PYDOC class "README.txt" as README class "Documentation" as DOC PYDOC "*" --o DOC README "*" --o DOC } package "Generators" as PGEN { /' |:sec:| '/ object "PlantUML" as PUML { PNG() SVG() } show PUML members object "Docutils" as DUT { GENERATORS } show DUT members object "Sphinx" as SPH { +extension[] GENERATORS } show SPH members class "Generator" as GEN { generates() } show GEN methods DUT ..|> GEN SPH ..|> GEN DUT <|-- SPH : <> PUML <.. SPH : <> } package "Output" as POUT { /' |:sec:| '/ object "README.html" as RHTML object "index.html" as IHTML } ' |:here:||:sec:| Relations UMLDIA <|.. PUML : <> RST <|.. DUT : <> RST --o PYDOC RST --o README DOC <|.. IHTML README <|.. RHTML (README, RHTML) .. GEN (DOC, IHTML) .. GEN ' |:here:| @enduml .. \|||||:here:||||| -------------------------------------------------- :rem:`||:sec:||`\ README chapters -------------------------------------------------- .. uml:: @startuml skinparam padding 1 ' |:here:| definitions !define dir_top dir = . !define dir_doc dir = doc !define parts_eq parts = !define dir_top_parts dir_top, parts_eq !define dir_doc_parts dir_doc, parts_eq ' parts definitions !definelong R_PARTS dir_top_parts - top - title - contents - abstract - body - defs - bottom !enddefinelong !definelong I_PARTS dir_doc_parts - top - title - abstract - index-contents.snip - index-header.snip - defs - index-footer.snip - bottom !enddefinelong !definelong O_PARTS dir_doc_parts - top - overview-header.snip - body - defs - overview-footer.snip - bottom !enddefinelong !definelong C_PARTS dir_doc_parts - top - title - abstract - chapter-header.snip - body - defs - chapter-footer.snip - bottom !enddefinelong package "Combined output" as OUTC { /' |:here:| '/ object "**index.html**" as ih { dir = doc/_build/html } object "Document.pdf" as ip { dir = doc/_build/latex } } package "Single chapter output" as OUTS { /' |:sec:| '/ object "README.pdf" as RP { dir_top } object "**README.html**" as RH { dir_top } object "README-topic1.html" as RH1 { dir_top } object "README-topic2.html" as RH2 { dir_top } } package "Chapters" as IC { /' |:sec:| '/ object "**README.txt**" as R { R_PARTS } object "README-topic1.txt" as RT1 { R_PARTS } object "README-topic2.txt" as RT2 { R_PARTS } } package "Index" as PI { /' |:sec:| '/ object "**index.rst**" as i { I_PARTS } object "toctree" as TT { - overview - topic1 - topic2 } } package "Generated chapters" as PC { /' |:sec:| '/ object "overview.rst" as O { O_PARTS } object "topic1.rst" as t1 { C_PARTS } object "topic2.rst" as t2 { C_PARTS } } ' |:here:| Helper Relations ih -[hidden] ip ip -[hidden] RP RP -[hidden] RH RH -[hidden] RH1 RH1 -[hidden] RH2 ' |:here:| Single Chapter Output RH ..|> R : < sphinx-build\nsda readme RP ..|> R : < sphinx-build\nsda readme\n--format pdf RH1 ..|> RT1 : < sphinx-build\nsda readme RH2 ..|> RT2 : < sphinx-build\nsda readme ' |:sec:| Combined Documentation Chapters R <|.. i : snippets\nsnr --key doc_index > R <|.. O : snippets\nsnr --key doc_overview > RT1 <|.. t1 : snippets\nsnr --key doc_chapter > RT2 <|.. t2 : snippets\nsnr --key doc_chapter > ' |:sec:| Combined Documentation Output ih ..|> i : < sphinx-build\nmake html ip ..|> i : < sphinx-build\nmake latexpdf ' |:sec:| toctree references i o- TT TT .u.> O TT .u.> t1 TT .u.> t2 ' |:here:| @enduml .. \||||:here:|||| -------------------------------------------------- :rem:`||:sec:||`\ README snippets -------------------------------------------------- .. uml:: :scale: 70% @startuml skinparam padding 2 hide methods !define SNIP rem !define SKIP skip !define QUOTE quote !define REM rem !define INDENT indent !definelong NO_OUT_BLOCKS !enddefinelong !definelong OUT_BLOCKS !enddefinelong package "README snippet setup" as RSS { ' |:here:| class "rst_new" as RSNP { -- template aliases -- snip_quote : alias QUOTE / REM .. doc_index .. doc_index : block alias SNIP / SKIP not_doc_index : block alias SKIP / SNIP di_indent : alias INDENT / REM not_di_indent : alias REM / INDENT .. minimal .. minimal : block alias SNIP / SKIP not_minimal : block alias SKIP / SNIP } hide RIND circle hide RCHP circle hide RMIN circle class "README.txt (doc_index)" as RIND { doc_index = SNIP not_doc_index = SKIP di_indent = INDENT not_di_indent = REM minimal = SKIP not_minimal = SNIP snip_quote = QUOTE -- output block aliases -- .. doc type .. doc_standalone not_doc_standalone doc_index not_doc_index doc_overview not_doc_overview doc_chapter not_doc_chapter .. part .. part_top part_title part_contents part_abstract part_body part_defs part_bottom } class "README.txt (chapter)" as RCHP { doc_index = SKIP not_doc_index = SNIP di_indent = REM not_di_indent = INDENT minimal = SKIP not_minimal = SNIP snip_quote = QUOTE -- output block aliases -- .. doc type .. doc_standalone not_doc_standalone doc_index not_doc_index doc_overview not_doc_overview doc_chapter not_doc_chapter .. part .. part_top part_title part_contents part_abstract part_body part_defs part_bottom } class "README.txt (minimal)" as RMIN { doc_index = SKIP not_doc_index = SNIP di_indent = REM not_di_indent = INDENT minimal = SNIP not_minimal = SKIP snip_quote = REM -- output block aliases -- .. doc type .. .. part .. } ' |:here:| } RSNP <|.. RIND : snn ~--mode rst\n~--key doc_index > RSNP <|.. RCHP : snn ~--mode rst > RSNP <|.. RMIN : snn ~--mode rst\n~--key minmal > ' |:here:| @enduml =================================================================== :rem:`|||:sec:|||`\ How to properly move chapter files and sections =================================================================== ---------------------------------------------------------------- :rem:`||:sec:||`\ Rename chapter file in same document directory ---------------------------------------------------------------- .. uml:: @startuml left to right direction package "Document **doc**" as doc_old { object "**chapter-old**" as chp_old { } object "**chapter-new**" as chp_new { } chp_old --> chp_new : renamed to } @enduml - Move :file:`README-chapter-old.txt` to :file:`README-chapter-new.txt`. - | Execute :kbd:`M-x grep-find RET chapter-old RET`. | Rename info for |xchp-old| to info for |xchp-new|. - | Clean + compile |xdoc-new|. | Analyze error messages and correct errors. --------------------------------------------------------------- :rem:`||:sec:||`\ Move chapter file to other document directory --------------------------------------------------------------- .. uml:: @startuml left to right direction package "Document **doc-old**" as doc_old { object "**chapter-old**" as chp_old { } } package "Document **doc-new**" as doc_new { object "**chapter-new**" as chp_new { } } chp_old --> chp_new : moved to @enduml - Move :file:`README-chapter-old.txt` from |xdoc-old| to :file:`README-chapter-new.txt` in |xdoc-new|. - | Execute :kbd:`M-x grep-find RET chapter-old RET` in |xdoc-old|. | Keep document references from :file:`doc/doc_def_standalone.inc` in :file:`doc/doc_def.inc`. | Move and rename info for |xchp-old| from |xdoc-old| to |xdoc-new|. - | Clean + compile |xdoc-new|. | Analyze error messages and correct errors. | E.g. get missing substitutions/references/images/data from |xdoc-old|. - | Clean + compile |xdoc-old|. | Analyze error messages and correct errors. ------------------------------------------------------------------------------------- :rem:`||:sec:||`\ How to move a section from one chapter file to another chapter file ------------------------------------------------------------------------------------- .. uml:: @startuml left to right direction package "Document **doc-old**" as doc_old { package "Chapter **old**" as chp_old { object "Section **old**" as sec_old { } } } package "Document **doc-new**" as doc_new { package "Chapter **new**" as chp_new { object "Section **new**" as sec_new { } } } sec_old --> sec_new : moved to @enduml - | Remove |xsec-old| section from |xchp-old|. | Insert as |xsec-new| in |xchp-new|. - | Clean + compile |xdoc-new|. | Analyze error messages and correct errors. | E.g. get missing substitutions/references/images/data from |xdoc-old|. - | Clean + compile |xdoc-old|. | Analyze error messages and correct errors. ------------------------------------------------------- :rem:`||:sec:||`\ Traceability of section modifications ------------------------------------------------------- |:todo:| incorrect, needs to be updated Case-by-case analysis: 1. Move sections before chapter documents: - Section |xsec-old| moved from |xchp-old| to |xchp-new| in |xdoc-old| - |xchp-new| moved from |xdoc-old| to |xchp-new| in |xdoc-new| - |xchp-new| moved to |xchp-new-new| in |xdoc-new| Consequences: - |xdoc-old| no longer exists. It must be looked up in the document processing logs. - |xdoc-new| no longer exists. It must be looked up in the document processing logs. - The version history of the section modification resides in the old repositories (|xdoc-old|, :doc-new). 2. Move chapter documents before sections: - |xdoc-old| moved to |xdoc-old-new| - |xdoc-new| moved to |xdoc-new-new| - Section |xsec| moved from |xdoc-old-new| to |xdoc-new-new| Consequences: - The version history of the section modification resides in the new repositories (|xdoc-old-new|, :doc-new-new), which is better. .. |xsec| replace:: **section** .. |xsec-old| replace:: **section-old** .. |xsec-new| replace:: **section-new** .. |xchp-old| replace:: **chapter-old** .. |xchp-new| replace:: **chapter-new** .. |xchp-new-new| replace:: **chapter-new-new** .. |xdoc| replace:: **doc** .. |xdoc-old| replace:: **doc-old** .. |xdoc-new| replace:: **doc-new** .. |xdoc-old-new| replace:: **doc-old-new** .. |xdoc-new-new| replace:: **doc-new-new** ================================================== :rem:`|||:sec:|||`\ ReST section overlines ================================================== `manual` (`book`) class: +--------------------------+------------+----------+---------+------------+-----------+--------------+ | File | Book title | Chapter | Section | Subsection | Paragraph | Subparagraph | +==========================+============+==========+=========+============+===========+==============+ | README.txt | ``###`` | ``===`` | ``---`` | ``~~~`` | ``...`` | | +--------------------------+------------+----------+---------+------------+-----------+--------------+ | README-chapter-topic.txt | | ``###`` | ``===`` | ``---`` | ``~~~`` | ``...`` | +--------------------------+------------+----------+---------+------------+-----------+--------------+ `howto` (`article`) class: +--------------------------+---------------+---------+------------+-----------+--------------+ | File | Article title | Section | Subsection | Paragraph | Subparagraph | +==========================+===============+=========+============+===========+==============+ | README.txt | ``###`` | ``===`` | ``---`` | ``~~~`` | ``...`` | +--------------------------+---------------+---------+------------+-----------+--------------+ | README-chapter-topic.txt | ``###`` | ``===`` | ``---`` | ``~~~`` | ``...`` | +--------------------------+---------------+---------+------------+-----------+--------------+ .. \|||||:here:||||| .. >>CODD Conclusion .. >>CODD Appendix A .. \|:here:| .. >>CODD Notes .. ================================================== .. :rem:`|||:sec:|||`\ Footnotes .. ================================================== :html:`
` .. \[#] .. include:: doc_defs.inc .. include:: doc_defs_combined.inc .. include:: abbrev_defs.inc .. .. \||<-snap->|| doc_standalone .. include:: doc/doc_defs_secret.inc .. \||<-snap->|| doc_standalone .. \||<-snap->|| not_doc_standalone .. include:: doc_defs_secret.inc .. \||<-snap->|| not_doc_standalone .. _`Wolfgang Scherer`: wolfgang.scherer@gmx.de