.. -*- coding: utf-8 -*- .. role:: sref(numref) .. role:: xref(numref) .. Copyright (C) 2018, Wolfgang Scherer, .. .. This file is part of Documentation Standard. .. .. Permission is granted to copy, distribute and/or modify this document .. under the terms of the GNU Free Documentation License, Version 1.3 .. or any later version published by the Free Software Foundation; .. with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. .. A copy of the license is included in the section entitled "GNU .. Free Documentation License". .. inline comments (with du_comment_role) .. role:: rem(comment) .. role:: html(raw) :format: html .. role:: shx(code) :language: sh .. rst-class:: xnarrow xmedium large xhuge xultra .. _`sec:Relevance of Documentation`: ################################################## :rem:`|||:sec:|||`\ Relevance of Documentation ################################################## .. >>CODD See `the components of a doctoral dissertation and their order `_ .. >>CODD Dedication .. >>CODD Epigraph .. epigraph:: `A map is not the territory it represents`_, but, if correct, it has a similar structure to the territory, which accounts for its usefulness. If the map could be ideally correct, it would include, in a reduced scale, the map of the map; the map of the map, of the map; and so on, endlessly, a fact first noticed by Royce. -- Alfred Korzybski, `Science and Sanity`_, `Chapter IV`_, p. 58. .. _`A map is not the territory it represents`: https://en.wikipedia.org/wiki/Map%E2%80%93territory_relation .. _`Chapter IV`: http://esgs.free.fr/uk/art/sands-ch04.pdf .. >>CODD Abstract .. compound:: .. \|:here:| The relevance of documentation, :math:`R_{doc}` deteriorates, whenever the source code is changed, but the documentation is not updated. The rate of deterioriation for a given piece of documentation increases with - the distance to the source code, :math:`D_{source} \ge 0` - the inavailability of a documentation editor, :math:`I_{edit} \ge 0` - the necessary effort for synchronization with the master document, :math:`E_{sync} \ge 0` - the inertia of the human components :math:`H_{inertia} \ge 1` involved. The |lrd|, |LRdoc| is therefore: .. math:: :label: eq_lrd_likeliness_relevance_documentation & L_{R_{doc}} = \left( \left( \frac{1 + ( {D_{source}} + {I_{edit}} + {E_{sync}} + {H_{inertia}} )}{2}\right)\cdot{H_{inertia}} \right)^{-1}\\ .. \|:here:| .. >>CODD Introduction .. _`sec:doc-relevance:Introduction`: ================================================== :rem:`|||:sec:|||`\ Introduction ================================================== The relevance of documentation is a measure of how similar the structure of the documentation is to the structure of the source code. This is closely connected to the `map–territory relation`_ of `General semantics`_. A thorough understanding of this principle really helps manage the expectations of what documentation of source code is, is not and can be. The fact, that documentation can never be accurate in the sense of a bijective mapping, does not mean it can not be helpful -- even essential -- for a human brain to construct an appropriate model. .. _`fig:they might be having sushi`: .. figctr:: I think they might be having sushi .. only:: html .. image:: _static/i-think-they-might-be-having-sushi.png :align: center :scale: 70% .. \|:todo:| here is an option to wrap figures in LaTeX https://stackoverflow.com/questions/16463051/how-to-create-floating-figures-in-restructuredtext-sphinx/30849577#30849577 .. only:: latex .. image:: _static/i-think-they-might-be-having-sushi.png :align: center :scale: 50% A major problem of software documentation arises from the fact, that the documented source code is often not available to the reader. Imagine, being given the following instructions for obtaining an item from a certain location: .. hlist:: :columns: 1 - go straight - go left - look at the sign, that says "Menu" - order the second item - bring it back However, you are only given access up to a fence with a locked gate, where you can see the straight part of the road and the left turn, but you cannot see the menu. At the locked gate, there is only a sign with yesterday's menu. Now you are supposed to predict what the second item on today's menu will be without being able to examine the actual menu (see :xref:`fig:they might be having sushi`). It is pretty obvious, that this cannot be reliably done, especially if the menu is changing daily to whatever the chef feels like. This describes the situation developers often find themselves in when working with proprietary software. But even when source code access is available -- since they are incidentally the developer, who works on it -- they are still expected to produce documentation for people that cannot understand the code; i.e., these people -- who may just as well be future versions of the developer himself -- choose to stay at the fence and refuse to walk through the gate. So developers may write things like: "And then look around in the source code, to find out what the real state of affairs is at any given moment." However, they cannot expect any particular reader to make the effort. See the `Sphinx`_ and `Doxygen`_ document generators in |chapter-sphinx| for excellent examples, how the entire source is included for cross-referencing\ [#]_. So, how much interest can you muster for the utterly boring description of a treasure hunt for an unspecified treasure in a country you don't know and don't have the means to ever get to? For me, that is one trivially self-evident answer to questions like "`What's with the aversion to documentation in the industry?`_". The rapid deterioration of relevance for documentation during the development phase being another. However, having no documentation at all is not a viable option and there are certainly ways to reduce the effort and increase the relevance of documentation. .. _`What's with the aversion to documentation in the industry?`: https://softwareengineering.stackexchange.com/a/202171/258871 .. >>CODD Chapter .. _`sec:Source and documentation management`: ========================================================== :rem:`|||:sec:|||`\ Source and documentation management ========================================================== Derived from the above equation, the best |lrd| is achieved when all deterioration factors, |Dsource|, |Iedit|, |Esync|, |Hinertia| are at a minimum, which makes :math:`L_{R_{doc}} = 1`. To discuss the |lrd|, it is necessary to define a model, which explains the parameters in their respective context (see :xref:`fig:Source and documentation management`). .. _`fig:Source and documentation management`: .. uml:: _static/source-doc-management.puml :caption: Source and documentation management .. note:: The required editor for a change is just one of a possible multitude of appropriate editors. It is not necessarily the preferred editor of a specific programmer (see :xref:`fig:Guaranteed and preferred editors`). .. _`fig:Guaranteed and preferred editors`: .. uml:: :caption: Guaranteed and preferred editors @startuml skinparam padding 1 /' |:here:| '/ class "Source\nEditor" as SEditor << (E,aqua) >> { } object "ed(1)" as ED object "vi(1)" as VI object "notepad" as NOTEPAD object "nano(1)" as NANO object "**emacs(1)**" as EMACS SEditor <|.. ED SEditor <|.. VI : **<>** SEditor <|.. NOTEPAD : **<>** SEditor <|.. NANO SEditor <|.. EMACS : **<>** @enduml =================================================== :rem:`|||:sec:|||`\ Minimum Distance to Source Code =================================================== A distance to source code, |Dsource| of |0| corresponds to the principle "**the source is the documentation**", i.e., there is no additional documentation for the source code. Substituting |Dsource| by |0| in equation :eq:`eq_lrd_likeliness_relevance_documentation` results in: .. math:: :label: eq_lrd_d_source_zero & \left( \left( \frac{1 + ( {I_{edit}} + {E_{sync}} + {H_{inertia}} )}{2}\right)^{H_{inertia}} \right)^{-1}\\ In this case, the source editor and the documentation editor are necessarily the same and therefore |Iedit| can be eliminated from equation :eq:`eq_lrd_d_source_zero`: .. math:: :label: eq_lrd_ds0_i_edit_zero & \left( \left( \frac{1 + ( {E_{sync}} + {H_{inertia}} )}{2}\right)^{H_{inertia}} \right)^{-1}\\ & L_{R_{doc}} = ( 1 + ( {D_{source}} + {E_{sync}} )^{H_{inertia}} )^{-1}\\ When the source code changes are made to the master source file, |Esync| is also |0| and equation :eq:`eq_lrd_ds0_i_edit_zero` is reduced to: .. math:: :label: eq_lrd_ds0_ie0_e_sync_zero & \left( \left( \frac{1 + {H_{inertia}}}{2}\right)^{H_{inertia}} \right)^{-1}\\ Without automatic documentation generators that have to be triggered manually, human inertia |Hinertia| is |1| and equation :eq:`eq_lrd_ds0_ie0_e_sync_zero` is reduced to |1|: .. math:: :label: eq_lrd_minimum & \left( \left( \frac{1 + 1}{2}\right)^{1} \right)^{-1} = 1\\ So it seems, that the only time, documentation is guaranteed to be entirely relevant, is if there is none. In all other cases, documentation and source strictly abide by the properties of the `map–territory relation`_\ [#]_. However, there are tools like grep(1), ctags(1), etags(1), cscope(1), even `Sphinx`_ and `doxygen`_, also (reluctantly) IDEs like Eclipse, which extract meta-information from undocumented source code to provide hyper linking facilities, which is in itself an extremely useful part of any documentation. Other Generators produce Nassi-Shneiderman diagrams (flowcharts, activity diagrams), dependency graphs, etc. Therefore the principle "**the source is the documentation**" is actually not so blunt or ridiculous as it may sound at first. However, since automatically generated documentation requires a compilation step that may require manual triggering, |Hinertia| may quickly become a signifanct factor. ================================================== :rem:`|||:sec:|||`\ Inertia ================================================== In the eternal sunshine of the spotless mind, each relevant source modification triggers a documentation update (see :xref:`fig:Sunny day documentation update`). .. _`fig:Sunny day documentation update`: .. uml:: :caption: Sunny day documentation update :scale: 50% @startuml skinparam padding 1 /' |:here:| '/ state "Editing Sources in the Sun" as SM { state "Source\nediting" as Editing state "Documentation\nupdate" as DocUpdate [*] --> Editing Editing -> DocUpdate : relevant\nsource\nchange DocUpdate --> Editing : updated Editing -> [*] : finished } /' |:here:| '/ @enduml But this is only true, when no human components are involved in the process of deciding, whether a source change is relevant for a documentation update. And this can only be the case when no manually created documentation parts have to be maintained. An example for this is the initial documentation produced by generators like `Sphinx`_ and `Doxygen`_. However, there is usually a more or less complex update decision process involved, which potentially results in increases of |Iedit| and human inertia |Hinertia| (see :xref:`fig:Rainy day documentation update`). .. _`fig:Rainy day documentation update`: .. uml:: :caption: Rainy day documentation update :scale: 50% @startuml skinparam padding 1 !define I_EDIT **Iedit** !define H_INERTIA **Hinertia** /' |:here:| '/ state "Editing Sources on a Rainy Day" as SM { state "Editing\nsource" as Editing ' state "Source and\nDocumentation\nsync'ed" as Synced state "Documentation\nupdate" as DocUpdate state "Update\ndecision" as Deciding state "Increment\nInertia" as IncInertia IncInertia : H_INERTIA += rand() state "Increment\nInavailability" as IncInavail IncInavail : I_EDIT += rand() [*] --> Editing Editing -> Deciding : source\nchanged Deciding --> Editing : not required Deciding --> IncInavail : not possible IncInavail --> Editing Deciding --> IncInertia : no update IncInertia --> Editing Deciding --> DocUpdate : do update DocUpdate --> Editing : updated Editing -> [*] : finished } /' |:here:| '/ @enduml The decision process in state `Update Decision` looks somewhat like the activity diagram in :xref:`fig:Documentation update decision`. Feel free to add an infinite number of excuses for not updating the documentation or postponing the update indefinitely. .. _`fig:Documentation update decision`: .. uml:: :caption: Documentation update decision @startuml skinparam padding 2 /' |:here:| '/ partition "Update Decision (Procrastination in the 9th Circle of Hell)" { start :res = undecided; repeat if (manual upate\n//not// required?) then (yes) :res = not required; elseif (deadline is near and\nedit takes too long?) then (yes) :res = no update; elseif (do you just //not//\nfeel like it?) then (yes) :res = no update; elseif (documentation editor\nis available?) then (yes) if (do you know\nhow to use it?) then (yes) :res = do update; else (no) :res = no update; endif elseif (editor is\ninstallable?) then (yes) if (you feel like\ninstalling?) then (yes) :install editor; else (no) :res = no update; endif else (no) :res = not possible; endif repeat while (res == undecided?) is (yes) if (do you no longer feel like it?) then (yes) :res = no update; endif -> return res; stop floating note right res is one of * not required * not possible * no update * do update end note } /' |:here:| '/ @enduml ============================================================== :rem:`|||:sec:|||`\ Single Editor for Source and Documentation ============================================================== When the documentation editor is the same as the source editor, the decisions about using or installing a separate documentation editor are removed. Therefore the result "**not possible**" is no longer returned (see :xref:`fig:Documentation Update Decision With Single Editor`). .. _`fig:Documentation Update Decision With Single Editor`: .. uml:: :caption: Documentation Update Decision With Single Editor @startuml skinparam padding 2 /' |:here:| '/ partition "Documentation Update Decision With Single Editor (The Dog Ate My Homework)" { start :res = undecided; repeat if (no manual upate\nrequired) then (yes) :res = not required; elseif (deadline is near\nedit takes too long) then (yes) :res = no update; elseif (do not feel like it) then (yes) :res = no update; else (no) :res = do update; endif repeat while (res == undecided) is (yes) if (no longer feel like it) then (yes) :res = no update; endif -> res; stop floating note right res is one of * not required * no update * do update end note } /' |:here:| '/ @enduml Consequently, the state machine for editing sources is also reduced. The state "**Increment Inavailability**" is removed and therefore |Iedit| is no longer incremented (see :xref:`fig:Editing with single editor`). .. _`fig:Editing with single editor`: .. uml:: :caption: Editing with single editor @startuml skinparam padding 1 /' |:here:| '/ !define H_INERTIA **Hinertia** state "Editing with Single Editor" as SM { state "Source\nediting" as Editing state "Documentation\nupdate" as DocUpdate state "Update\ndecision" as Deciding state "Increment\nInertia" as IncInertia IncInertia : H_INERTIA += rand() [*] --> Editing Editing -> Deciding : source\nchanged Deciding --> Editing : not required Deciding --> IncInertia : no update IncInertia --> Editing Deciding --> DocUpdate : do update DocUpdate --> Editing : updated Editing -> [*] : finished } /' |:here:| '/ @enduml Therefore |Iedit| is always |0| and equation :eq:`eq_lrd_likeliness_relevance_documentation` is reduced to: .. math:: :label: eq_lrd_i_edit_zero & L_{R_{doc}} = \left( \left( \frac{1 + ( {D_{source}} + {E_{sync}} + {H_{inertia}} )}{2}\right)\cdot{H_{inertia}} \right)^{-1}\\ .. note:: Please, note, that in addition to a documentation editor, there may be separate documentation generators required to produce some form of final documentation (e.g., SVG, PNG, JPEG, HTML, PDF). However, the lack of these does not contribute to the deterioration of relevance, as long as the documentation source is updated. If it is found, that the update of a piece of documentation without a visual representation created by the documentation generator is not possible, this decision may have to be revisited. ================================================== :rem:`|||:sec:|||`\ Synchronization ================================================== When there is a single file on just one computer, there is obviously no synchronization necessary and therefore :math:`E_{sync} = 0`. Sometimes it is better to copy a project directory and work on the copy so that others are not disturbed by the intermediate states of development. That is called a `branch` or `working on a branch`. The original project file is the `master`. As long as nothing leaves the branch directory, the documentation is not affected by the effort necessary for synchronizing the branch with the master. It is only, when e.g. a binary version from the branch is delivered to a production system, that the master documentation becomes less relevant until the branch with all documentation changes is synchronized (`merged`) with the master. This is the situation when :math:`E_{sync}` becomes :math:`> 0`. Human forgetfulness and general sloppiness are major factor for discrepancies, where branches are left to rot on flash drives and, newer versions are overwritten by older versions, etc. This is why the process is nowadays highly automated by distributed version control systems (DVC). DVCs are quite good, but sometimes you don't want to check in changes, that need some final touch. In that case, you are back to copy or remote copy (scp(1), rsync(1)). So all other stages are still useful: - backup file - diff(1), patch(1), diff3(1) - scp(1), rsync(1) - VCS - DVCS -------------------------------------------------- :rem:`||:sec:||`\ qs-gen-sync.pl (.sync.rc) -------------------------------------------------- .. role:: shx(code) :language: sh There are two aliases for `backup` and `restore`: bsy send stuff to remote host :shx:`./sync.sh --backup` rsy get stuff from remote host :shx:`./sync.sh --restore` Generate initial `.sync.rc`: .. code-block:: sh isy -n -------------------------------------------------- :rem:`||:sec:||`\ diff3 -------------------------------------------------- |:todo:| finish description of diff3(1) :: diff3 doc/index.rst README.txt doc/overview.rst ================================================== :rem:`|||:sec:|||`\ Battling Human Inertia ================================================== The only way to defeat human inertia, |Hinertia|, is automation. See also :xref:`fig:Geeks and repetitive tasks` |:todo:| finish description of scripting sh(1), make(1) - make/scripts are memory banks for knowledge .. _`fig:Geeks and repetitive tasks`: .. figctr:: Geeks and repetitive tasks .. image:: _static/alternate-view-of-automation.png Source: `Another way to think about geeks and repetitive tasks - Jon Udell`_ .. _`Another way to think about geeks and repetitive tasks - Jon Udell`: https://blog.jonudell.net/2012/01/09/another-way-to-think-about-geeks-and-repetitive-tasks/ .. .. image:: _static/geeks-and-repetitive-tasks.png Source: `Bruno Oliveira`_ .. _`Bruno Oliveira`: https://plus.google.com/102451193315916178828/posts/MGxauXypb1Y .. _`sec:doc-relevance - Integrated Development Environment`: ---------------------------------------------------- :rem:`||:sec:||`\ Integrated Development Environment ---------------------------------------------------- An `IDE`_ strives to be an *integrated* solution for cross-referencing, build automation, generated documentation. In that sense, emacs(1) can also be classified as an `IDE`_. `Eclipse`_ is ridiculously hard to extend out of the box with plug-ins written in Java. E.g., in a forum question from 2016 about `how to extend the default Java editor`_, somebody just wants to add a few keywords to the Java editor and gets links to a `PDF slideshow about JDT`_ and to `Xtext`_. Neither are anywhere near trivial. .. _`Xtext`: https://eclipse.org/Xtext/index.html .. _`PDF Slideshow about JDT`: https://www.eclipse.org/eclipse/platform-text/eclipseCon/talk.pdf .. _`How to extend the default Java editor`: https://www.eclipse.org/forums/index.php/t/1075624/ `EASE`_ does provide integration of script engines into `Eclipse`_ giving access to the workbench. This engine may be able to handle couple of keywords, if it can be added to `Eclipse`_ extension points (hooks), which needs to be researched |:todo:|. So in 2017 eclipse announces another `convoluted overstructured object oriented integration`_, which still does not reach easily into an editing buffer (see also `Internal and External Editors`_). .. _`convoluted overstructured object oriented integration`: https://www.eclipse.org/community/eclipse_newsletter/2017/october/article3.php .. _`Internal and External Editors`: https://help.eclipse.org/mars/index.jsp?topic=%2Forg.eclipse.platform.doc.isv%2Freference%2Fextension-points%2Forg_eclipse_ui_editors.html See also https://stackoverflow.com/questions/26912785/eclipse-jdt-syntax-highlighting-of-constants from 2014, which is still unanswered. The answers in https://stackoverflow.com/questions/13802131/in-an-eclipse-plugin-how-can-i-programmatically-highlight-lines-of-codes-in-the, give a variety of strategies to follow, but none of the answers gives a step-by-step example. Here is another failed attempt https://www.eclipse.org/forums/index.php/t/489221/ for adding some highlighting to the `Eclipse`_ editor. All in all `Eclipse`_ has still not reached the efficient state of the art provided by emacs(1) for decades. emacs(1) can be extended ridiculously simple with a multitude of tools glued together by Elisp. As discussed in the article `Using Automation to Assist – not replace – Manual Testing`_, automating entire workflows is very much ineffective (see :xref:`Using Automation to assist`). .. _`Using Automation to assist`: .. figctr:: Using Automation to assist .. image:: _static/reality-of-automation.png Source: `Using Automation to Assist – not replace – Manual Testing`_ .. _`Using Automation to Assist – not replace – Manual Testing`: https://smartbear.de/blog/test-and-monitor/using-automation-to-assist-not-replace-manual-test/?l=ua In that context, a tightly integrated `IDE`_ -- like `Eclipse`_ or Visual Studio -- is the ineffective attempt to provide an entirely automated workflow in a one-size-fits-all manner. In contrast, the loosely integrated approach with emacs(1) and tools provides a framework for task automation which can adapt to a variety of workflows. A monolithic `IDE`_ usually has a plethora of cryptic configuration settings to provide some sort of rudimentary flexibility (customization), whereas the tool based `IDE`_ allows full modification of each automation step down to the code level. .. \|:here:| .. >>CODD Conclusion .. >>CODD Appendix A .. \|:here:| .. >>CODD Notes .. ================================================== .. :rem:`|||:sec:|||`\ Footnotes .. ================================================== :html:`
` .. [#] `Sphinx`_ even includes the source code for the documentation itself. However, the `map–territory relation`_ still holds, since there is no description of the process that transforms the documentation source code into the final output. .. [#] Considering that the source code of a program is itself a map of an executable, requiring a processor (interpreter) and possibly a compiler to produce another representation, there really is no simple escape from the `map–territory relation`_. .. include:: doc_defs.inc .. include:: doc_defs_combined.inc .. include:: abbrev_defs.inc .. .. \||<-snap->|| doc_standalone .. include:: doc/doc_defs_secret.inc .. \||<-snap->|| doc_standalone .. \||<-snap->|| not_doc_standalone .. include:: doc_defs_secret.inc .. \||<-snap->|| not_doc_standalone .. _`Wolfgang Scherer`: wolfgang.scherer@gmx.de