CONTENT

1 MOTIVATION
2 c2m.sh
- 2.1 DESCRIPTION
- 2.2 USAGE
- 2.3 ISSUES
- 2.4 OPEN
- 2.5 REQUIRED
- 2.6 ADDITIONAL FILES
- 2.7 SEE ALSO
- 2.8 EXAMPLE PAGELIST
3 warnstats.sh
- 3.1 DESCRIPTION
- 3.2 USAGE
- 3.3 NOTES
- 3.4 ADDITIONAL FILES
- 3.5 EXAMPLE OUTPUT

`MOTIVATION`

The scripts where created mainly to ease the work of the doc team. Two tasks are supported:

Automate the work in case multiple content shall be "moved" from the DeveloperWiki to ReadTheDocs (c2m.sh)
Analyse warning messages of the doc build process, identify related modules/projects/rst-files and ease the debug process (warnstats.sh)

Both scripts can be found in the doc/tools directory contained in the doc repo. Please check also README.md and the header of both scripts for additional information.

Feel free to use them in your ONAP project to improve documentation ...

Of course ... your feedback is highly welcome.

`c2m.sh`

DESCRIPTION

c2m automates additional tasks required in case you want to export and convert a set of wiki pages. The export and first conversion to markdown is done by confluence2md, provided by viaboxx.

c2m processes a list of (to be exported) wiki pages, creates corresponding export directories, exports and converts pages (in various formats if required), opens an editor and cleans up afterwards. c2m checks also for problematic content in the export and creates a warning in case of detection.

USAGE

^{c2m.sh <your-page-list>}

^{Before executing copy the confluence2md-2.1-fat.jar file from Maven repository to your working directory otherwise it won't work}

ISSUES

markdown (md) output of confluence2md contains sometimes tags that are somehow "merged" with the topic headline; manual edit is required here

OPEN

confluence2md does not support all of the currently used confluence page types (structured-macros) - result for unsupported pages is "not satisfying"; enhancements (java) are required
workaround: copy the needed content to a new "blanc page" without using problematic macros and export it instead of the original page
opt: toc creation in root document in case you export a tree of documents to separate files

REQUIRED

pandoc
retext
confluence2md from https://repo1.maven.org/maven2/de/viaboxx/markdown/confluence2md/2.1/
java

ADDITIONAL FILES

example.pagelist: an example pagelist to demonstrate functionality

EXAMPLE PAGELIST

example pagelist (field descriptions below); it uses the delimiter "|" for the four fields per line
copy/paste page id and title from wiki; to get the wiki page_id you have to login to the wiki, open the page and choose e.g. the history
depth: use depth to follow down the child-pages hierarchy if required: -1=infinte, 0=no children, #=number of child-pages to follow
every hierarchy "0" entry will lead into the creation of a dedicated working directory where the page and child-pages are stored
for better readability you can add spaces to the list, but use "|" as a delimiter
lines starting with a # are filtered by c2m

^{# hierarchy | page_id | page_title | depth}

^{0          | 1018748 | ONAP Portal                     | 0}
^{1.1      | 1018759 | ONAP Portal for users           | 0}
^{1.2      | 1018762 | ONAP Portal for administrators | 0}
^{1.2.1    | 1018764 | Admins                          | 0}
^{1.2.2     | 1018811 | Users                           | 0}
^{1.2.3     | 1018821 | Portal Admins                   | 0}
^{1.2.4    | 1018826 | Application Onboarding          | 0}
^{1.2.5    | 1018832 | Widget Onboarding               | 0}
^{1.2.6     | 1018835 | Edit Functional Menu            | 0}
^{1.2.7     | 16004953 | Portal Microservices Onboarding | 0}

in case you want to export to only one single output page (that contains all child-pages of the above example) use:

^{0 | 1018748 | ONAP Portal | -1}

`warnstats.sh`

DESCRIPTION

warnstats helps to find the onap modules (projects) and rst-files which are responsible for the most warnings during the documentation build process. It requires a tox build logfile, parses it line by line, prints out some statistics and provides links to the local rst file, its html version, the related link to readthedocs and as well the doc8 test result for the rst.

"rst html web doc8_(00000)" links can be opened easily by using mouse-over/context menu functionality of bash. files will be opened in your browser and rst editor!

Number of warnings found in the logfile are shown as "nnnnn", e.g. "00129"

Number of doc8 warnings/errors in a rst file are shown as "doc8_(nnnnn)", e.g. "doc8_(00002)".

USAGE

^{warnstatst.sh <your-tox-build-logfile>}

NOTES

create the required tox logfile when you locally build the onap documentation:

^{tox | tee tox-build-logfile}

ADDITIONAL FILES

example-tox-build-log: an example tox logfile to demonstrate functionality (created links in the warnstats output may not work if you use this example)

EXAMPLE OUTPUT

^{(onapdocs) user@ubuntu:~/onapdocs/doc/tools$ ./warnstats.sh ../tox-build.log}

^{warnstats version 1.6.0 (2020-04-03)}
^{html build directory ..... /home/user/onapdocs/doc/docs/_build/html}
^{web base url ............. https://docs.onap.org/en/latest}
^{doc8 command ............. doc8 --verbose}
^{doc8 results directory ... /home/user/onapdocs/doc/tools/doc8_results}
^{tox logfile .............. ../tox-build.log}

^{00656 LINES WITH WARNING IN FILE '../tox-build.log'}

^{################################################################################}
^{~~~ MESSAGES LONG ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}
^{################################################################################}

^{00656 WARNINGS IN TOTAL WITH 169 UNIQUE MESSAGES}
^{00129 | document_isnt_included_in_any_toctree}
^{00056 | Title_underline_too_short_}
^{00053 | Could_not_lex_literal_block_as_json__Highlighting_skipped_}
^{00035 | Unexpected_indentation_}
^{00029 | Block_quote_ends_without_a_blank_line_unexpected_unindent_}
^{- - snip - - - - - - - - - - - - - -}
^{00001 | Citation_Log_Architecture_is_not_referenced_}
^{00001 | Citation_Eventlet_is_not_referenced_}
^{00001 | Citation_Django_Document_is_not_referenced_}
^{00001 | Citation_django_deploy_is_not_referenced_}
^{00001 | Anonymous_hyperlink_mismatch_7_references_but_0_targets_}

^{################################################################################}
^{~~~ MESSAGES SHORTENED (FOR SIMPLE GROUPING) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}
^{################################################################################}

^{00656 WARNINGS IN TOTAL WITH 46 UNIQUE MESSAGES}
^{00129 | document_isnt_in}
^{00094 | Could_not_lex_li}
^{00072 | Duplicate_explic}
^{00056 | Title_underline_}
^{00039 | Unknown_target_n}
^{- - snip - - - - - - - - - - - - - -}
^{00001 | Citation_Log_Arc}
^{00001 | Citation_Eventle}
^{00001 | Citation_Django_}
^{00001 | Citation_django_}
^{00001 | Anonymous_hyperl}

^{################################################################################}
^{~~~ MODULES ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}
^{################################################################################}

^{00656 WARNINGS IN TOTAL IN 37 MODULES}
^{00123 | dcaegen2.git}
^{00078 | ccsdk}
^{00043 | vnfrqts}
^{00043 | multicloud}
^{00039 | dmaap}
^{- - snip - - - - - - - - - - - - - -}
^{00001 | vid.git}
^{00001 | usecase-ui.git}
^{00001 | so}
^{00001 | logging-analytics}
^{00001 | docs_use-cases}

^{################################################################################}
^{~~~ MODULES WITH RSTFILES ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}
^{################################################################################}

^{00656 WARNINGS IN TOTAL IN APPROX. 241 RST FILES}
^{aaf | index.rst | rst html web doc8_(00008) | 00006}
^{aaf | README.rst | rst html web doc8_(00000) | 00001}
^{aaf | service.rst | rst html web doc8_(00140) | 00002}
^{aaf | usage.rst | rst html web doc8_(00007) | 00006}
^{aai | AAIRESTAPI_AMSTERDAM.rst | rst html web doc8_(00008) | 00001}
^{- - snip - - - - - - - - - - - - - -}
^{vnfrqts | VES_Registration_3_2.rst | rst html web doc8_(00076) | 00001}
^{vnfsdk | csar-validation.rst | rst html web doc8_(00010) | 00002}
^{vnfsdk | index.rst | rst html web doc8_(00003) | 00002}
^{vnfsdk | release-notes.rst | rst html web doc8_(00009) | 00003}
^{vnfsdk | VNFSDK-Marketplace-userguide-vendors.rst | rst html web doc8_(00002) | 00001}

^{################################################################################}
^{~~~ RSTFILES ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}
^{################################################################################}

^{00656 WARNINGS IN TOTAL IN APPROX. 241 RST FILES}
^{00030 | dmaap | Installation.rst | rst html web doc8_(00023)}
^{00019 | ccsdk | workflow.rst | rst html web doc8_(00002)}
^{00017 | dcaegen2.git | userguide.rst | rst html web doc8_(00051)}
^{00016 | dcaegen2.git | mod-onboardingapi.rst | rst html web doc8_(00006)}
^{00015 | appc.git | APPC_LCM_API_Guide.rst | rst html web doc8_(01121)}
^{- - snip - - - - - - - - - - - - - -}
^{00001 | aai | bulkApi.rst | rst html web doc8_(00011)}
^{00001 | aai | AAIRESTAPI_DUBLIN.rst | rst html web doc8_(00542)}
^{00001 | aai | AAIRESTAPI_CASABLANCA.rst | rst html web doc8_(00535)}
^{00001 | aai | AAIRESTAPI_AMSTERDAM.rst | rst html web doc8_(00008)}
^{00001 | aaf | README.rst | rst html web doc8_(00000)}

ONAP Wiki

DOC Tools

`MOTIVATION`

`c2m.sh`

DESCRIPTION

USAGE

ISSUES

OPEN

REQUIRED

ADDITIONAL FILES

SEE ALSO

EXAMPLE PAGELIST

`warnstats.sh`

DESCRIPTION

USAGE

NOTES

ADDITIONAL FILES

EXAMPLE OUTPUT