Commit 3c9a0d09 authored by Michal Berger's avatar Michal Berger Committed by Tomasz Zawadzki
Browse files

doc: Add "CI Tools" section 



This section is meant to describe tooling present in the SPDK repo
which is used to validate all the submitted patches. Also, add
first subpage describing use of shfmt.

Change-Id: Ib9d14c8eb28108d6dfdc27e140c008be070a16f5
Signed-off-by: default avatarMichal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2455


Community-CI: Mellanox Build Bot
Tested-by: default avatarSPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: default avatarJim Harris <james.r.harris@intel.com>
Reviewed-by: default avatarShuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
parent 3c2190c2

doc/Doxyfile

+2 −0
Original line number Diff line number Diff line
@@ -795,6 +795,7 @@ INPUT += \ 
                         misc.md \

                         driver_modules.md \

                         tools.md \

                         ci_tools.md \

                         performance_reports.md \



# All remaining pages are listed here in alphabetical order by filename.
@@ -834,6 +835,7 @@ INPUT += \ 
                         overview.md \

                         peer_2_peer.md \

                         porting.md \

                         shfmt.md \

                         spdkcli.md \

                         spdk_top.md \

                         ssd_internals.md \

doc/ci_tools.md

0 → 100644
+6 −0
Original line number Diff line number Diff line 
# CI Tools {#ci_tools}



Section describing tools used by CI to verify integrity of the submitted

patches ([status](https://ci.spdk.io)).



- @subpage shfmt

doc/index.md

+4 −0
Original line number Diff line number Diff line
@@ -32,6 +32,10 @@ 


@copydoc tools



# CI Tools



@copydoc ci_tools



# Performance Reports



@copydoc performance_reports

doc/shfmt.md

0 → 100644
+146 −0
Original line number Diff line number Diff line 
# shfmt {#shfmt}



# In this document {#shfmt_toc}



* @ref shfmt_overview

* @ref shfmt_usage

* @ref shfmt_installation

* @ref shfmt_examples



# Overview {#shfmt_overview}



The majority of tests (and scripts overall) in the SPDK repo are written

in Bash (with a quite significant emphasis on "Bashism"), thus a style

formatter, shfmt, was introduced to help keep the .sh code consistent

across the entire repo. For more details on the tool itself, please see

[shfmt](https://github.com/mvdan/sh).



# Usage {#shfmt_usage}



On the CI pool, the shfmt is run against all the updated .sh files that

have been committed but not merged yet. Additionally, shfmt will pick

all .sh present in the staging area when run locally from our pre-commit

hook (via check_format.sh). In case any style errors are detected, a

patch with needed changes is going to be generated and either build (CI)

or the commit will be aborted. Said patch can be then easily applied:



~~~{.sh}

# Run from the root of the SPDK repo

patch --merge -p0 <shfmt-3.1.0.patch

~~~



The name of the patch is derived from the version of shfmt that is

currently in use (3.1.0 is currently supported).



Please, see ./scripts/check_format.sh for all the arguments the shfmt

is run with. Additionally, @ref shfmt_examples has more details on how

each of the arguments behave.



# Installation {#shfmt_installation}



The shfmt can be easily installed via pkgdep.sh:



~~~{.sh}

./scripts/pkgdep.sh -d

~~~



This will install all the developers tools, including shfmt, on the

local system. The precompiled binary will be saved, by default, to

/opt/shfmt and then linked under /usr/bin. Both paths can be changed

by setting SHFMT_DIR and SHFMT_DIR_OUT in the environment. Example:



~~~{.sh}

SHFMT_DIR=/keep_the_binary_here \

SHFMT_DIR_OUT=/and_link_it_here \

  ./scripts/pkgdep.sh -d

~~~



# Examples {#shfmt_examples}



~~~{.sh}

#######################################

if foo=$(bar); then

  echo "$foo"

fi



exec "$foo" \

  --bar \

  --foo



# indent_style = tab



if foo=$(bar); then

        echo "$foo"

fi



exec foobar \

        --bar \

        --foo

######################################

if foo=$(bar); then

        echo "$foo" && \

        echo "$(bar)"

fi

# binary_next_line = true

if foo=$(bar); then

        echo "$foo" \

                && echo "$(bar)"

fi



# Note that each break line is also being indented:



if [[ -v foo ]] \

&& [[ -v bar ]] \

&& [[ -v foobar ]]; then

	echo "This is foo"

fi

# ->

if [[ -v foo ]] \

        && [[ -v bar ]] \

        && [[ -v foobar ]]; then

        echo "This is foo"

fi



# Currently, newlines are being escaped even if syntax-wise

# they are not needed, thus watch for the following:

if [[ -v foo

        && -v bar

        && -v foobar ]]; then

        echo "This is foo"

fi

#->

if [[ -v foo && -v \

        bar && -v \

        foobar ]]; then

        echo "This is foo"

fi

# This, unfortunately, also breaks the -bn behavior.

# (see https://github.com/mvdan/sh/issues/565) for details.

######################################

case "$FOO" in

        BAR)

        echo "$FOO" ;;

esac

# switch_case_indent = true

case "$FOO" in

        BAR)

                echo "$FOO"

                ;;

esac

######################################

exec {foo}>bar

:>foo

exec {bar}<foo

# -sr

exec {foo}> bar

: > foo

exec {bar}< foo

######################################

# miscellaneous, enforced by shfmt

(( no_spacing_at_the_beginning & ~and_no_spacing_at_the_end ))

: $(( no_spacing_at_the_beginning & ~and_no_spacing_at_the_end ))



# ->

((no_spacing_at_the_beginning & ~and_no_spacing_at_the_end))

: $((no_spacing_at_the_beginning & ~and_no_spacing_at_the_end))

~~~