doc: Begin documenting the directory structure

INPUT                  = ../include/spdk \

                         mainpage.txt \

                         directory_structure.txt \

                         ioat/index.txt \

                         nvme/index.txt \

                         nvme/async_completion.txt \

/*-

 *   BSD LICENSE

 *

 *   Copyright (c) Intel Corporation.

 *   All rights reserved.

 *

 *   Redistribution and use in source and binary forms, with or without

 *   modification, are permitted provided that the following conditions

 *   are met:

 *

 *     * Redistributions of source code must retain the above copyright

 *       notice, this list of conditions and the following disclaimer.

 *     * Redistributions in binary form must reproduce the above copyright

 *       notice, this list of conditions and the following disclaimer in

 *       the documentation and/or other materials provided with the

 *       distribution.

 *     * Neither the name of Intel Corporation nor the names of its

 *       contributors may be used to endorse or promote products derived

 *       from this software without specific prior written permission.

 *

 *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS

 *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT

 *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR

 *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT

 *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,

 *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT

 *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,

 *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY

 *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT

 *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE

 *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

 */

/**

\page directory_structure SPDK Directory Structure

SPDK is primarily a collection of C libraries intended to be consumed directly by

applications, but the repository also contains many examples and full-fledged applications.

This will provide a general overview of what is where in the repository.

\section dir_app Applications

The \c app top-level directory contains four applications:

 - \c app/iscsi_tgt: An iSCSI target

 - \c app/nvmf_tgt: An NVMe-oF target

 - \c app/iscsi_top: Informational tool (like \c top) that tracks activity in the

    iSCSI target.

 - \c app/trace: A tool for processing trace points output from the iSCSI and

    NVMe-oF targets.

The application binaries will be in their respective directories after compiling and all

can be run with no arguments to print out their command line arguments. For the iSCSI

and NVMe-oF targets, they both need a configuration file (-c option). Fully commented

examples of the configuration files live in the \c etc/spdk directory.

\section dir_build Build Collateral

The \c build directory contains all of the static libraries constructed during

the build process. The \c lib directory combined with the \c include/spdk

directory are the official outputs of an SPDK release, if it were to be packaged.

\section dir_doc Documentation

The \c doc top-level directory contains all of SPDK's documentation. API Documentation

is created using Doxygen directly from the code, but more general articles and longer

explanations reside in this directory, as well as the Doxygen config file.

To build the documentation, just type `make` within the doc directory.

\section dir_examples Examples

The \c examples top-level directory contains a set of examples intended to be used

for reference. These are different than the applications, which are doing a "real"

task that could reasonably be deployed. The examples are instead either heavily

contrived to demonstrate some facet of SPDK, or aren't considered complete enough

to warrant tagging them as a full blown SPDK application.

This is a great place to learn about how SPDK works. In particular, check out

\c examples/nvme/hello_world.

\section dir_include Include

The \c include directory is where all of the header files are located. The public API

is all placed in the \c spdk subdirectory of \c include and we highly

recommend that applications set their include path to the top level \c include

directory and include the headers by prefixing \c spdk/ like this:

\code #include "spdk/nvme.h" \endcode

Most of the headers here correspond with a library in the \c lib directory and will be

covered in that section. There are a few headers that stand alone, however. They are:

 - \c assert.h

 - \c barrier.h

 - \c endian.h

 - \c fd.h

 - \c mmio.h

 - \c queue.h and \c queue_extras.h

 - \c string.h

There is also an \c spdk_internal directory that contains header files widely included

by libraries within SPDK, but that are not part of the public API and would not be

installed on a user's system.

\section dir_lib Libraries

The \c lib directory contains the real heart of SPDK. Each component is a C library with

its own directory under \c lib.

\subsection dir_bdev Block Device Abstraction Layer

The \c bdev directory contains a block device abstraction layer that is currently used

within the iSCSI and NVMe-oF targets. The public interface is \c include/spdk/bdev.h.

This library lacks clearly defined responsibilities as of this writing and instead does a

number of

things:

 - Translates from a common \c block protocol to specific protocols like NVMe or to system

  calls like libaio. There are currently three block device backend modules that can be

  plugged in - libaio, SPDK NVMe, CephRBD, and a RAM-based backend called malloc.

 - Provides a mechanism for composing virtual block devices from physical devices (to do

  RAID and the like).

 - Handles some memory allocation for data buffers.

This layer also could be made to do I/O queueing or splitting in a general way. We're open

to design ideas and discussion here.

\subsection dir_conf Configuration File Parser

The \c conf directory contains configuration file parser. The public header

is \c include/spdk/conf.h. The configuration file format is kind of like INI,

except that the directives are are "Name Value" instead of "Name = Value". This is

the configuration format for both the iSCSI and NVMe-oF targets.

... Lots more libraries that need to be described ...

\section dir_mk Makefile Fragments

The \c mk directory contains a number of shared Makefile fragments used in the build system.

\section dir_scripts Scripts

The \c scripts directory contains convenient scripts for a number of operations. The two most

important are \c check_format.sh, which will use astyle and pep8 to check C, C++, and Python

coding style against our defined conventions, and \c setup.sh which binds and unbinds devices

from kernel drivers.

\section dir_tests Tests

The \c test directory contains all of the tests for SPDK's components and the subdirectories mirror

the structure of the entire repository. The tests are a mixture of unit tests and functional tests.

*/

into userspace and operating in a polled mode instead of relying on interrupts,

which avoids kernel context switches and eliminates interrupt handling overhead.

\section general General Information

 - \ref directory_structure

\section modules Modules

- \ref nvme