Merge tag 'docs-4.10' of git://git.lwn.net/linux

Pull documentation update from Jonathan Corbet:
 "These are the documentation changes for 4.10.

  It's another busy cycle for the docs tree, as the sphinx conversion
  continues. Highlights include:

   - Further work on PDF output, which remains a bit of a pain but
     should be more solid now.

   - Five more DocBook template files converted to Sphinx. Only 27 to
     go... Lots of plain-text files have also been converted and
     integrated.

   - Images in binary formats have been replaced with more
     source-friendly versions.

   - Various bits of organizational work, including the renaming of
     various files discussed at the kernel summit.

   - New documentation for the device_link mechanism.

  ... and, of course, lots of typo fixes and small updates"

* tag 'docs-4.10' of git://git.lwn.net/linux: (193 commits)
  dma-buf: Extract dma-buf.rst
  Update Documentation/00-INDEX
  docs: 00-INDEX: document directories/files with no docs
  docs: 00-INDEX: remove non-existing entries
  docs: 00-INDEX: add missing entries for documentation files/dirs
  docs: 00-INDEX: consolidate process/ and admin-guide/ description
  scripts: add a script to check if Documentation/00-INDEX is sane
  Docs: change sh -> awk in REPORTING-BUGS
  Documentation/core-api/device_link: Add initial documentation
  core-api: remove an unexpected unident
  ppc/idle: Add documentation for powersave=off
  Doc: Correct typo, "Introdution" => "Introduction"
  Documentation/atomic_ops.txt: convert to ReST markup
  Documentation/local_ops.txt: convert to ReST markup
  Documentation/assoc_array.txt: convert to ReST markup
  docs-rst: parse-headers.pl: cleanup the documentation
  docs-rst: fix media cleandocs target
  docs-rst: media/Makefile: reorganize the rules
  docs-rst: media: build SVG from graphviz files
  docs-rst: replace bayer.png by a SVG image
  ...

1 files changed, 497 insertions, 0 deletions

diff --git a/Documentation/process/2.Process.rst b/Documentation/process/2.Process.rst
new file mode 100644
index 000000000000..ce5561bb3f8e
--- /dev/null
+++ b/Documentation/process/2.Process.rst
@@ -0,0 +1,497 @@
+.. _development_process:
+
+How the development process works
+=================================
+
+Linux kernel development in the early 1990's was a pretty loose affair,
+with relatively small numbers of users and developers involved.  With a
+user base in the millions and with some 2,000 developers involved over the
+course of one year, the kernel has since had to evolve a number of
+processes to keep development happening smoothly.  A solid understanding of
+how the process works is required in order to be an effective part of it.
+
+The big picture
+---------------
+
+The kernel developers use a loosely time-based release process, with a new
+major kernel release happening every two or three months.  The recent
+release history looks like this:
+
+	======  =================
+	2.6.38	March 14, 2011
+	2.6.37	January 4, 2011
+	2.6.36	October 20, 2010
+	2.6.35	August 1, 2010
+	2.6.34	May 15, 2010
+	2.6.33	February 24, 2010
+	======  =================
+
+Every 2.6.x release is a major kernel release with new features, internal
+API changes, and more.  A typical 2.6 release can contain nearly 10,000
+changesets with changes to several hundred thousand lines of code.  2.6 is
+thus the leading edge of Linux kernel development; the kernel uses a
+rolling development model which is continually integrating major changes.
+
+A relatively straightforward discipline is followed with regard to the
+merging of patches for each release.  At the beginning of each development
+cycle, the "merge window" is said to be open.  At that time, code which is
+deemed to be sufficiently stable (and which is accepted by the development
+community) is merged into the mainline kernel.  The bulk of changes for a
+new development cycle (and all of the major changes) will be merged during
+this time, at a rate approaching 1,000 changes ("patches," or "changesets")
+per day.
+
+(As an aside, it is worth noting that the changes integrated during the
+merge window do not come out of thin air; they have been collected, tested,
+and staged ahead of time.  How that process works will be described in
+detail later on).
+
+The merge window lasts for approximately two weeks.  At the end of this
+time, Linus Torvalds will declare that the window is closed and release the
+first of the "rc" kernels.  For the kernel which is destined to be 2.6.40,
+for example, the release which happens at the end of the merge window will
+be called 2.6.40-rc1.  The -rc1 release is the signal that the time to
+merge new features has passed, and that the time to stabilize the next
+kernel has begun.
+
+Over the next six to ten weeks, only patches which fix problems should be
+submitted to the mainline.  On occasion a more significant change will be
+allowed, but such occasions are rare; developers who try to merge new
+features outside of the merge window tend to get an unfriendly reception.
+As a general rule, if you miss the merge window for a given feature, the
+best thing to do is to wait for the next development cycle.  (An occasional
+exception is made for drivers for previously-unsupported hardware; if they
+touch no in-tree code, they cannot cause regressions and should be safe to
+add at any time).
+
+As fixes make their way into the mainline, the patch rate will slow over
+time.  Linus releases new -rc kernels about once a week; a normal series
+will get up to somewhere between -rc6 and -rc9 before the kernel is
+considered to be sufficiently stable and the final 2.6.x release is made.
+At that point the whole process starts over again.
+
+As an example, here is how the 2.6.38 development cycle went (all dates in
+2011):
+
+	==============  ===============================
+	January 4	2.6.37 stable release
+	January 18	2.6.38-rc1, merge window closes
+	January 21	2.6.38-rc2
+	February 1	2.6.38-rc3
+	February 7	2.6.38-rc4
+	February 15	2.6.38-rc5
+	February 21	2.6.38-rc6
+	March 1		2.6.38-rc7
+	March 7		2.6.38-rc8
+	March 14	2.6.38 stable release
+	==============  ===============================
+
+How do the developers decide when to close the development cycle and create
+the stable release?  The most significant metric used is the list of
+regressions from previous releases.  No bugs are welcome, but those which
+break systems which worked in the past are considered to be especially
+serious.  For this reason, patches which cause regressions are looked upon
+unfavorably and are quite likely to be reverted during the stabilization
+period.
+
+The developers' goal is to fix all known regressions before the stable
+release is made.  In the real world, this kind of perfection is hard to
+achieve; there are just too many variables in a project of this size.
+There comes a point where delaying the final release just makes the problem
+worse; the pile of changes waiting for the next merge window will grow
+larger, creating even more regressions the next time around.  So most 2.6.x
+kernels go out with a handful of known regressions though, hopefully, none
+of them are serious.
+
+Once a stable release is made, its ongoing maintenance is passed off to the
+"stable team," currently consisting of Greg Kroah-Hartman.  The stable team
+will release occasional updates to the stable release using the 2.6.x.y
+numbering scheme.  To be considered for an update release, a patch must (1)
+fix a significant bug, and (2) already be merged into the mainline for the
+next development kernel.  Kernels will typically receive stable updates for
+a little more than one development cycle past their initial release.  So,
+for example, the 2.6.36 kernel's history looked like:
+
+	==============  ===============================
+	October 10	2.6.36 stable release
+	November 22	2.6.36.1
+	December 9	2.6.36.2
+	January 7	2.6.36.3
+	February 17	2.6.36.4
+	==============  ===============================
+
+2.6.36.4 was the final stable update for the 2.6.36 release.
+
+Some kernels are designated "long term" kernels; they will receive support
+for a longer period.  As of this writing, the current long term kernels
+and their maintainers are:
+
+	======  ======================  ===========================
+	2.6.27	Willy Tarreau		(Deep-frozen stable kernel)
+	2.6.32	Greg Kroah-Hartman
+	2.6.35	Andi Kleen		(Embedded flag kernel)
+	======  ======================  ===========================
+
+The selection of a kernel for long-term support is purely a matter of a
+maintainer having the need and the time to maintain that release.  There
+are no known plans for long-term support for any specific upcoming
+release.
+
+
+The lifecycle of a patch
+------------------------
+
+Patches do not go directly from the developer's keyboard into the mainline
+kernel.  There is, instead, a somewhat involved (if somewhat informal)
+process designed to ensure that each patch is reviewed for quality and that
+each patch implements a change which is desirable to have in the mainline.
+This process can happen quickly for minor fixes, or, in the case of large
+and controversial changes, go on for years.  Much developer frustration
+comes from a lack of understanding of this process or from attempts to
+circumvent it.
+
+In the hopes of reducing that frustration, this document will describe how
+a patch gets into the kernel.  What follows below is an introduction which
+describes the process in a somewhat idealized way.  A much more detailed
+treatment will come in later sections.
+
+The stages that a patch goes through are, generally:
+
+ - Design.  This is where the real requirements for the patch - and the way
+   those requirements will be met - are laid out.  Design work is often
+   done without involving the community, but it is better to do this work
+   in the open if at all possible; it can save a lot of time redesigning
+   things later.
+
+ - Early review.  Patches are posted to the relevant mailing list, and
+   developers on that list reply with any comments they may have.  This
+   process should turn up any major problems with a patch if all goes
+   well.
+
+ - Wider review.  When the patch is getting close to ready for mainline
+   inclusion, it should be accepted by a relevant subsystem maintainer -
+   though this acceptance is not a guarantee that the patch will make it
+   all the way to the mainline.  The patch will show up in the maintainer's
+   subsystem tree and into the -next trees (described below).  When the
+   process works, this step leads to more extensive review of the patch and
+   the discovery of any problems resulting from the integration of this
+   patch with work being done by others.
+
+-  Please note that most maintainers also have day jobs, so merging
+   your patch may not be their highest priority.  If your patch is
+   getting feedback about changes that are needed, you should either
+   make those changes or justify why they should not be made.  If your
+   patch has no review complaints but is not being merged by its
+   appropriate subsystem or driver maintainer, you should be persistent
+   in updating the patch to the current kernel so that it applies cleanly
+   and keep sending it for review and merging.
+
+ - Merging into the mainline.  Eventually, a successful patch will be
+   merged into the mainline repository managed by Linus Torvalds.  More
+   comments and/or problems may surface at this time; it is important that
+   the developer be responsive to these and fix any issues which arise.
+
+ - Stable release.  The number of users potentially affected by the patch
+   is now large, so, once again, new problems may arise.
+
+ - Long-term maintenance.  While it is certainly possible for a developer
+   to forget about code after merging it, that sort of behavior tends to
+   leave a poor impression in the development community.  Merging code
+   eliminates some of the maintenance burden, in that others will fix
+   problems caused by API changes.  But the original developer should
+   continue to take responsibility for the code if it is to remain useful
+   in the longer term.
+
+One of the largest mistakes made by kernel developers (or their employers)
+is to try to cut the process down to a single "merging into the mainline"
+step.  This approach invariably leads to frustration for everybody
+involved.
+
+How patches get into the Kernel
+-------------------------------
+
+There is exactly one person who can merge patches into the mainline kernel
+repository: Linus Torvalds.  But, of the over 9,500 patches which went
+into the 2.6.38 kernel, only 112 (around 1.3%) were directly chosen by Linus
+himself.  The kernel project has long since grown to a size where no single
+developer could possibly inspect and select every patch unassisted.  The
+way the kernel developers have addressed this growth is through the use of
+a lieutenant system built around a chain of trust.
+
+The kernel code base is logically broken down into a set of subsystems:
+networking, specific architecture support, memory management, video
+devices, etc.  Most subsystems have a designated maintainer, a developer
+who has overall responsibility for the code within that subsystem.  These
+subsystem maintainers are the gatekeepers (in a loose way) for the portion
+of the kernel they manage; they are the ones who will (usually) accept a
+patch for inclusion into the mainline kernel.
+
+Subsystem maintainers each manage their own version of the kernel source
+tree, usually (but certainly not always) using the git source management
+tool.  Tools like git (and related tools like quilt or mercurial) allow
+maintainers to track a list of patches, including authorship information
+and other metadata.  At any given time, the maintainer can identify which
+patches in his or her repository are not found in the mainline.
+
+When the merge window opens, top-level maintainers will ask Linus to "pull"
+the patches they have selected for merging from their repositories.  If
+Linus agrees, the stream of patches will flow up into his repository,
+becoming part of the mainline kernel.  The amount of attention that Linus
+pays to specific patches received in a pull operation varies.  It is clear
+that, sometimes, he looks quite closely.  But, as a general rule, Linus
+trusts the subsystem maintainers to not send bad patches upstream.
+
+Subsystem maintainers, in turn, can pull patches from other maintainers.
+For example, the networking tree is built from patches which accumulated
+first in trees dedicated to network device drivers, wireless networking,
+etc.  This chain of repositories can be arbitrarily long, though it rarely
+exceeds two or three links.  Since each maintainer in the chain trusts
+those managing lower-level trees, this process is known as the "chain of
+trust."
+
+Clearly, in a system like this, getting patches into the kernel depends on
+finding the right maintainer.  Sending patches directly to Linus is not
+normally the right way to go.
+
+
+Next trees
+----------
+
+The chain of subsystem trees guides the flow of patches into the kernel,
+but it also raises an interesting question: what if somebody wants to look
+at all of the patches which are being prepared for the next merge window?
+Developers will be interested in what other changes are pending to see
+whether there are any conflicts to worry about; a patch which changes a
+core kernel function prototype, for example, will conflict with any other
+patches which use the older form of that function.  Reviewers and testers
+want access to the changes in their integrated form before all of those
+changes land in the mainline kernel.  One could pull changes from all of
+the interesting subsystem trees, but that would be a big and error-prone
+job.
+
+The answer comes in the form of -next trees, where subsystem trees are
+collected for testing and review.  The older of these trees, maintained by
+Andrew Morton, is called "-mm" (for memory management, which is how it got
+started).  The -mm tree integrates patches from a long list of subsystem
+trees; it also has some patches aimed at helping with debugging.
+
+Beyond that, -mm contains a significant collection of patches which have
+been selected by Andrew directly.  These patches may have been posted on a
+mailing list, or they may apply to a part of the kernel for which there is
+no designated subsystem tree.  As a result, -mm operates as a sort of
+subsystem tree of last resort; if there is no other obvious path for a
+patch into the mainline, it is likely to end up in -mm.  Miscellaneous
+patches which accumulate in -mm will eventually either be forwarded on to
+an appropriate subsystem tree or be sent directly to Linus.  In a typical
+development cycle, approximately 5-10% of the patches going into the
+mainline get there via -mm.
+
+The current -mm patch is available in the "mmotm" (-mm of the moment)
+directory at:
+
+	http://www.ozlabs.org/~akpm/mmotm/
+
+Use of the MMOTM tree is likely to be a frustrating experience, though;
+there is a definite chance that it will not even compile.
+
+The primary tree for next-cycle patch merging is linux-next, maintained by
+Stephen Rothwell.  The linux-next tree is, by design, a snapshot of what
+the mainline is expected to look like after the next merge window closes.
+Linux-next trees are announced on the linux-kernel and linux-next mailing
+lists when they are assembled; they can be downloaded from:
+
+	http://www.kernel.org/pub/linux/kernel/next/
+
+Linux-next has become an integral part of the kernel development process;
+all patches merged during a given merge window should really have found
+their way into linux-next some time before the merge window opens.
+
+
+Staging trees
+-------------
+
+The kernel source tree contains the drivers/staging/ directory, where
+many sub-directories for drivers or filesystems that are on their way to
+being added to the kernel tree live.  They remain in drivers/staging while
+they still need more work; once complete, they can be moved into the
+kernel proper.  This is a way to keep track of drivers that aren't
+up to Linux kernel coding or quality standards, but people may want to use
+them and track development.
+
+Greg Kroah-Hartman currently maintains the staging tree.  Drivers that
+still need work are sent to him, with each driver having its own
+subdirectory in drivers/staging/.  Along with the driver source files, a
+TODO file should be present in the directory as well.  The TODO file lists
+the pending work that the driver needs for acceptance into the kernel
+proper, as well as a list of people that should be Cc'd for any patches to
+the driver.  Current rules require that drivers contributed to staging
+must, at a minimum, compile properly.
+
+Staging can be a relatively easy way to get new drivers into the mainline
+where, with luck, they will come to the attention of other developers and
+improve quickly.  Entry into staging is not the end of the story, though;
+code in staging which is not seeing regular progress will eventually be
+removed.  Distributors also tend to be relatively reluctant to enable
+staging drivers.  So staging is, at best, a stop on the way toward becoming
+a proper mainline driver.
+
+
+Tools
+-----
+
+As can be seen from the above text, the kernel development process depends
+heavily on the ability to herd collections of patches in various
+directions.  The whole thing would not work anywhere near as well as it
+does without suitably powerful tools.  Tutorials on how to use these tools
+are well beyond the scope of this document, but there is space for a few
+pointers.
+
+By far the dominant source code management system used by the kernel
+community is git.  Git is one of a number of distributed version control
+systems being developed in the free software community.  It is well tuned
+for kernel development, in that it performs quite well when dealing with
+large repositories and large numbers of patches.  It also has a reputation
+for being difficult to learn and use, though it has gotten better over
+time.  Some sort of familiarity with git is almost a requirement for kernel
+developers; even if they do not use it for their own work, they'll need git
+to keep up with what other developers (and the mainline) are doing.
+
+Git is now packaged by almost all Linux distributions.  There is a home
+page at:
+
+	http://git-scm.com/
+
+That page has pointers to documentation and tutorials.
+
+Among the kernel developers who do not use git, the most popular choice is
+almost certainly Mercurial:
+
+	http://www.selenic.com/mercurial/
+
+Mercurial shares many features with git, but it provides an interface which
+many find easier to use.
+
+The other tool worth knowing about is Quilt:
+
+	http://savannah.nongnu.org/projects/quilt/
+
+Quilt is a patch management system, rather than a source code management
+system.  It does not track history over time; it is, instead, oriented
+toward tracking a specific set of changes against an evolving code base.
+Some major subsystem maintainers use quilt to manage patches intended to go
+upstream.  For the management of certain kinds of trees (-mm, for example),
+quilt is the best tool for the job.
+
+
+Mailing lists
+-------------
+
+A great deal of Linux kernel development work is done by way of mailing
+lists.  It is hard to be a fully-functioning member of the community
+without joining at least one list somewhere.  But Linux mailing lists also
+represent a potential hazard to developers, who risk getting buried under a
+load of electronic mail, running afoul of the conventions used on the Linux
+lists, or both.
+
+Most kernel mailing lists are run on vger.kernel.org; the master list can
+be found at:
+
+	http://vger.kernel.org/vger-lists.html
+
+There are lists hosted elsewhere, though; a number of them are at
+lists.redhat.com.
+
+The core mailing list for kernel development is, of course, linux-kernel.
+This list is an intimidating place to be; volume can reach 500 messages per
+day, the amount of noise is high, the conversation can be severely
+technical, and participants are not always concerned with showing a high
+degree of politeness.  But there is no other place where the kernel
+development community comes together as a whole; developers who avoid this
+list will miss important information.
+
+There are a few hints which can help with linux-kernel survival:
+
+- Have the list delivered to a separate folder, rather than your main
+  mailbox.  One must be able to ignore the stream for sustained periods of
+  time.
+
+- Do not try to follow every conversation - nobody else does.  It is
+  important to filter on both the topic of interest (though note that
+  long-running conversations can drift away from the original subject
+  without changing the email subject line) and the people who are
+  participating.
+
+- Do not feed the trolls.  If somebody is trying to stir up an angry
+  response, ignore them.
+
+- When responding to linux-kernel email (or that on other lists) preserve
+  the Cc: header for all involved.  In the absence of a strong reason (such
+  as an explicit request), you should never remove recipients.  Always make
+  sure that the person you are responding to is in the Cc: list.  This
+  convention also makes it unnecessary to explicitly ask to be copied on
+  replies to your postings.
+
+- Search the list archives (and the net as a whole) before asking
+  questions.  Some developers can get impatient with people who clearly
+  have not done their homework.
+
+- Avoid top-posting (the practice of putting your answer above the quoted
+  text you are responding to).  It makes your response harder to read and
+  makes a poor impression.
+
+- Ask on the correct mailing list.  Linux-kernel may be the general meeting
+  point, but it is not the best place to find developers from all
+  subsystems.
+
+The last point - finding the correct mailing list - is a common place for
+beginning developers to go wrong.  Somebody who asks a networking-related
+question on linux-kernel will almost certainly receive a polite suggestion
+to ask on the netdev list instead, as that is the list frequented by most
+networking developers.  Other lists exist for the SCSI, video4linux, IDE,
+filesystem, etc. subsystems.  The best place to look for mailing lists is
+in the MAINTAINERS file packaged with the kernel source.
+
+
+Getting started with Kernel development
+---------------------------------------
+
+Questions about how to get started with the kernel development process are
+common - from both individuals and companies.  Equally common are missteps
+which make the beginning of the relationship harder than it has to be.
+
+Companies often look to hire well-known developers to get a development
+group started.  This can, in fact, be an effective technique.  But it also
+tends to be expensive and does not do much to grow the pool of experienced
+kernel developers.  It is possible to bring in-house developers up to speed
+on Linux kernel development, given the investment of a bit of time.  Taking
+this time can endow an employer with a group of developers who understand
+the kernel and the company both, and who can help to train others as well.
+Over the medium term, this is often the more profitable approach.
+
+Individual developers are often, understandably, at a loss for a place to
+start.  Beginning with a large project can be intimidating; one often wants
+to test the waters with something smaller first.  This is the point where
+some developers jump into the creation of patches fixing spelling errors or
+minor coding style issues.  Unfortunately, such patches create a level of
+noise which is distracting for the development community as a whole, so,
+increasingly, they are looked down upon.  New developers wishing to
+introduce themselves to the community will not get the sort of reception
+they wish for by these means.
+
+Andrew Morton gives this advice for aspiring kernel developers
+
+::
+
+	The #1 project for all kernel beginners should surely be "make sure
+	that the kernel runs perfectly at all times on all machines which
+	you can lay your hands on".  Usually the way to do this is to work
+	with others on getting things fixed up (this can require
+	persistence!) but that's fine - it's a part of kernel development.
+
+(http://lwn.net/Articles/283982/).
+
+In the absence of obvious problems to fix, developers are advised to look
+at the current lists of regressions and open bugs in general.  There is
+never any shortage of issues in need of fixing; by addressing these issues,
+developers will gain experience with the process while, at the same time,
+building respect with the rest of the development community.