PracticalOSSEngineering: Documentation
1. Introduction
Somebody Should Set The Title For This Chapter!
2. Open Content Writing Cycle
2.0.1 Writing Cycle Exercise
2.2 Common Documentation Project Structures and Tools
2.3 Documenting Technical and Community Proceedings
= Introduction =
= Using documentation as a way to get involved =
== Exercise - getting involved ==
== Principle - getting involved ==
== Example - getting involved ==
= Collaborating on open documentation =
== Exercise - collaborating on a document ==
== Principle - collaborating on a document ==
== Example - collaborating on a document ==
= Document for the community of contributors =
== Exercise - document for community ==
== Example - document for community ==
== Principle - document for community ==
= Documenting technical and community proceedings =
== Exercise - documenting proceedings ==
== Example - documenting proceedings ==
== Principle - documenting proceedings ==
= Common tools and processes for open source documentation =
== Exercise - common tools and processes ==
== Example - common tools and processes ==
== Principle - common tools and processes ==
= Introduction =
People are going to tell you documentation doesn't matter. In a perfect world, it doesn't! If all code were self-evident and all programs worked for users without their ever needing help, perhaps we could do without documentation. Of course, that would require all people to think and act in similar ways; we may never be without documentation.
From a practical standpoint, here's a piece of code. What does it do? Take 2 minutes and see how far you can get just by looking at the code snippet alone.
def findAllPrevious(self, name=None, attrs={}, text=None, limit=None,
**kwargs):
return self._findAll(name, attrs, text, limit, self.previousGenerator,
**kwargs)
Even if you had written this code yourself 6 months ago, how likely would you be to remember what it does now?
Okay. How about this?
def findAllPrevious(self, name=None, attrs={}, text=None, limit=None,
**kwargs):
"""Returns all items that match the given criteria and appear
before this Tag in the document."""
return self._findAll(name, attrs, text, limit, self.previousGenerator,
**kwargs)
Okay. Now go to this link.
http://www.crummy.com/software/BeautifulSoup/documentation.html#findAllPrevious(name,%20attrs,%20text,%20limit,%20**kwargs)%20and%20findPrevious(name,%20attrs,%20text,%20**kwargs)
Which of those methods gives you the quickest idea of how to use that particular piece of code? If you had to figure out how to use this XML parsing library and make something in the next 10 minutes, which level of documentation would you hope the code would have?
According to Wikipedia (
http://en.wikipedia.org/wiki/Software_documentation), documentation "is written text that accompanies computer software. It either explains how it operates or how to use it." For software engineering purposes, operate/use includes manipulating the source code as well as using the application. Documentation is a sanity-preserving tool for users of and potential contributors to your project, including your future self... and perhaps your future boss or coworker.
This is different from content written to document an event, such as notes for a class or technical conference, or minutes from the proceedings of a meeting. In open source projects, similar tools and processes are used in this type of documenting. It is worth mentioning this form of documentation because keeping transparent proceedings of meetings and similar events is a useful skill for open source software developers.
At the end of this chapter you should be able to:
1. Use documentation as a way to become immediately useful as a project newcomer (reference to TOSW phrase #N).
2. Engage in an open collaboration on documentation, whether in source code or on a wiki.
3. Document your project in a way that will encourage other people to discover, use, and contribute to it.
4. Participate as part of a community in documenting a proceeding or event.
5. Explain the value of documentation to open source engineering.
Kat Walsh: "Not just encourage other people to discover and use it themselves, but make them want to share it with other people and go 'hey, look how good this is.'"
Every technical effort has a requirement for documentation. The project's ability to make writing, editing, and publishing as easy as possible is the main key to attracting and retaining contributors. In the Mozilla example later in this chapter, there is a clear difference that scalable documentation processes, tools, and teams makes to the success of an open source engineering effort.
According to Mozilla (
http://www.dria.org/wordpress/archives/2006/03/15/401/), developer contributions to documentation greatly increased along with a new community of writers, editors, and translators. This all resulted from reducing the barriers to success that made developers to frustrated to document properly.
= Using documentation as a way to get involved =
== Exercise - getting involved ==
1. Pick an open source project (or a subset of a project) that you think is interesting. It must be an area that you've never contributed to before.
2. Pick a feature that sounds tantalizing but is not clearly documented.
3. Proceed to figure out how to use the feature by documenting its implementation and proper usage.
4. When you get stuck, contact the original author (reference: how to ask good questions) or ask on IRC. Tell people that you're trying to write documentation about X but are getting stuck on Y.
== Principle - getting involved ==
Documentation has a reputation as being the "easy" way to begin contributing to a project. This is more true where the new contributor is already experienced or an expert in the material. It is much less likely to be true where the project has not considered how to enable good and easy documentation, or the technical content is very high.
When developers resist documenting, it is usually around several themes:
1. The tools are painful to use.
2. Writing is not a principal skill.
3. It takes too much time, usually because of 1 and 2.
The techniques for creating a successful documentation project that attracts contributions from actual project developers are mirrored answers to these problems:
1. Focus on making the tools easy to use.
2. Turn writing in to a "brain dump, that is, an outpouring of information without concern to writing standards (grammar, spelling, or structure).
3. Enable non-developers to provide significant contributions by saving the developers from the actual work of writing, which is editing.
Dumping down words in to a text editor is not that hard; many developers write copiously in email, for example. The documentor's challenge is finding this content wherever it is (mailing lists, IRC discussions, random wiki pages) and editing it in to something comprehensive that reveals content holes for filling. The documentation project's success hinges on the ability to make all of that accessible to new writing contributors, so they can begin meaningful work from the very start.
A metric of this success is when any random experienced contributor is asked a question by a new contributor, and in answering, insists that the answer be documented, for example, on the project wiki. In this way new contributors are turned in to documentors who share the work burden from the existing contributors.
== Example - getting involved ==
EXAMPLE NEEDED
= Collaborating on open documentation =
== Exercise - collaborating on a document ==
Work with at least one other person. For this lesson you will need:
* A working wiki instance
* A topic, either real or imagined for the exercise
Running time: 1 hour
It is possible to do this exercise as, for example, a series of email exchanges. The reason for using the wiki is to connect with other essentials of open source engineering: central collaboration, version control, and full visibility. For example, the technical expert can watch edits occur as they saved to a wiki page, and use page version control to check what was changed and why.
1. One person is the technical expert, picking a topic they can write at some length about without spending concerned with grammar, spelling, and structure. The other persons are editors and writers; with more than one editor/writer, exchange roles equally throughout the exercise.
2. The technical expert writes in to a wiki page as quickly as possible, focusing on technical accuracy and completion of information. Do not worry about clarity to non-experts, for example. This step in the exercise expects that the expert will leave gaps of information or understanding
3. The first editor passes through this content, giving it structure, resolving grammar and spelling mistakes, and noting (in comments or the wiki talk page) any questions or points about understanding/clarity of the content. This editor should not rewrite any content beyond simple grammar fixes.
4. The first writer then reads through the content, rewriting for clarity, and drawing clear circles around the holes in the content. Holes may be assumptions the expert made about reader understanding, or steps missing in a process, for example. The writer also attempts to gain expertise through the reading; rewriting attempts should show this expertise gain, which helps the technical expert when they reappear.
5. At this point, other writers can intervene, collaborate directly, or do a second rewriting session.
6. The expert, who may have been watching the edit stream as it occurred, can enter at this point (or any point earlier) to provide technical answers and edits. In particular, the expert is doing the "technical edit", which ensures that the facts are correct. (In practice, this role can be provided by another expert, so experts can share technical editing of each other's content.)
7. During the technical edit and afterward in a new expertise content dump, the writing is likely to become unclear, gain errors, or break established voice, patterns, or structure. Cycle through steps 2 to 6 until the document is complete and readable to a new reader or contributor.
== Principle - collaborating on a document ==
When working on content where developers are an important content source, keeping the tooling and processes close to the developers existing workflow is to the documentators advantage. It becomes possible to collaborate directly on content, even in real time. One method is to iterate on this pattern:
a. Expert dumps content
b. Editor organizes and edits
c. Writer rewrites and edits, noting questions for writers (in comments, wiki talk pages, etc.)
d. Expert reviews, fixes mistakes, fills out missing parts, dumps new content
e. Cycle through b, c, d until content is determined to be ready
f. Ongoing, others may notice problems, make suggestions, or add new content; at that point, cycle again through b, c, d
This process takes advantage of collaboration through extreme visibility. With each step viewable in real time and as historical data, it makes it possible for an expert, an editor, or a writer to enter the workflow stream at any point. The new person can learn what has gone on before, why it has occurred (thanks to explanations in content save comments), and find a place to begin contributing very quickly.
The process also allows existing contributors to enter and leave the process easily. For example, a contributor can go on vacation for several weeks. The work continues in her absence, and when she returns, she can enter the workflow in any of the roles she is prepared for.
== Example - collaborating on a document ==
EXAMPLE NEEDED
= Documenting technical and community proceedings =
This section is focused on reasons and methods for documenting community proceedings, such as developer conferences or technical steering group meetings.
These are the skills used by standards organizations around the world, and are very similar when used as part of the open source development model. Where open source projects might differ on the specifics of how something is done (the tools), the reasons why and the type of information captured and revealed are common across free communities.
== Exercise - documenting proceedings ==
EXERCISE NEEDED
== Example - documenting proceedings ==
EXAMPLE NEEDED
== Principle - documenting proceedings ==
PRINCIPLE NEEDED
= Common tools and processes for open source documentation =
== Exercise - common tools and processes ==
EXERCISE NEEDED
== Principle - common tools and processes ==
PRINCIPLE NEEDED
* wiki
* tex
* DocBook XML
* XHTML
* ODF
* CMS-loaded (Wordpress to Drupal)
== Example - common tools and processes ==
For a number of years, the Mozilla developer documentation was written using one of the venerable documenting tools, DocBook. For many years, DocBook used SGML, the meta-markup language that as a superset for all other similar markup languages, such as HTML. Ideally, the similarity to HTML and the common familiarity with writing in HTML should have worked to Mozilla's advantage; after all, "Everyone knows how to write in HTML." It was also true that many also despised writing in HTML. SGML's much larger tagset and toolchain requirements, such as having to compile content to view it for every change, also worked against adoption. Later year changes to using XML, the new super-set, did not help perception or reality.
What Mozilla Documentation leadership saw was that although Mozilla developers had access to the SCM and the ability to edit DocBook directly, they generally did not. This is an example of where the tools work to raise the barrier to everyone, regardless of experience and position in the project.
When the Mozilla team migrated all the content permanently to a wiki, completely abandoning the DocBook SGML/XML, they witnessed several expected and unexpected effects:
* Existing developer contributions increased by several magnitudes.
* Developers were increasingly likely to do small edits to content due to how easy it was to edit via MediaWiki.
* The reduced writing costs encouraged the creating of more documents: tests, scratch ideas, smaller bits, temporary work, etc.
* Non-developers began to edit the content; within a relatively short period of time, a team of people nearly twice the size of the developer team were regularly reading, editing, and re-writing developer content in real time.
This is a good example of following the described documenting methodology. The wiki tooling provided the wider Mozilla contributor community the ability to easily cycle through the "dump, edit, write, tech edit, return" pattern.