-@c -*- coding: us-ascii; mode: texinfo; -*-
+@c -*- coding: utf-8; mode: texinfo; -*-
@node Issues
@chapter Issues
+This chapter deals with defects, feature requests, and
+miscellaneous development tasks.
+
@menu
-* Introduction to issues::
-* Issue classification::
-* Adding issues to the tracker::
-* Checking regression tests::
-* Checking old issues::
+* Introduction to issues::
+* Bug Squad checklists::
+* Issue classification::
+* Adding issues to the tracker::
+* Summary of project status::
+* Finding the cause of a regression::
@end menu
@node Introduction to issues
@section Introduction to issues
-First, @qq{issue} isn't just a politically-correct term for
-@qq{bug}. We use the same tracker for feature requests and code
-TODOs, so the term @qq{bug} wouldn't be accurate.
+@warning{Unless otherwise specified, all the tasks in this chapter
+are @qq{simple} tasks: they can be done by a normal user with
+nothing more than a web browser, email, and lilypond.}
+
+@qq{Issues} isn't just a politically-correct term for @qq{bug}.
+We use the same tracker for feature requests and code TODOs, so
+the term @qq{bug} wouldn't be accurate. Despite the difference
+between @qq{issue} and @qq{bug}, we call our team of contributors
+who organize issues the @emph{Bug Squad}.
+
+The Bug Squad is mainly composed of non-programmers -- their job
+is to @emph{organize} issues, not solve them. Their duties
+include removing false bug reports, ensuring that any real bug
+report contains enough information for developers, and checking
+that a developer's fix actually resolves the problem.
+
+New volunteers for the Bug Squad should contact the
+@ref{Meisters, Bug Meister}.
+
+
+@node Bug Squad checklists
+@section Bug Squad checklists
+
+We highly recommend that you configure your email client to put
+messages from:
+
+@example
+@@googlecode.com
+@end example
+
+@noindent
+into a separate folder than other emails to @code{bug-lilypond}.
+
+
+@subsubheading New emails to @code{bug-lilypond}
+
+Every new email to @code{bug-lilypond} should be handled within
+@strong{24 hours} in the first method which is applicable:
+
+@enumerate
+
+@item
+If the email is a question about how to use LilyPond, direct them
+politely to @code{lilypond-user}.
+
+@item
+If a bug report is not in the form of a Tiny example, direct the
+user to resubmit the report after reading @rweb{Tiny examples}.
+
+@item
+If anything is unclear, ask the user for more information.
+
+How does the graphical output differ from what the user expected?
+What version of lilypond was used (if not given) and operating
+system (if this is a suspected cause of the problem)? In short,
+if you cannot understand what the problem is, ask the user to
+explain more. It is the user's responsibility to explain the
+problem, not your reponsibility to understand it.
+
+@item
+If the behavior is expected, the user should be told to read the
+documentation, or asked to clarify how he misread the docs and how
+the docs could be improved.
+
+@item
+If the issue already exists in the tracker, send an email to that
+effect.
+
+@item
+Accept the report as described in
+@ref{Adding issues to the tracker}.
+
+@end enumerate
+
+All emails should be CC'd to the @code{bug-lilypond} list so that
+other Bug Squad members know that you have processed the email.
+
+@warning{There is no option for @qq{ignore the bug report} -- if
+you cannot find a reason to reject the report, you must accept
+it.}
+
+
+@subheading Updates / discussion about issues
+
+We try to keep discussions about issues on the tracker, but
+sometimes it spills over onto email. If discussion has ended with
+no patch / resolution and at least @strong{3 days} have passed,
+then either:
+
+@itemize
+
+@item
+Summarize the recent discussion on the tracker, and add a link to
+the original discussion.
+
+@item
+Add the comment @qq{there was some technical discussion which I
+could not understand}, and include a link to the original
+discussion.
+
+We do not expect Bug Squad members to be programmers, or even to
+be moderately-skilled users. Your job is to keep track of issue
+reports; it is @emph{perfectly acceptable} to not understand
+discussions between advanced users and/or developers.
+
+@end itemize
+
+
+@subheading Regular maintenance
+
+After @strong{every release} (both stable and unstable):
+
+@itemize
+
+@item
+Regression test comparison: if anything has changed suspiciously,
+ask if it was deliberate. The official comparison is online, at:
+
+@c NOTE: leave this here. In this case, it's worth duplicating
+@c the link. -gp
+@example
+@uref{http://lilypond.org/test/}
+@end example
+
+More information is available from in
+@ref{Precompiled regression tests}.
+
+@item
+Issues to verify: try to reproduce the bug with the latest
+version; if you cannot reproduce the bug, mark the item
+@qq{Verified} (i.e. @qq{the fix has been verified to work}).
+
+@example
+@uref{http://code.google.com/p/lilypond/issues/list?can=7}
+@end example
+
+@end itemize
+
+Once every @strong{two weeks} or so:
+
+@itemize
+
+@item
+Check for any incorrectly-classified items in the tracker. This
+generally just means looking at the grid to see any items without
+a Type or Priority.
+
+@item
+Check for any items with @code{label:patch}. If it's been more
+than a week since the last action on the issue, send an email to
+-devel to remind them about it. If the patch was withdrawn for
+more work, then remove the @code{patch} label.
+
+@example
+@uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:patch}
+@end example
+
+@end itemize
+
+@subheading Irregular maintenance
+
+@warning{These tasks are a lot of work; gathering more volunteers
+to help is definitely recommended. However, the Bug Squad should
+handle the organization and training of new volunteers.}
+
+Once every year or two:
+
+@itemize
+
+@item
+Checking all regtests: although we have a system for checking the
+regtests between two versions, occasionally a bug will slip
+through the cracks. It is therefore good to manually examine all
+the regtests (compare the images to the text description). More
+information is available from in @ref{Regression tests}.
+
+
+@item
+Checking all issues: we try to mark each Issue @q{fixed} when we
+fix it, but occasionally one or two issues will slip through the
+cracks. It is therefore good to check all Issues. If you see the
+same (broken) output as the initial report, then simply post a
+@qq{Problem still exists in 2.x.y} message to the issue.
-Second, the classification of what counts as a bug vs. feature
-request, and the priorities assigned to bugs, are a matter of
-concern @strong{for developers only}. If you are curious about
-the classification, read on, but don't complain that your
-particular issue is higher priority or counts as a bug rather than
-a feature request.
+@end itemize
@node Issue classification
@section Issue classification
-Status values:
+The Bug Squad should classify issues according to the guidelines
+given by developers. Every issue should have a Status, Type, and
+Priority; the other fields are optional.
+
+@subheading Status (mandatory)
+
+Open issues:
@itemize
@item
-New: the item was added by a non-member. Should be reviewed by
-the Bug Meister.
+New: the item was added by a non-member, despite numerous warnings
+not to do this. Should be reviewed by a member of the Bug Squad.
@item
-Accepted: the Bug Meister added it, or reviewed the item.
+Accepted: the Bug Squad added it, or reviewed the item.
@item
-Started: a programmer is working on a bugfix. (used infrequently,
-but should be used more often)
+Started: a contributor is working on a fix. Owner should change
+to be this contributor.
@end itemize
-Closed status values:
+
+Closed issues:
@itemize
Duplicate: issue already exists in the tracker.
@item
-Fixed: programmer claims to have fixed the bug. The Bug Meister
-should check the input code in an official binary release.
+Fixed: a contributor claims to have fixed the bug. The Bug
+Squad should check the fix with the next official binary release
+(not by compiling the source from git). Owner should be set to
+that contributor.
@item
-Verified: Bug Meister has confirmed that the issue is closed.
+Verified: Bug Squad has confirmed that the issue is closed. This
+means that nobody should ever need look at the report again -- if
+there is any information in the issue that should be kept, open a
+new issue for that info.
@end itemize
-Type labels:
+
+@subheading Owner (optional)
+
+Newly-added issues should have @emph{no owner}. When a
+contributor indicates that he has Started or Fixed an item, he
+should become the owner.
+
+
+@subheading Type (mandatory)
+
+The issue's Type should be the first relevant item in this list.
@itemize
@item
-Type-Defect: a problem that requires no (or very little) new code
-to fix.
+Type-Collision: overlapping notation.
+
+@item
+Type-Defect: a problem in the core program. (the @code{lilypond}
+binary, scm files, fonts, etc).
@item
-Type-Enhancement: a problem (or new feature) that requries a
-significant amount of new code.
+Type-Documentation: inaccurate, missing, confusing, or desired
+additional info. Must be fixable by editing a texinfo, ly, or scm
+file.
@item
-Type-Collision: overlapping notation. (this label takes
-precedence over -Defect and -Enhancement)
+Type-Build: problem or desired features in the build system. This
+includes the makefiles, stepmake, python scripts, and GUB.
@item
-Type-Task: not used, I think. TODO: start using it or delete it.
+Type-Scripts: problem or desired feature in the non-build-system
+scripts. Mostly used for convert-ly, lilypond-book, etc.
+@item
+Type-Enhancement: a feature request for the core program. The
+distinction between enhancement and defect isn't extremely clear;
+when in doubt, mark it as enhancement.
@item
-Type-Other: anything else. TODO: start using it or delete it.
+Type-Other: anything else.
@end itemize
-Priority labels:
+
+@subheading Priority (mandatory)
+
+Currently, only Critical items will block a stable release.
@itemize
@item
-Priority-High: lilypond segfaults.
+Priority-Critical: lilypond segfaults, or a regression occurred
+within the last two stable versions. (i.e. when developing 2.13,
+any regression against 2.12 or 2.10 counts)
@item
-Priority-Regression: it used to work.
+Priority-High: highly embarrassing items, and any regression
+against a version earlier than two stable versions (i.e. when
+developing 2.13, any regression against 2.8 or earlier). This
+level is also used for issues which produce no output and fail to
+give the user a clue about what's wrong.
@item
-Priority-Medium: normal priority; this is the highest priority a
-non-crashing, non-regression bug report can receive.
-(regardless of the perceived importance)
+Priority-Medium: normal priority.
@item
Priority-Low: less important than normal.
@end itemize
-Opsys lables: pretty self-explanatory.
+The difference between Priority-Medium and Priority-Low is not
+well-defined, both in this policy and in practice. The only
+answer we can give at the moment is @qq{look at existing items in
+of the same type, and try to guess whether the priority is closer
+to the Medium items or Low items}. We're aware of the ambiguity,
+and won't complain if somebody picks a @q{wrong} value for
+Medium/Low.
+
-Other lables:
+@subheading Opsys (optional)
+
+Issues that only affect specific operating systems.
+
+
+@subheading Other items (optional)
+
+Other labels:
@itemize
@item
-Security: not used. TODO: delete, unless anybody is serious about
-this.
+Regression: it used to @strong{deliberately} work in an earlier
+stable release. If the earlier output was accidental (i.e. we
+didn't try to stop a collision, but it just so happened that two
+grobs didn't collide), then breaking it does not count as a
+regression.
@item
-Performance: not used. TODO: delete.
+Patch: a patch to fix an issue is attached.
@item
-Usability: not used. TODO: delete.
+Frog: the fix is believed to be suitable for a new contributor
+(does not require a great deal of knowledge about LilyPond). The
+issue should also have an estimated time in a comment.
@item
Maintainability: hinders developent of LilyPond. For example,
improvements to the build system, or @qq{helper} python scripts.
@item
-Bounty: somebody is willing to pay for the fix.
+Bounty: somebody is willing to pay for the fix. Only add this tag
+if somebody has offered an exact figure in US dollars or euros.
@item
-Engraving-nitpick: output is not beautiful, but not strictly
-speaking @qq{wrong}. For example, a slur shape which does not
-collide with any notation, but looks ugly.
+Warning: graphical output is fine, but lilypond prints a
+false/misleading warning message. Alternately, a warning should
+be printed (such as a bar line error), but was not. Also applies
+to warnings when compiling the source code or generating
+documentation.
@item
-Warning-nitpick: graphical output is fine, but lilypond prints a
-false/misleading warning message.
+Security: might potentially be used.
+
+@item
+Performance: might potentially be used.
@end itemize
+If you particularly want to add an label not in the list, go
+ahead, but this is not recommended.
+
@node Adding issues to the tracker
@section Adding issues to the tracker
-FIXME: prettify.
+@warning{This should only be done by the Bug Squad or experienced
+developers. Normal users should not do this; instead, they should
+follow the guidelines for @rweb{Bug reports}.}
+
+In order to assign labels to issues, Bug Squad members should log
+in to their google account before adding an item.
+
+@subsubheading Normal issues
+
+@enumerate
+
+@item
+Check if the issue falls into any previous category given on the
+relevant checklists in @ref{Bug Squad checklists}. If in doubt,
+add a new issue for a report. We would prefer to have some
+incorrectly-added issues rather than lose information that should
+have been added.
+
+@item
+Add the issue and classify it according to the guidelines in
+@ref{Issue classification}. In particular, the item should have
+@code{Status}, @code{Type-}, and @code{Priority-} labels.
+
+@item
+After adding the issue, please send a response email to the same
+group(s) that the initial patch was sent to. If the initial email
+was sent to multiple mailing lists (such as both @code{user} and
+@code{bugs}), then reply to all those mailing lists as well. The
+email should contain a link to the issue you just added.
+
+@end enumerate
+
+
+@subsubheading Patch reminders
+
+@warning{This is not a Bug Squad responsibility; we have a
+separate person handling this task.}
+
+There is a special category of issues: reminders of an existing
+patch. These should be added if a patch has been sent to a
+lilypond mailing list (generally @code{lilypond-devel}, but they
+sometimes appear on @code{bug-lilypond} as well) and has had no
+discussion for at least @strong{3 days}. Do not add issues for
+patches under active discussion.
+
+Before adding a patch-reminder issue, do a quick check to see if
+it was pushed without sending any email. This can be checked for
+searching for relevant terms (from the patch subject or commit
+message) on the webgit page:
+
+@example
+@uref{http://git.savannah.gnu.org/gitweb/?p=lilypond.git}
+@end example
+
+After adding the issue, please send a response email to the same
+group(s) that the initial patch was sent to. If the initial email
+was sent to multiple mailing lists (such as both @code{bugs} and
+@code{devel}), then reply to all those mailing lists as well. The
+email should contain a link to the issue you just added.
+
+
+
+@node Summary of project status
+@section Summary of project status
+
+The best overview of our current status is given by the grid view:
+
+@example
+@uref{http://code.google.com/p/lilypond/issues/list?mode=grid&y=Priority&x=Type&cells=ids}
+@end example
+
+Also of interest might be the issues hindering future development:
-only done by Bug Meister, unless you're really certain you know
-what you're doing.
+@example
+@uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:Maintainability&mode=grid&y=Priority&x=Type&cells=ids}
+@end example
+Finally, issues tagged with @code{Frog} indicates a task suitable
+for a relatively new contributor. The time given is a quick
+(inaccurate) estimate of the time required for somebody who is
+familiar with material in this manual, but does not know anything
+else about LilyPond development.
+@example
+@uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:Frog&mode=grid&y=Priority&x=Type&cells=ids}
+@end example
-@node Checking regression tests
-@section Checking regression tests
-Although we have a system for checking the regtests between two versions,
-occasionally a bug will slip through the cracks. It is therefore good to
-manually compare all the regtests twice a year or so.
+@node Finding the cause of a regression
+@section Finding the cause of a regression
+@warning{This is not a @qq{simple} task; it requires a fair amount
+of technical knowledge.}
+Git has special functionality to help tracking down the exact
+commit which causes a problem. See the git manual page for
+@code{git bisect}. This is a job that non-programmers can do,
+although it requires familiarity with git, ability to compile
+LilyPond, and generally a fair amount of technical knowledge. An
+in-depth explanation of this process will not be given here.
-@node Checking old issues
-@section Checking old issues
+Even if you are not familiar with git or are not able to compile
+LilyPond you can still help to narrow down the cause of a
+regression simply by downloading the binary releases of different
+LilyPond versions and testing them for the regression. Knowing
+which version of LilyPond first exhibited the regression is
+helpful to a developer as it shortens the @code{git bisect}
+procedure described above.
-We try to mark each Issue @q{fixed} when we fix it, but
-occasionally one or two issues will slip through the cracks.
-Also, sometimes the Bug Meister will forget to verify that an
-issue has been fixed.
+Once a problematic commit is identified, the programmers' job is
+much easier. In fact, for most regression bugs, the majority of
+the time is spent simply finding the problematic commit.
-It is therefore good to check all Issues once every 18 months or
-so. If you see the same (broken) output as the initial report,
-then simply post a "Problem still exists in 2.x.y" message to the
-issue.
+More information is in @ref{Regression tests}.
projects / lilypond.git / blobdiff