[lilypond.git] / Documentation / contributor / issues.itexi

@c -*- coding: utf-8; mode: texinfo; -*-
@node Issues
@chapter Issues

This chapter deals with defects, feature requests, and
miscellaneous development tasks.

@menu
* Introduction to issues and the Bug Squad::
* Issue classification::
* Adding issues to the tracker::
* Checking and verifying issues::
* Summary of project status::
* Finding the cause of a regression::
@end menu


@node Introduction to issues and the Bug Squad
@section Introduction to issues and the Bug Squad

@warning{Unless otherwise specified, all the tasks in this chapter
are @qq{simple} tasks: they can be done by a normal user with
nothing more than a web browser, email, and lilypond.}

@qq{Issues} isn't just a politically-correct term for @qq{bug}.
We use the same tracker for feature requests and code TODOs, so
the term @qq{bug} wouldn't be accurate.  Despite the difference
between @qq{issue} and @qq{bug}, we call our team of contributors
who organize issues the @emph{Bug Squad}.

The Bug Squad is mainly composed of non-programmers -- their job
is to @emph{organize} issues, not solve them.  Their duties
include removing false bug reports, ensuring that any real bug
report contains enough information for developers, and checking
that a developer's fix actually resolves the problem.

Every email to the bug list should be handled within @strong{24
hours} in one of four ways:

@enumerate

@item
If the behaviour is expected, the user should be told to read the
documentation, or asked to clarify how he misread the docs and how
the docs could be improved.

@item
If the issue already exists in the tracker, send an email to that
effect.

@item
Ask the user for more information, such as lilypond version number
(if not given) and operating system (if this is a suspected cause
of the problem).

@item
Accept the report as described in
@ref{Adding issues to the tracker}.

@end enumerate


@node Issue classification
@section Issue classification

The classification of what counts as a bug vs. feature request,
and the priorities assigned to bugs, are a matter of concern
@strong{for developers only}.  The Bug Squad should classify
issues according to the guidelines given by developers.

Every issue should have a Status, Type, and Priority; the other
fields are optional.

@subheading Status (mandatory)

Open issues:

@itemize

@item
New: the item was added by a non-member, despite numerous warnings
not to do this.  Should be reviewed by a member of the Bug Squad.

@item
Accepted: the Bug Squad added it, or reviewed the item.

@item
Started: a contributor is working on a fix.  Owner should change
to be this contributor.

@end itemize


Closed issues:

@itemize

@item
Invalid: issue should not have been added in the current state.

@item
Duplicate: issue already exists in the tracker.

@item
Fixed: a contributor claims to have fixed the bug.  The Bug
Squad should check the fix with the next official binary release
(not by compiling the source from git).  Owner should be set to
that contributor.

@item
Verified: Bug Squad has confirmed that the issue is closed.

@end itemize


@subheading Owner (optional)

Newly-added issues should have @emph{no owner}.  When a
contributor indicates that he has Started or Fixed an item, he
should become the owner.


@subheading Type (mandatory)

The issue's Type should be the first relevant item in this list.

@itemize

@item
Type-Collision: overlapping notation.

@item
Type-Defect: a problem in the core program.  (the @code{lilypond}
binary, scm files, fonts, etc).

@item
Type-Documentation: inaccurate, missing, confusing, or desired
additional info.  Must be fixable by editing a texinfo, ly, or scm
file.

@item
Type-Build: problem or desired features in the build system.  This
includes the makefiles, stepmake, python scripts, and GUB.

@item
Type-Scripts: problem or desired feature in the non-build-system
scripts.  Mostly used for convert-ly, lilypond-book, etc.

@item
Type-Enhancement: a feature request for the core program.  The
distinction between enhancement and defect isn't extremely clear;
when in doubt, mark it as enhancement.

@item
Type-Other: anything else.

@end itemize


@subheading Priority (mandatory)

Currently, only Critical items will block a stable release.

@itemize

@item
Priority-Critical: lilypond segfaults, or a regression occurred
within the last two stable versions.  (i.e. when developing 2.13,
any regression against 2.12 or 2.10 counts)

@item
Priority-High: highly embarrassing items, and any regression
against a version earlier than two stable versions (i.e. when
developing 2.13, any regression against 2.8 or earlier).  This
level is also used for issues which produce no output and fail to
give the user a clue about what's wrong.

@item
Priority-Medium: normal priority.

@item
Priority-Low: less important than normal.

@item
Priority-Postponed: no fix planned.  Generally used for things
like Ancient notation, which nobody wants to touch.

@end itemize

The difference between Priority-Medium and Priority-Low is not
well-defined, both in this policy and in practice.  The only
answer we can give at the moment is @qq{look at existing items in
of the same type, and try to guess whether the priority is closer
to the Medium items or Low items}.  We're aware of the ambiguity,
and won't complain if somebody picks a @q{wrong} value for
Medium/Low.


@subheading Opsys (optional)

Issues that only affect specific operating systems.


@subheading Other items (optional)

Other labels:

@itemize

@item
Regression: it used to @strong{deliberately} work in an earlier
stable release.  If the earlier output was accidental (i.e. we
didn't try to stop a collision, but it just so happened that two
grobs didn't collide), then breaking it does not count as a
regression.

@item
Patch: a patch to fix an issue is attached.

@item
Frog: the fix is believed to be suitable for a new contributor
(does not require a great deal of knowledge about LilyPond).  The
issue should also have an estimated time in a comment.

@item
Maintainability: hinders developent of LilyPond.  For example,
improvements to the build system, or @qq{helper} python scripts.

@item
Bounty: somebody is willing to pay for the fix.

@item
Warning: graphical output is fine, but lilypond prints a
false/misleading warning message.  Alternately, a warning should
be printed (such as a bar line error), but was not.  Also applies
to warnings when compiling the source code or generating
documentation.

@item
Security: might potentially be used.

@item
Performance: might potentially be used.

@end itemize

If you particularly want to add an label not in the list, go
ahead, but this is not recommended.


@node Adding issues to the tracker
@section Adding issues to the tracker

@warning{This should only be done by the Bug Squad or experienced
developers.  Normal users should not do this; instead, they should
follow the guidelines for @rweb{Bug reports}.}

In order to assign labels to issues, Bug Squad members should log
in to their google account before adding an item.

@subsubheading Normal issues

@itemize

@item
Check if the issue already exists in the tracker.

@item
If the initial bug report contains lilypond examples which do not
follow the guidelines in @rweb{Tiny examples}, either ask the
reporter to create such an example, or make one yourself.

If you are willing to spend the effort, we would prefer it if the
Bug Squad member created a tiny example themselves instead of
asking the user to do so.

@item
Add the issue and classify it according to the guidelines in
@ref{Issue classification}.  In particular, the item should have
@code{Status}, @code{Type-}, and @code{Priority-} labels.

@item
After adding the issue, please send a response email to the same
group(s) that the initial patch was sent to.  If the initial email
was sent to multiple mailing lists (such as both @code{user} and
@code{bugs}), then reply to all those mailing lists as well.  The
email should contain a link to the issue you just added.

@end itemize


@subsubheading Patch reminders

There is a special category of issues: reminders of an existing
patch.  These should be added if a patch has been sent to a
lilypond mailing list (generally @code{lilypond-devel}, but they
sometimes appear on @code{bug-lilypond} as well) and has had no
discussion for at least @strong{3 days}.  Do not add issues for
patches under active discussion.

Before adding a patch-reminder issue, do a quick check to see if
it was pushed without sending any email.  This can be checked for
searching for relevant terms (from the patch subject or commit
message) on the webgit page:

@example
@uref{http://git.savannah.gnu.org/gitweb/?p=lilypond.git}
@end example

After adding the issue, please send a response email to the same
group(s) that the initial patch was sent to.  If the initial email
was sent to multiple mailing lists (such as both @code{bugs} and
@code{devel}), then reply to all those mailing lists as well.  The
email should contain a link to the issue you just added.


@node Checking and verifying issues
@section Checking and verifying issues

After each release (both stable and unstable):

@itemize

@item
Regression test comparison: check the relevant version in
@uref{http://lilypond.org/test/}.

@item
Issues to verify:
@uref{http://code.google.com/p/lilypond/issues/list?can=7}.

@end itemize

Once every two weeks or so:

@itemize

@item
Check for any incorrectly-classified items in the tracker.  This
generally just means looking at the grid to see any items without
a Type or Priority.

@item
Check for any items with @code{label:patch}.  If it's been more
than a week since the last action on the issue, send an email to
-devel to remind them about it.  If the patch was withdrawn for
more work, then remove the @code{patch} label.

@example
@uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:patch}
@end example

@end itemize

Once every year or two: (yes, these are large tasks; gathering
more volunteers to help is definitely recommended)

@itemize

@item
Checking all regtests: although we have a system for checking the
regtests between two versions, occasionally a bug will slip
through the cracks.  It is therefore good to manually examine all
the regtests (compare the images to the text description).

@item
Checking all issues: we try to mark each Issue @q{fixed} when we
fix it, but occasionally one or two issues will slip through the
cracks.  It is therefore good to check all Issues. If you see the
same (broken) output as the initial report, then simply post a
@qq{Problem still exists in 2.x.y} message to the issue.

@end itemize


@node Summary of project status
@section Summary of project status

The best overview of our current status is given by the grid view:

@example
@uref{http://code.google.com/p/lilypond/issues/list?mode=grid&y=Priority&x=Type&cells=ids}
@end example

Also of interest might be the issues hindering future development:

@example
@uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:Maintainability&mode=grid&y=Priority&x=Type&cells=ids}
@end example

Finally, issues tagged with @code{Frog} indicates a task suitable
for a relatively new contributor.  The time given is a quick
(inaccurate) estimate of the time required for somebody who is
familiar with material in this manual, but does not know anything
else about LilyPond development.

@example
@uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:Frog&mode=grid&y=Priority&x=Type&cells=ids}
@end example


@node Finding the cause of a regression
@section Finding the cause of a regression

@warning{This is not a @qq{simple} task; it requires a fair amount
of technical knowledge.}

Git has special functionality to help tracking down the exact
commit which causes a problem.  See the git manual page for
@code{git bisect}.  This is a job that non-programmers can do,
although it requires familiarity with git, ability to compile
LilyPond, and generally a fair amount of technical knowledge.  An
in-depth explanation of this process will not be given here.

Even if you are not familiar with git or are not able to compile
LilyPond you can still help to narrow down the cause of a
regression simply by downloading the binary releases of different
LilyPond versions and testing them for the regression.  Knowing
which version of LilyPond first exhibited the regression is
helpful to a developer as it shortens the @code{git bisect}
procedure described above.

Once a problematic commit is identified, the programmers' job is
much easier.  In fact, for most regression bugs, the majority of
the time is spent simply finding the problematic commit.