-@c -*- coding: us-ascii; mode: texinfo; -*-
+@c -*- coding: utf-8; mode: texinfo; -*-
@node Issues
@chapter Issues
miscellaneous development tasks.
@menu
-* Introduction to issues::
-* Issue classification::
-* Adding issues to the tracker::
-* Checking and verifying issues::
+* Introduction to issues::
+* Bug Squad setup::
+* Bug Squad checklists::
+* Issue classification::
+* Adding issues to the tracker::
* Summary of project status::
* Finding the cause of a regression::
@end menu
@node Introduction to issues
@section Introduction to issues
-First, @qq{issue} isn't just a politically-correct term for
-@qq{bug}. We use the same tracker for feature requests and code
-TODOs, so the term @qq{bug} wouldn't be accurate.
+@warning{Unless otherwise specified, all the tasks in this chapter
+are @qq{simple} tasks: they can be done by a normal user with
+nothing more than a web browser, email, and lilypond.}
-Second, the classification of what counts as a bug vs. feature
-request, and the priorities assigned to bugs, are a matter of
-concern @strong{for developers only}. If you are curious about
-the classification, read on, but don't complain that your
-particular issue is higher priority or counts as a bug rather than
-a feature request.
+@qq{Issues} isn't just a politically-correct term for @qq{bug}.
+We use the same tracker for feature requests and code TODOs, so
+the term @qq{bug} wouldn't be accurate. Despite the difference
+between @qq{issue} and @qq{bug}, we call our team of contributors
+who organize issues the @emph{Bug Squad}.
+
+The Bug Squad is mainly composed of non-programmers -- their job
+is to @emph{organize} issues, not solve them. Their duties
+include removing false bug reports, ensuring that any real bug
+report contains enough information for developers, and checking
+that a developer's fix actually resolves the problem.
+
+New volunteers for the Bug Squad should contact the
+@ref{Meisters, Bug Meister}.
+
+
+@node Bug Squad setup
+@section Bug Squad setup
+
+We highly recommend that you configure your email to use effective
+sorting; this can reduce your workload @emph{immensely}. The
+email folders names were chosen specifically to make them work if
+you sort your folders alphabetically.
+
+@enumerate
+
+@item
+Skim through every section of this chapter, @ref{Issues}. Read in
+detail any sections called @qq{Bug Squad...}, or any page linked
+from @ref{Bug Squad checklists}.
+
+@item
+If you do not have one already, create a gmail account and send
+the email address to the @ref{Meisters, Bug Meister}.
+
+@item
+Subscribe your gmail account to @code{bug-lilypond}.
+
+@item
+Configure your google code account:
+
+@enumerate
+
+@item
+Sign in to google code by clicking in the top-right corner of:
+
+@example
+@uref{http://code.google.com/p/lilypond/issues/list}
+@end example
+
+@item
+Go to your @qq{Profile}, and select @qq{Settings}.
+
+@item
+Scroll down to @qq{Issue change notification}, and make sure that
+you have @emph{selected} @qq{If I starred the issue}.
+
+@end enumerate
+
+@item
+Configure your email client:
+
+@enumerate
+
+@item
+Any email sent with your gmail address in the @code{To:} or
+@code{CC:} fields should go to a @code{bug-answers} folder.
+
+When setting up your filtering rules, be aware that Google Code
+might use different versions of your email address, such as ones
+ending in @code{@@googlemail.com} or @code{@@gmail.com}.
+
+@item
+Any other email either from, or CC'd to,
+
+@example
+lilypond@@googlecode.com
+@end example
+
+@noindent
+should go into a separate @code{bug-ignore} folder. Alternately,
+you may automatically delete these emails.
+
+You will @strong{not read} these emails as part of your Bug Squad
+duties. If you are curious, go ahead and read them later, but it
+does @strong{not} count as Bug Squad work.
+
+@item
+Any other email sent to (or CC'd to):
+
+@example
+bug-lilypond
+@end example
+
+@noindent
+should go into a separate @code{bug-current} folder.
+
+@end enumerate
+
+@end enumerate
+
+
+@node Bug Squad checklists
+@section Bug Squad checklists
+
+When you do Bug Squad work, start at the top of this page and work
+your way down. Stop when you've done 15 minutes.
+
+Please use the email sorting described in @ref{Bug Squad setup}.
+This means that (as Bug Squad members) you will only ever respond
+to emails sent or CC'd to the @code{bug-lilypond} mailing list.
+
+
+@subsubheading Emails to you personally
+
+You are not expected to work on Bug Squad matters outside of your
+15 minutes, but sometimes a confused user will send a bug report
+(or an update to a report) to you personally. If that happens,
+please forward such emails to the @code{bug-lilypond} list so that
+the currently-active Bug Squad member(s) can handle the message.
+
+
+@subsubheading Daily schedule
+
+The Bug Meister is omitted from the daily schedule.
+
+@example
+Sunday: Valentin
+Monday: Dmytro
+Tuesday: James Bailey
+Wednesday: Ralph
+Thursday: Patrick
+Friday: Urs
+Saturday: Kieren
+@end example
+
+
+@subsubheading Emails to @code{bug-answers}
+
+Some of these emails will be comments on issues that you added to
+the tracker.
+
+@itemize
+If they are asking for more information, give the additional
+information.
+
+@item
+If the email says that the issue was classified in some other
+manner, read the rationale given and take that into account for
+the next issue you add.
+
+@item
+Otherwise, move them to your @code{bug-ignore} folder.
+
+@end itemize
+
+Some of these emails will be discussions about Bug Squad work;
+read those.
+
+
+@subsubheading Emails to @code{bug-current}
+
+Dealing with these emails is your main task. Your job is to get
+rid of these emails in the first method which is applicable:
+
+@enumerate
+@item
+If the email has already been handled by a Bug Squad member (i.e.
+check to see who else has replied to it), delete it.
+
+@item
+If the email is a question about how to use LilyPond, reply with
+this response:
+
+@example
+For questions about how to use LilyPond, please read our
+documentation available from:
+ @uref{http://lilypond.org/website/manuals.html}
+or ask the lilypond-user mailing list.
+@end example
+
+@item
+If the email mentions @qq{the latest git}, or any version number
+that has not yet been officially released, forward it to
+@code{lilypond-devel}.
+
+@item
+If a bug report is not in the form of a Tiny example, direct the
+user to resubmit the report with this response:
+
+@example
+I'm sorry, but due to our limited resources for handling bugs, we
+can only accept reports in the form of Tiny examples. Please see
+step 2 in our bug reporting guidelines:
+ @uref{http://lilypond.org/website/bug-reports.html}
+@end example
+
+@item
+If anything is unclear, ask the user for more information.
+
+How does the graphical output differ from what the user expected?
+What version of lilypond was used (if not given) and operating
+system (if this is a suspected cause of the problem)? In short,
+if you cannot understand what the problem is, ask the user to
+explain more. It is the user's responsibility to explain the
+problem, not your responsibility to understand it.
+
+@item
+If the behavior is expected, the user should be told to read the
+documentation:
+
+@example
+I believe that this is the expected behaviour -- please read our
+documentation about this topic. If you think that it really is a
+mistake, please explain in more detail. If you think that the
+docs are unclear, please suggest an improvement as described by
+@qq{Simple tasks -- Documentation} on:
+ @uref{http://lilypond.org/website/help-us.html}
+@end example
+
+@item
+If the issue already exists in the tracker, send an email to that
+effect:
+
+@example
+This issue has already been reported; you can follow the
+discussion and be notified about fixes here:
+@end example
+
+@noindent
+(copy+paste the google code issue URL)
+
+@item
+Accept the report as described in
+@ref{Adding issues to the tracker}.
+
+@end enumerate
+
+All emails should be CC'd to the @code{bug-lilypond} list so that
+other Bug Squad members know that you have processed the email.
+
+@warning{There is no option for @qq{ignore the bug report} -- if
+you cannot find a reason to reject the report, you must accept
+it.}
+
+
+@ignore
+@c Try omitting this from Bug Squad duties
+
+@subheading Updates / discussion about issues
+
+We try to keep discussions about issues on the tracker, but
+sometimes it spills over onto email. If discussion has ended with
+no patch / resolution and at least @strong{3 days} have passed,
+then either:
+
+@itemize
+
+@item
+Summarize the recent discussion on the tracker, and add a link to
+the original discussion.
+
+@item
+Add the comment @qq{there was some technical discussion which I
+could not understand}, and include a link to the original
+discussion.
+
+We do not expect Bug Squad members to be programmers, or even to
+be moderately-skilled users. Your job is to keep track of issue
+reports; it is @emph{perfectly acceptable} to not understand
+discussions between advanced users and/or developers.
+
+@end itemize
+@end ignore
+
+
+@subheading Regular maintenance
+
+After @strong{every release} (both stable and unstable):
+
+@itemize
+
+@item
+Regression test comparison: if anything has changed suspiciously,
+ask if it was deliberate. The official comparison is online, at:
+
+@c NOTE: leave this here. In this case, it's worth duplicating
+@c the link. -gp
+@example
+@uref{http://lilypond.org/test/}
+@end example
+
+More information is available from in
+@ref{Precompiled regression tests}.
+
+
+@item
+Issues to verify: try to reproduce the bug with the latest
+version; if you cannot reproduce the bug, mark the item
+@qq{Verified} (i.e. @qq{the fix has been verified to work}).
+
+@example
+@uref{http://code.google.com/p/lilypond/issues/list?can=7}
+@end example
+
+A few (approximately 10%) of these fixed issues relate to the
+build system or fundamental architecture changes; there is no way
+for you to verify these. Leave those issues alone; somebody else
+will handle them.
+
+@end itemize
+
+
+@ignore
+@c try omitting from daily tasks for now. -gp
+
+Once every @strong{two weeks} or so:
+
+@itemize
+
+@item
+Check for any incorrectly-classified items in the tracker. This
+generally just means looking at the grid to see any items without
+a Type or Priority.
+
+@item
+Check for any items with @code{label:patch}. If it's been more
+than a week since the last action on the issue, send an email to
+-devel to remind them about it. If the patch was withdrawn for
+more work, then remove the @code{patch} label.
+
+@example
+@uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:patch}
+@end example
+
+@end itemize
+
+
+@subheading Irregular maintenance
+
+@warning{These tasks are a lot of work; gathering more volunteers
+to help is definitely recommended. However, the Bug Squad should
+handle the organization and training of new volunteers.}
+
+Once every year or two:
+
+@itemize
+
+@item
+Checking all regtests: although we have a system for checking the
+regtests between two versions, occasionally a bug will slip
+through the cracks. It is therefore good to manually examine all
+the regtests (compare the images to the text description). More
+information is available from in @ref{Regression tests}.
+
+
+@item
+Checking all issues: we try to mark each Issue @q{fixed} when we
+fix it, but occasionally one or two issues will slip through the
+cracks. It is therefore good to check all Issues. If you see the
+same (broken) output as the initial report, then simply post a
+@qq{Problem still exists in 2.x.y} message to the issue.
+
+@end itemize
+
+@end ignore
@node Issue classification
@section Issue classification
-Status values:
+The Bug Squad should classify issues according to the guidelines
+given by developers. Every issue should have a Status, Type, and
+Priority; the other fields are optional.
+
+@subheading Status (mandatory)
+
+Open issues:
@itemize
@item
-New: the item was added by a non-member. Should be reviewed by
-the Bug Meister.
+New: the item was added by a non-member, despite numerous warnings
+not to do this. Should be reviewed by a member of the Bug Squad.
@item
-Accepted: the Bug Meister added it, or reviewed the item.
+Accepted: the Bug Squad added it, or reviewed the item.
@item
-Started: a programmer is working on a bugfix. (used infrequently,
-but should be used more often)
+Started: a contributor is working on a fix. Owner should change
+to be this contributor.
@end itemize
-Closed status values:
+
+Closed issues:
@itemize
Duplicate: issue already exists in the tracker.
@item
-Fixed: programmer claims to have fixed the bug. The Bug Meister
-should check the input code in an official binary release.
+Fixed: a contributor claims to have fixed the bug. The Bug
+Squad should check the fix with the next official binary release
+(not by compiling the source from git). Owner should be set to
+that contributor.
@item
-Verified: Bug Meister has confirmed that the issue is closed.
+Verified: Bug Squad has confirmed that the issue is closed. This
+means that nobody should ever need look at the report again -- if
+there is any information in the issue that should be kept, open a
+new issue for that info.
@end itemize
-Type labels:
+
+@subheading Owner (optional)
+
+Newly-added issues should have @emph{no owner}. When a
+contributor indicates that he has Started or Fixed an item, he
+should become the owner.
+
+
+@subheading Type (mandatory)
+
+The issue's Type should be the first relevant item in this list.
@itemize
@item
-Type-Defect: a problem that requires no (or very little) new code
-to fix.
+Type-Collision: overlapping notation.
+
+@item
+Type-Defect: a problem in the core program. (the @code{lilypond}
+binary, scm files, fonts, etc).
@item
-Type-Enhancement: a problem (or new feature) that requries a
-significant amount of new code.
+Type-Documentation: inaccurate, missing, confusing, or desired
+additional info. Must be fixable by editing a texinfo, ly, or scm
+file.
@item
-Type-Collision: overlapping notation. (this label takes
-precedence over -Defect and -Enhancement)
+Type-Build: problem or desired features in the build system. This
+includes the makefiles, stepmake, python scripts, and GUB.
@item
-Type-Task: not used, I think. TODO: start using it or delete it.
+Type-Scripts: problem or desired feature in the non-build-system
+scripts. Mostly used for convert-ly, lilypond-book, etc.
+@item
+Type-Enhancement: a feature request for the core program. The
+distinction between enhancement and defect isn't extremely clear;
+when in doubt, mark it as enhancement.
@item
-Type-Other: anything else. TODO: start using it or delete it.
+Type-Other: anything else.
@end itemize
-Priority labels:
+
+@subheading Priority (mandatory)
+
+Currently, only Critical items will block a stable release.
@itemize
@item
-Priority-High: lilypond segfaults.
+Priority-Critical: lilypond segfaults, or a regression occurred
+within the last two stable versions. (i.e. when developing 2.13,
+any regression against 2.12 or 2.10 counts)
@item
-Priority-Regression: it used to work.
+Priority-High: highly embarrassing items, and any regression
+against a version earlier than two stable versions (i.e. when
+developing 2.13, any regression against 2.8 or earlier). This
+level is also used for issues which produce no output and fail to
+give the user a clue about what's wrong.
@item
-Priority-Medium: normal priority; this is the highest priority a
-non-crashing, non-regression bug report can receive.
-(regardless of the perceived importance)
+Priority-Medium: normal priority.
@item
Priority-Low: less important than normal.
@end itemize
-Opsys lables: pretty self-explanatory.
+The difference between Priority-Medium and Priority-Low is not
+well-defined, both in this policy and in practice. The only
+answer we can give at the moment is @qq{look at existing items in
+of the same type, and try to guess whether the priority is closer
+to the Medium items or Low items}. We're aware of the ambiguity,
+and won't complain if somebody picks a @q{wrong} value for
+Medium/Low.
+
+
+@subheading Opsys (optional)
+
+Issues that only affect specific operating systems.
+
+
+@subheading Other items (optional)
-Other lables:
+Other labels:
@itemize
+@item
+Regression: it used to @strong{deliberately} work in an earlier
+stable release. If the earlier output was accidental (i.e. we
+didn't try to stop a collision, but it just so happened that two
+grobs didn't collide), then breaking it does not count as a
+regression.
+
@item
Patch: a patch to fix an issue is attached.
issue should also have an estimated time in a comment.
@item
-Security: not used. TODO: delete, unless anybody is serious about
-this.
-
-@item
-Performance: not used. TODO: delete.
-
-@item
-Usability: not used. TODO: delete.
+Maintainability: hinders development of LilyPond. For example,
+improvements to the build system, or @qq{helper} python scripts.
@item
-Maintainability: hinders developent of LilyPond. For example,
-improvements to the build system, or @qq{helper} python scripts.
+Bounty: somebody is willing to pay for the fix. Only add this tag
+if somebody has offered an exact figure in US dollars or euros.
@item
-Bounty: somebody is willing to pay for the fix.
+Warning: graphical output is fine, but lilypond prints a
+false/misleading warning message. Alternately, a warning should
+be printed (such as a bar line error), but was not. Also applies
+to warnings when compiling the source code or generating
+documentation.
@item
-Engraving-nitpick: output is not beautiful, but not strictly
-speaking @qq{wrong}. For example, a slur shape which does not
-collide with any notation, but looks ugly.
+Security: might potentially be used.
@item
-Warning-nitpick: graphical output is fine, but lilypond prints a
-false/misleading warning message.
+Performance: might potentially be used.
@end itemize
+If you particularly want to add an label not in the list, go
+ahead, but this is not recommended.
+
@node Adding issues to the tracker
@section Adding issues to the tracker
-FIXME: prettify.
+@warning{This should only be done by the Bug Squad or experienced
+developers. Normal users should not do this; instead, they should
+follow the guidelines for @rweb{Bug reports}.}
+
+In order to assign labels to issues, Bug Squad members should log
+in to their google account before adding an item.
-only done by Bug Meister, unless you're really certain you know
-what you're doing.
+@subsubheading Normal issues
+@enumerate
+@item
+Check if the issue falls into any previous category given on the
+relevant checklists in @ref{Bug Squad checklists}. If in doubt,
+add a new issue for a report. We would prefer to have some
+incorrectly-added issues rather than lose information that should
+have been added.
-@node Checking and verifying issues
-@section Checking and verifying issues
+@item
+Add the issue and classify it according to the guidelines in
+@ref{Issue classification}. In particular, the item should have
+@code{Status}, @code{Type-}, and @code{Priority-} labels.
-After each release (both stable and unstable):
+Include output with the first applicable method:
@itemize
@item
-Regression test comparison: check the relevant version in
-@uref{http://lilypond.org/test/}.
+If the issue has a notation example which fits in one system,
+generate a small @file{bug.preview.png} file with:
+
+@example
+lilypond -dpreview bug.ly
+@end example
@item
-Issues to verify:
-@uref{http://code.google.com/p/lilypond/issues/list?can=7}.
+If the issue has an example which requires more than one system
+(i.e. a spacing bug), generate a @file{bug.png} file with:
-@end itemize
+@example
+lilypond --png bug.ly
+@end example
-Once every year or so:
+@item
+If the issue requires multi-page output, then generate a
+@file{bug.pdf} file with the normal:
-@itemize
+@example
+lilypond --png bug.ly
+@end example
-@item
-Checking all regtests: although we have a system for checking the
-regtests between two versions, occasionally a bug will slip
-through the cracks. It is therefore good to manually examine all
-the regtests twice a year or so (compare the images to the text
-description).
+@end itemize
@item
-Checking all issues: we try to mark each Issue @q{fixed} when we
-fix it, but occasionally one or two issues will slip through the
-cracks. It is therefore good to check all Issues. If you see the
-same (broken) output as the initial report, then simply post a
-"Problem still exists in 2.x.y" message to the issue.
+After adding the issue, please send a response email to the same
+group(s) that the initial patch was sent to. If the initial email
+was sent to multiple mailing lists (such as both @code{user} and
+@code{bugs}), then reply to all those mailing lists as well. The
+email should contain a link to the issue you just added.
+
+@end enumerate
+
+
+@subsubheading Patch reminders
+
+@warning{This is not a Bug Squad responsibility; we have a
+separate person handling this task.}
+
+There is a special category of issues: reminders of an existing
+patch. These should be added if a patch has been sent to a
+lilypond mailing list (generally @code{lilypond-devel}, but they
+sometimes appear on @code{bug-lilypond} as well) and has had no
+discussion for at least @strong{3 days}. Do not add issues for
+patches under active discussion.
+
+Before adding a patch-reminder issue, do a quick check to see if
+it was pushed without sending any email. This can be checked for
+searching for relevant terms (from the patch subject or commit
+message) on the webgit page:
+
+@example
+@uref{http://git.savannah.gnu.org/gitweb/?p=lilypond.git}
+@end example
+
+After adding the issue, please send a response email to the same
+group(s) that the initial patch was sent to. If the initial email
+was sent to multiple mailing lists (such as both @code{bugs} and
+@code{devel}), then reply to all those mailing lists as well. The
+email should contain a link to the issue you just added.
-@end itemize
@node Summary of project status
The best overview of our current status is given by the grid view:
@example
-@uref{
-http://code.google.com/p/lilypond/issues/list?mode=grid&y=Priority&x=Type&cells=ids
-}
+@uref{http://code.google.com/p/lilypond/issues/list?mode=grid&y=Priority&x=Type&cells=ids}
@end example
Also of interest might be the issues hindering future development:
@example
-@uref{
-http://code.google.com/p/lilypond/issues/list?can=2&q=label:Maintainability&mode=grid&y=Priority&x=Type&cells=ids
-}
+@uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:Maintainability&mode=grid&y=Priority&x=Type&cells=ids}
@end example
Finally, issues tagged with @code{Frog} indicates a task suitable
else about LilyPond development.
@example
-@uref{
-http://code.google.com/p/lilypond/issues/list?can=2&q=label:Frog&mode=grid&y=Priority&x=Type&cells=ids
-}
+@uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:Frog&mode=grid&y=Priority&x=Type&cells=ids}
@end example
@node Finding the cause of a regression
@section Finding the cause of a regression
+@warning{This is not a @qq{simple} task; it requires a fair amount
+of technical knowledge.}
+
Git has special functionality to help tracking down the exact
commit which causes a problem. See the git manual page for
-@code{git bisect}.
-
-This is a job that non-programmers can do; once a problematic
-commit is identified, the programmers' job is much easier. In
-fact, for most regression bugs, the majority of the time is spent
-simply finding the problematic commit.
-
+@code{git bisect}. This is a job that non-programmers can do,
+although it requires familiarity with git, ability to compile
+LilyPond, and generally a fair amount of technical knowledge. An
+in-depth explanation of this process will not be given here.
+
+Even if you are not familiar with git or are not able to compile
+LilyPond you can still help to narrow down the cause of a
+regression simply by downloading the binary releases of different
+LilyPond versions and testing them for the regression. Knowing
+which version of LilyPond first exhibited the regression is
+helpful to a developer as it shortens the @code{git bisect}
+procedure described above.
+
+Once a problematic commit is identified, the programmers' job is
+much easier. In fact, for most regression bugs, the majority of
+the time is spent simply finding the problematic commit.
+
+More information is in @ref{Regression tests}.
projects / lilypond.git / blobdiff