X-Git-Url: https://git.donarmstrong.com/?a=blobdiff_plain;f=Documentation%2Fcontributor%2Fissues.itexi;h=5ea4b952d349831514013941d66746f47e8455cd;hb=c25520ce458d8b035b965980c87f424f9c5b3eab;hp=9756d46f4d61684f2e6f6c66fc1d531816afe0be;hpb=a5fd30e7288eae7bc4417cff39a53bb7d75ecb60;p=lilypond.git diff --git a/Documentation/contributor/issues.itexi b/Documentation/contributor/issues.itexi index 9756d46f4d..5ea4b952d3 100644 --- a/Documentation/contributor/issues.itexi +++ b/Documentation/contributor/issues.itexi @@ -7,38 +7,389 @@ miscellaneous development tasks. @menu * Introduction to issues:: +* Bug Squad setup:: +* Bug Squad checklists:: * Issue classification:: * Adding issues to the tracker:: -* Checking and verifying issues:: +* Patch handling:: * Summary of project status:: -* Finding the cause of a regression:: @end menu @node Introduction to issues @section Introduction to issues -First, @qq{issue} isn't just a politically-correct term for -@qq{bug}. We use the same tracker for feature requests and code -TODOs, so the term @qq{bug} wouldn't be accurate. - -Second, the classification of what counts as a bug vs. feature -request, and the priorities assigned to bugs, are a matter of -concern @strong{for developers only}. The Bug Squad should -classify issues according to the guidelines given by developers. - @warning{Unless otherwise specified, all the tasks in this chapter are @qq{simple} tasks: they can be done by a normal user with nothing more than a web browser, email, and lilypond.} +@qq{Issues} isn't just a politically-correct term for @qq{bug}. +We use the same tracker for feature requests and code TODOs, so +the term @qq{bug} wouldn't be accurate. Despite the difference +between @qq{issue} and @qq{bug}, we call our team of contributors +who organize issues the @emph{Bug Squad}. + +The Bug Squad is mainly composed of non-programmers -- their job +is to @emph{organize} issues, not solve them. Their duties +include removing false bug reports, ensuring that any real bug +report contains enough information for developers, and checking +that a developer's fix actually resolves the problem. + +New volunteers for the Bug Squad should contact the +@ref{Meisters, Bug Meister}. + + +@node Bug Squad setup +@section Bug Squad setup + +We highly recommend that you configure your email to use effective +sorting; this can reduce your workload @emph{immensely}. The +email folders names were chosen specifically to make them work if +you sort your folders alphabetically. + +@enumerate + +@item +Read every section of this chapter, @ref{Issues}. + +@item +If you do not have one already, create a gmail account and send +the email address to the @ref{Meisters, Bug Meister}. + +@item +Subscribe your gmail account to @code{bug-lilypond}. + +@item +Configure your google code account: + +@enumerate + +@item +Wait until your gmail account is listed in: + +@example +@uref{http://code.google.com/p/lilypond/people/list} +@end example + +@item +Sign in to google code by clicking in the top-right corner of: + +@example +@uref{http://code.google.com/p/lilypond/issues/list} +@end example + +You cannot log if you have Google Sharing +@uref{http://www.googlesharing.net/} enabled. + +@item +Go to your @qq{Profile}, and select @qq{Settings}. + +@item +Scroll down to @qq{Issue change notification}, and make sure that +you have @emph{selected} @qq{If I starred the issue}. + +@end enumerate + +@item +Configure your email client: + +@enumerate + +@item +Any email sent with your gmail address in the @code{To:} or +@code{CC:} fields should go to a @code{bug-answers} folder. + +When setting up your filtering rules, be aware that Google Code +might use different versions of your email address, such as ones +ending in @code{@@googlemail.com} or @code{@@gmail.com}. + +@item +Any other email either from, or CC'd to, + +@example +lilypond@@googlecode.com +@end example + +@noindent +should go into a separate @code{bug-ignore} folder. Alternately, +you may automatically delete these emails. + +You will @strong{not read} these emails as part of your Bug Squad +duties. If you are curious, go ahead and read them later, but it +does @strong{not} count as Bug Squad work. + +@item +Any other email sent to (or CC'd to): + +@example +bug-lilypond +@end example + +@noindent +should go into a separate @code{bug-current} folder. + +@end enumerate + +@end enumerate + + +@node Bug Squad checklists +@section Bug Squad checklists + +When you do Bug Squad work, start at the top of this page and work +your way down. Stop when you've done 15 minutes. + +Please use the email sorting described in @ref{Bug Squad setup}. +This means that (as Bug Squad members) you will only ever respond +to emails sent or CC'd to the @code{bug-lilypond} mailing list. + + +@subsubheading Emails to you personally + +You are not expected to work on Bug Squad matters outside of your +15 minutes, but sometimes a confused user will send a bug report +(or an update to a report) to you personally. If that happens, +please forward such emails to the @code{bug-lilypond} list so that +the currently-active Bug Squad member(s) can handle the message. + + +@subsubheading Daily schedule + +The Bug Meister is omitted from the daily schedule. + +@example +Sunday: Colin +Monday: Dmytro +Tuesday: James Bailey +Wednesday: Ralph +Thursday: Patrick +Friday: Urs +Saturday: Kieren +@end example + + +@subsubheading Emails to @code{bug-answers} + +Some of these emails will be comments on issues that you added to +the tracker. + +@itemize +If they are asking for more information, give the additional +information. + +@item +If the email says that the issue was classified in some other +manner, read the rationale given and take that into account for +the next issue you add. + +@item +Otherwise, move them to your @code{bug-ignore} folder. + +@end itemize + +Some of these emails will be discussions about Bug Squad work; +read those. + + +@subsubheading Emails to @code{bug-current} + +Dealing with these emails is your main task. Your job is to get +rid of these emails in the first method which is applicable: + +@enumerate +@item +If the email has already been handled by a Bug Squad member (i.e. +check to see who else has replied to it), delete it. + +@item +If the email is a question about how to use LilyPond, reply with +this response: + +@example +For questions about how to use LilyPond, please read our +documentation available from: + @uref{http://lilypond.org/website/manuals.html} +or ask the lilypond-user mailing list. +@end example + +@item +If the email mentions @qq{the latest git}, or any version number +that has not yet been officially released, forward it to +@code{lilypond-devel}. + +@item +If a bug report is not in the form of a Tiny example, direct the +user to resubmit the report with this response: + +@example +I'm sorry, but due to our limited resources for handling bugs, we +can only accept reports in the form of Tiny examples. Please see +step 2 in our bug reporting guidelines: + @uref{http://lilypond.org/website/bug-reports.html} +@end example + +@item +If anything is unclear, ask the user for more information. + +How does the graphical output differ from what the user expected? +What version of lilypond was used (if not given) and operating +system (if this is a suspected cause of the problem)? In short, +if you cannot understand what the problem is, ask the user to +explain more. It is the user's responsibility to explain the +problem, not your responsibility to understand it. + +@item +If the behavior is expected, the user should be told to read the +documentation: + +@example +I believe that this is the expected behaviour -- please read our +documentation about this topic. If you think that it really is a +mistake, please explain in more detail. If you think that the +docs are unclear, please suggest an improvement as described by +@qq{Simple tasks -- Documentation} on: + @uref{http://lilypond.org/website/help-us.html} +@end example + +@item +If the issue already exists in the tracker, send an email to that +effect: + +@example +This issue has already been reported; you can follow the +discussion and be notified about fixes here: +@end example + +@noindent +(copy+paste the google code issue URL) + +@item +Accept the report as described in +@ref{Adding issues to the tracker}. + +@end enumerate + +All emails should be CC'd to the @code{bug-lilypond} list so that +other Bug Squad members know that you have processed the email. + +@warning{There is no option for @qq{ignore the bug report} -- if +you cannot find a reason to reject the report, you must accept +it.} + + +@ignore +@c Try omitting this from Bug Squad duties + +@subheading Updates / discussion about issues + +We try to keep discussions about issues on the tracker, but +sometimes it spills over onto email. If discussion has ended with +no patch / resolution and at least @strong{3 days} have passed, +then either: + +@itemize + +@item +Summarize the recent discussion on the tracker, and add a link to +the original discussion. + +@item +Add the comment @qq{there was some technical discussion which I +could not understand}, and include a link to the original +discussion. + +We do not expect Bug Squad members to be programmers, or even to +be moderately-skilled users. Your job is to keep track of issue +reports; it is @emph{perfectly acceptable} to not understand +discussions between advanced users and/or developers. + +@end itemize +@end ignore + + +@subheading Regular maintenance + +After @strong{every release} (both stable and unstable): + +@itemize + +@item +Regression test comparison: if anything has changed suspiciously, +ask if it was deliberate. The official comparison is online, at: + +@c NOTE: leave this here. In this case, it's worth duplicating +@c the link. -gp +@example +@uref{http://lilypond.org/test/} +@end example + +More information is available from in +@ref{Precompiled regression tests}. + + +@item +Issues to verify: try to reproduce the bug with the latest +official GUB version; if you cannot reproduce the bug, mark the +item @qq{Verified} (i.e. @qq{the fix has been verified to work}). + +@example +@uref{http://code.google.com/p/lilypond/issues/list?can=7} +@end example + +A few (approximately 10%) of these fixed issues relate to the +build system or fundamental architecture changes; there is no way +for you to verify these. Leave those issues alone; somebody else +will handle them. + +@item +Check for any incorrectly-classified items in the tracker. This +generally just means looking at the grid to see any items without +a Type or Priority. + +@end itemize + + +@ignore +@c try omitting from daily tasks for now. -gp + +@subheading Irregular maintenance + +@warning{These tasks are a lot of work; gathering more volunteers +to help is definitely recommended. However, the Bug Squad should +handle the organization and training of new volunteers.} + +Once every year or two: + +@itemize + +@item +Checking all regtests: although we have a system for checking the +regtests between two versions, occasionally a bug will slip +through the cracks. It is therefore good to manually examine all +the regtests (compare the images to the text description). More +information is available from in @ref{Regression tests}. + + +@item +Checking all issues: we try to mark each Issue @q{fixed} when we +fix it, but occasionally one or two issues will slip through the +cracks. It is therefore good to check all Issues. If you see the +same (broken) output as the initial report, then simply post a +@qq{Problem still exists in 2.x.y} message to the issue. + +@end itemize + +@end ignore + @node Issue classification @section Issue classification -Every issue should have a Status and Type; the other fields are -optional. +The Bug Squad should classify issues according to the guidelines +given by developers. Every issue should have a Status, Type, and +Priority; the other fields are optional. -@subheading Status +@subheading Status (mandatory) Open issues: @@ -75,20 +426,22 @@ Squad should check the fix with the next official binary release that contributor. @item -Verified: Bug Squad has confirmed that the issue is closed. +Verified: Bug Squad has confirmed that the issue is closed. This +means that nobody should ever need look at the report again -- if +there is any information in the issue that should be kept, open a +new issue for that info. @end itemize -@subheading Owner +@subheading Owner (optional) Newly-added issues should have @emph{no owner}. When a contributor indicates that he has Started or Fixed an item, he should become the owner. - -@subheading Type +@subheading Type (mandatory) The issue's Type should be the first relevant item in this list. @@ -125,65 +478,124 @@ Type-Other: anything else. @end itemize -@subheading Priority +@subheading Priority (mandatory) Currently, only Critical items will block a stable release. @itemize @item -Priority-Critical: lilypond segfaults, or a regression occurred -within the last two stable versions. (i.e. when developing 2.13, -any regression against 2.12 or 2.10 counts) +Priority-Critical: LilyPond segfaults, a regression (see below) +against a previous stable version or a regression against a fix +developed for this version. This does not apply where the +@qq{regression} occurred because a feature was removed +deliberately - this is not a bug. @item -Priority-High: highly embarrassing items, and any regression -against a version earlier than two stable versions (i.e. when -developing 2.13, any regression against 2.8 or earlier). This -level is also used for issues which produce no output and fail to -give the user a clue about what's wrong. +Priority-High: An issue which produces output which does not +accurately reflect the input (e.g. where the user would expect +an accidental, but none is shown) or which produces aesthetically +poor output in a situation which could be expected to crop up +frequently in real-world music. It should not be used where the +problem can be avoided with a simple workaround. It can also +be used to flag where new code in a development version is not +functioning as it should. This level is also used for issues +which produce no output and fail to give the user a clue about +what's wrong. @item -Priority-Medium: normal priority. +Priority-Medium: Normal priority - use this as the default. @item -Priority-Low: less important than normal. +Priority-Low: A minor problem which produces slightly undesirable +output, or which will only occur in contrived examples, or which +is very easily worked around. @item Priority-Postponed: no fix planned. Generally used for things -like Ancient notation, which nobody wants to touch. +which nobody wants to touch. @end itemize -The difference between Priority-Medium and Priority-Low is not -well-defined, both in this policy and in practice. The only -answer we can give at the moment is @qq{look at existing items in -of the same type, and try to guess whether the priority is closer -to the Medium items or Low items}. We're aware of the ambiguity, -and won't complain if somebody picks a @q{wrong} value for -Medium/Low. - +Note that these are initial classifications and can be subject +to change by others in the development team. For example, a +regression against an old stable version which hasn't been +noticed for a long time and which is unlikely to get fixed could +be downgraded from Priority-Critical by one of the programmers. -@subheading Opsys +@subheading Opsys (optional) Issues that only affect specific operating systems. +@subheading Patch (optional) + +Normal Bug Squad members should not add or modify Patch issues; +leave them to the Patch Meister. + +@itemize + +@item +Patch-new: the patch has not been checked for @qq{obvious} +mistakes. When in doubt, use this tag. + +@item +Patch-review: the patch has no @qq{obvious} mistakes (as checked +by the Patch Meister), and is ready for review from main +developers. + +Developers with git push ability can use this category, skipping +over @code{patch-new}. + +@item +Patch-needs_work: a developer has some concerns about the patch. +This does not necessarily mean that the patch must be changed; in +some cases, the developer's concerns can be resolved simply by +discussion the situation or providing notation examples. + +If the patch is updated, the category should be changed to +@code{patch-new} (for normal contributors) or @code{patch-new} +(for developers who are very confident about their patch). + +@item +Patch-abandoned: the author has not responded to review comments +for a few months. + +@end itemize -@subheading Other items +@subheading Other items (optional) Other labels: @itemize @item -Regression: it used to @strong{deliberately} work in an earlier +Regression: it used to work intentionally in an earlier stable release. If the earlier output was accidental (i.e. we didn't try to stop a collision, but it just so happened that two grobs didn't collide), then breaking it does not count as a regression. +To help decide whether the change is a regression, and therefore +should be Priority-Critical, please adopt the following process: + +@enumerate + @item -Patch: a patch to fix an issue is attached. +Are you certain the change is OK? If so, do nothing. + +@item +Are you certain that the change is bad? Add it to the tracker +as a Critical issue, regression. + +@item +If you're not certain either way, add it to the tracker as a +Critical issue, regression but be aware that it may be +recategorised or marked invalid. + +@end enumerate + +In particular, anything that breaks a regression test is a +regression. @item Frog: the fix is believed to be suitable for a new contributor @@ -191,11 +603,12 @@ Frog: the fix is believed to be suitable for a new contributor issue should also have an estimated time in a comment. @item -Maintainability: hinders developent of LilyPond. For example, +Maintainability: hinders development of LilyPond. For example, improvements to the build system, or @qq{helper} python scripts. @item -Bounty: somebody is willing to pay for the fix. +Bounty: somebody is willing to pay for the fix. Only add this tag +if somebody has offered an exact figure in US dollars or euros. @item Warning: graphical output is fine, but lilypond prints a @@ -212,37 +625,75 @@ Performance: might potentially be used. @end itemize -If you particularly want to add an label not in the list, go +If you particularly want to add a label not in the list, go ahead, but this is not recommended. @node Adding issues to the tracker @section Adding issues to the tracker -This should only be done by the Bug Squad or experienced +@warning{This should only be done by the Bug Squad or experienced developers. Normal users should not do this; instead, they should -follow the guidelines for @rweb{Bug reports}. +follow the guidelines for @rweb{Bug reports}.} -@subsubheading Normal issues +In order to assign labels to issues, Bug Squad members should log +in to their google account before adding an item. + +@enumerate + +@item +Check if the issue falls into any previous category given on the +relevant checklists in @ref{Bug Squad checklists}. If in doubt, +add a new issue for a report. We would prefer to have some +incorrectly-added issues rather than lose information that should +have been added. + +@item +Add the issue and classify it according to the guidelines in +@ref{Issue classification}. In particular, the item should have +@code{Status}, @code{Type-}, and @code{Priority-} labels. + +Include output with the first applicable method: @itemize @item -Check if the issue already exists in the tracker. +If the issue has a notation example which fits in one system, +generate a small @file{bug.preview.png} file with: + +@example +lilypond -dpreview bug.ly +@end example + +@item +If the issue has an example which requires more than one system +(i.e. a spacing bug), generate a @file{bug.png} file with: + +@example +lilypond --png bug.ly +@end example @item -If the initial bug report contains lilypond examples which do not -follow the guidelines in @rweb{Tiny examples}, either ask the -reporter to create such an example, or make one yourself. +If the issue requires one or two pages of output, then generate a +@file{bug.png} file with the normal: -If you are willing to spend the effort, we would prefer it if the -Bug Squad member created a tiny example themselves instead of -asking the user to do so. +@example +lilypond --png bug.ly +@end example @item -Add the issue and classify it according to the guidelines in -@ref{Issue classification}. In particular, the item should have a -@code{Status}, @code{Type-}, and @code{Priority-}. +If the issue cannot be shown with less than three pages, then +generate a @file{bug.pdf} file with: + +@example +lilypond --pdf bug.ly +@end example + +Note that this is likely to be extremely rare; most bugs should fit +into the first two categories above. + + +@end itemize @item After adding the issue, please send a response email to the same @@ -251,18 +702,44 @@ was sent to multiple mailing lists (such as both @code{user} and @code{bugs}), then reply to all those mailing lists as well. The email should contain a link to the issue you just added. -@end itemize +@end enumerate -@subsubheading Patch reminders -There is a special category of issues: reminders of an existing -patch. These should be added if a patch has been sent to a -lilypond mailing list (generally @code{lilypond-devel}, but they -sometimes appear on @code{bug-lilypond} as well) and has had no -discussion for at least @strong{3 days}. Do not add issues for -patches under active discussion. +@node Patch handling +@section Patch handling +@warning{This is not a Bug Squad responsibility; we have a +separate person handling this task.} + +There is a single Patch Meister, and a number of Patch Helpers +(rename this?). The list of known patches awaiting review is: + +@example +@uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:patch&sort=patch} +@end example + + +@subheading Helpers: adding patches + +The primary duty is to add patches to the google tracker; we have +a bad track record of losing patches in email. Patches generally +come to the @code{lilypond-devel} mailing list, but are sometimes +sent to @code{bug-lilypond}, @code{lilypond-users}, or +@code{frogs} mailing list instead. + +@itemize +@item +Unless a patch is clearly in response to an existing issue, add a +new issue with the @code{Patch-new} label and a link to the patch +(either on the mailing list archives or the codereview url). + +Issue numbers are cheap; losing developers because they got fed up +with us losing their hard work is expensive. + +@c if we enter patches immediately, I don't think this is relevant. +@ignore +@item Before adding a patch-reminder issue, do a quick check to see if it was pushed without sending any email. This can be checked for searching for relevant terms (from the patch subject or commit @@ -271,90 +748,110 @@ message) on the webgit page: @example @uref{http://git.savannah.gnu.org/gitweb/?p=lilypond.git} @end example +@end ignore +@item +If the patch is clearly in response to an existing issue, then +update that issue with the @code{Patch-new} label and a link to +the patch (either on the mailing list archives or the codereview +url). + +@item After adding the issue, please send a response email to the same -group(s) that the initial patch was sent to. If the initial email -was sent to multiple mailing lists (such as both @code{bugs} and -@code{devel}), then reply to all those mailing lists as well. The -email should contain a link to the issue you just added. +group(s) that the initial patch was sent to. +If the initial email was sent to multiple mailing lists (such as +both @code{bugs} and @code{devel}), then reply to all those +mailing lists as well. The email should contain a link to the +issue you just added. -@node Checking and verifying issues -@section Checking and verifying issues +@end itemize -After each release (both stable and unstable): +@subheading Helpers: @code{Patch-review} label -@itemize +The secondary duty is to do make sure that every issue in the +tracker with a @code{Patch-review} label has passed these +@qq{obvious} tests: +@itemize @item -Regression test comparison: check the relevant version in -@uref{http://lilypond.org/test/}. +Applies automatically to git master. -@item -Issues to verify: -@uref{http://code.google.com/p/lilypond/issues/list?can=7}. +It's ok to have offsets, but not conflicts. -@end itemize +@item +Regtest comparison looks ok; no unexpected changes. -Once every two weeks or so: +@item +Descriptive subject line. -@itemize +Avoid subjects like @qq{fixes 123}; instead write @qq{Doc: discuss +stacking-dir for BassFigureAlignment (fix 123)}. @item -Check for any incorrectly-classified items in the tracker. This -generally just means looking at the grid to see any items without -a Type or Priority. +Compiles docs from scratch. Only check this if you have reason to +suspect it might not work. @item -Check for any items with @code{label:patch}. If it's been more -than a week since the last action on the issue, send an email to --devel to remind them about it. If the patch was withdrawn for -more work, then remove the @code{patch} label. +(maybe) -@example -@uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:patch} -@end example +Check code indentation and style. This should be easier post-GOP +when we have a better-defined code style. @end itemize -Once every year or two: (yes, these are large tasks; gathering -more volunteers to help is definitely recommended) + +@subheading Patch Meister + +The Patch Meister will: @itemize @item -Checking all regtests: although we have a system for checking the -regtests between two versions, occasionally a bug will slip -through the cracks. It is therefore good to manually examine all -the regtests (compare the images to the text description). +send @qq{countdown} emails to +@code{lilypond-devel} when patches appear to be ready. @item -Checking all issues: we try to mark each Issue @q{fixed} when we -fix it, but occasionally one or two issues will slip through the -cracks. It is therefore good to check all Issues. If you see the -same (broken) output as the initial report, then simply post a -@qq{Problem still exists in 2.x.y} message to the issue. +send general requests to review patches, or even nasty requests to +review patches. + +@item +downgrade patches from @code{Patch-review} to +@code{Patch-needs_work} as appropriate. + +@item +downgrade patches from @code{Patch-needs_work} to +@code{Patch-abandoned} if no actions have been taken in four +weeks. @end itemize + + @node Summary of project status @section Summary of project status -The best overview of our current status is given by the grid view: +@subsubheading Project overview + +Grid view provides the best overview: @example @uref{http://code.google.com/p/lilypond/issues/list?mode=grid&y=Priority&x=Type&cells=ids} @end example -Also of interest might be the issues hindering future development: +@subsubheading Hindering development + +These issues stop or slow development work: @example @uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:Maintainability&mode=grid&y=Priority&x=Type&cells=ids} @end example -Finally, issues tagged with @code{Frog} indicates a task suitable -for a relatively new contributor. The time given is a quick +@subsubheading Easy tasks + +Issues tagged with @code{Frog} indicates a task suitable for a +relatively new contributor. The time given is a quick (inaccurate) estimate of the time required for somebody who is familiar with material in this manual, but does not know anything else about LilyPond development. @@ -363,28 +860,13 @@ else about LilyPond development. @uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:Frog&mode=grid&y=Priority&x=Type&cells=ids} @end example +@subsubheading Patches to review -@node Finding the cause of a regression -@section Finding the cause of a regression - -@warning{This is not a @qq{simple} task; it requires a fair amount -of technical knowledge.} +Patches which have no @qq{obvious} problems: -Git has special functionality to help tracking down the exact -commit which causes a problem. See the git manual page for -@code{git bisect}. This is a job that non-programmers can do, -although it requires familiarity with git, ability to compile -LilyPond, and generally a fair amount of technical knowledge. +@example +@uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:patch-review} +@end example -Even if you are not familiar with git or are not able to compile -LilyPond you can still help to narrow down the cause of a -regression simply by downloading the binary releases of different -LilyPond versions and testing them for the regression. Knowing -which version of LilyPond first exhibited the regression is -helpful to a developer as it shortens the @code{git bisect} -procedure described above. -Once a problematic commit is identified, the programmers' job is -much easier. In fact, for most regression bugs, the majority of -the time is spent simply finding the problematic commit.