1 @c -*- coding: utf-8; mode: texinfo; -*-
5 This chapter deals with defects, feature requests, and
6 miscellaneous development tasks.
9 * Introduction to issues::
11 * Bug Squad checklists::
12 * Issue classification::
13 * Adding issues to the tracker::
15 * Summary of project status::
19 @node Introduction to issues
20 @section Introduction to issues
22 @warning{Unless otherwise specified, all the tasks in this chapter
23 are @qq{simple} tasks: they can be done by a normal user with
24 nothing more than a web browser, email, and lilypond.}
26 @qq{Issues} isn't just a politically-correct term for @qq{bug}.
27 We use the same tracker for feature requests and code TODOs, so
28 the term @qq{bug} wouldn't be accurate. Despite the difference
29 between @qq{issue} and @qq{bug}, we call our team of contributors
30 who organize issues the @emph{Bug Squad}.
32 The Bug Squad is mainly composed of non-programmers -- their job
33 is to @emph{organize} issues, not solve them. Their duties
34 include removing false bug reports, ensuring that any real bug
35 report contains enough information for developers, and checking
36 that a developer's fix actually resolves the problem.
38 New volunteers for the Bug Squad should contact the
39 @ref{Meisters, Bug Meister}.
43 @section Bug Squad setup
45 We highly recommend that you configure your email to use effective
46 sorting; this can reduce your workload @emph{immensely}. The
47 email folders names were chosen specifically to make them work if
48 you sort your folders alphabetically.
53 Read every section of this chapter, @ref{Issues}.
56 If you do not have one already, create a gmail account and send
57 the email address to the @ref{Meisters, Bug Meister}.
60 Subscribe your gmail account to @code{bug-lilypond}.
63 Configure your google code account:
68 Wait until your gmail account is listed in:
71 @uref{http://code.google.com/p/lilypond/people/list}
75 Sign in to google code by clicking in the top-right corner of:
78 @uref{http://code.google.com/p/lilypond/issues/list}
81 You cannot log on if you have Google Sharing enabled
82 @uref{http://www.googlesharing.net/}.
85 Go to your @qq{Profile}, and select @qq{Settings}.
88 Scroll down to @qq{Issue change notification}, and make sure that
89 you have @emph{selected} @qq{If I starred the issue}.
94 Configure your email client:
99 Any email sent with your gmail address in the @code{To:} or
100 @code{CC:} fields should go to a @code{bug-answers} folder.
102 When setting up your filtering rules, be aware that Google Code
103 might use different versions of your email address, such as ones
104 ending in @code{@@googlemail.com} or @code{@@gmail.com}.
107 Any other email either from, or CC'd to,
110 lilypond@@googlecode.com
114 should go into a separate @code{bug-ignore} folder. Alternately,
115 you may automatically delete these emails.
117 You will @strong{not read} these emails as part of your Bug Squad
118 duties. If you are curious, go ahead and read them later, but it
119 does @strong{not} count as Bug Squad work.
122 Any other email sent to (or CC'd to):
129 should go into a separate @code{bug-current} folder.
136 @node Bug Squad checklists
137 @section Bug Squad checklists
139 When you do Bug Squad work, start at the top of this page and work
140 your way down. Stop when you've done 20 minutes.
142 Please use the email sorting described in @ref{Bug Squad setup}.
143 This means that (as Bug Squad members) you will only ever respond
144 to emails sent or CC'd to the @code{bug-lilypond} mailing list.
147 @subsubheading Emails to you personally
149 You are not expected to work on Bug Squad matters outside of your
150 20 minutes, but sometimes a confused user will send a bug report
151 (or an update to a report) to you personally. If that happens,
152 please forward such emails to the @code{bug-lilypond} list so that
153 the currently-active Bug Squad member(s) can handle the message.
156 @subsubheading Daily schedule
158 @c spacing is deliberate to help reinforce the "cyclic" nature
171 @subsubheading Emails to @code{bug-answers}
173 Some of these emails will be comments on issues that you added to
177 If they are asking for more information, give the additional
181 If the email says that the issue was classified in some other
182 manner, read the rationale given and take that into account for
183 the next issue you add.
186 Otherwise, move them to your @code{bug-ignore} folder.
190 Some of these emails will be discussions about Bug Squad work;
194 @subsubheading Emails to @code{bug-current}
196 Dealing with these emails is your main task. Your job is to get
197 rid of these emails in the first method which is applicable:
201 If the email has already been handled by a Bug Squad member (i.e.
202 check to see who else has replied to it), delete it.
205 If the email is a question about how to use LilyPond, reply with
209 For questions about how to use LilyPond, please read our
210 documentation available from:
211 @uref{http://lilypond.org/website/manuals.html}
212 or ask the lilypond-user mailing list.
216 If the email mentions @qq{the latest git}, or any version number
217 that has not yet been officially released, forward it to
218 @code{lilypond-devel}.
221 If a bug report is not in the form of a Tiny example, direct the
222 user to resubmit the report with this response:
225 I'm sorry, but due to our limited resources for handling bugs, we
226 can only accept reports in the form of Tiny examples. Please see
227 step 2 in our bug reporting guidelines:
228 @uref{http://lilypond.org/website/bug-reports.html}
232 If anything is unclear, ask the user for more information.
234 How does the graphical output differ from what the user expected?
235 What version of lilypond was used (if not given) and operating
236 system (if this is a suspected cause of the problem)? In short,
237 if you cannot understand what the problem is, ask the user to
238 explain more. It is the user's responsibility to explain the
239 problem, not your responsibility to understand it.
242 If the behavior is expected, the user should be told to read the
246 I believe that this is the expected behaviour -- please read our
247 documentation about this topic. If you think that it really is a
248 mistake, please explain in more detail. If you think that the
249 docs are unclear, please suggest an improvement as described by
250 @qq{Simple tasks -- Documentation} on:
251 @uref{http://lilypond.org/website/help-us.html}
255 If the issue already exists in the tracker, send an email to that
259 This issue has already been reported; you can follow the
260 discussion and be notified about fixes here:
264 (copy+paste the google code issue URL)
267 Accept the report as described in
268 @ref{Adding issues to the tracker}.
272 All emails should be CC'd to the @code{bug-lilypond} list so that
273 other Bug Squad members know that you have processed the email.
275 @warning{There is no option for @qq{ignore the bug report} -- if
276 you cannot find a reason to reject the report, you must accept
281 @c Try omitting this from Bug Squad duties
283 @subheading Updates / discussion about issues
285 We try to keep discussions about issues on the tracker, but
286 sometimes it spills over onto email. If discussion has ended with
287 no patch / resolution and at least @strong{3 days} have passed,
293 Summarize the recent discussion on the tracker, and add a link to
294 the original discussion.
297 Add the comment @qq{there was some technical discussion which I
298 could not understand}, and include a link to the original
301 We do not expect Bug Squad members to be programmers, or even to
302 be moderately-skilled users. Your job is to keep track of issue
303 reports; it is @emph{perfectly acceptable} to not understand
304 discussions between advanced users and/or developers.
310 @subheading Regular maintenance
312 After @strong{every release} (both stable and unstable):
317 Regression test comparison: if anything has changed suspiciously,
318 ask if it was deliberate. If the text output from LilyPond (the
319 logfile) changes, the differences will be displayed with a +
320 before text added to the logfile and - before any text removed
321 from the logfile. This may or may not be suspicious.
323 There is one test designed to produce output every time the
324 regtests are created. @code{test-output-distance.ly} creates
325 randomly spaced notes and will always have different output if the
326 regtest checker is working.
328 The official comparison is online, at:
330 @c NOTE: leave this here. In this case, it's worth duplicating
333 @uref{http://lilypond.org/test/}
336 More information is available from in
337 @ref{Precompiled regression tests}.
341 Issues to verify: try to reproduce the bug with the latest
342 official GUB version; if you cannot reproduce the bug, mark the
343 item @qq{Verified} (i.e. @qq{the fix has been verified to work}).
346 @uref{http://code.google.com/p/lilypond/issues/list?can=7}
349 A few (approximately 10%) of these fixed issues relate to the
350 build system or fundamental architecture changes; there is no way
351 for you to verify these. Leave those issues alone; somebody else
355 Check for any incorrectly-classified items in the tracker. This
356 generally just means looking at the grid to see any items without
363 @c try omitting from daily tasks for now. -gp
365 @subheading Irregular maintenance
367 @warning{These tasks are a lot of work; gathering more volunteers
368 to help is definitely recommended. However, the Bug Squad should
369 handle the organization and training of new volunteers.}
371 Once every year or two:
376 Checking all regtests: although we have a system for checking the
377 regtests between two versions, occasionally a bug will slip
378 through the cracks. It is therefore good to manually examine all
379 the regtests (compare the images to the text description). More
380 information is available from in @ref{Regression tests}.
384 Checking all issues: we try to mark each Issue @q{fixed} when we
385 fix it, but occasionally one or two issues will slip through the
386 cracks. It is therefore good to check all Issues. If you see the
387 same (broken) output as the initial report, then simply post a
388 @qq{Problem still exists in 2.x.y} message to the issue.
395 @node Issue classification
396 @section Issue classification
398 The Bug Squad should classify issues according to the guidelines
399 given by developers. Every issue should have a Status, Type, and
400 Priority; the other fields are optional.
402 @subheading Status (mandatory)
409 New: the item was added by a non-member, despite numerous warnings
410 not to do this. Should be reviewed by a member of the Bug Squad.
413 Accepted: the Bug Squad added it, or reviewed the item.
416 Started: a contributor is working on a fix. Owner should change
417 to be this contributor.
427 Invalid: issue should not have been added in the current state.
430 Duplicate: issue already exists in the tracker.
433 Fixed: a contributor claims to have fixed the bug. The Bug
434 Squad should check the fix with the next official binary release
435 (not by compiling the source from git). Owner should be set to
439 Verified: Bug Squad has confirmed that the issue is closed. This
440 means that nobody should ever need look at the report again -- if
441 there is any information in the issue that should be kept, open a
442 new issue for that info.
447 @subheading Owner (optional)
449 Newly-added issues should have @emph{no owner}. When a
450 contributor indicates that he has Started or Fixed an item, he
451 should become the owner.
454 @subheading Type (mandatory)
456 The issue's Type should be the first relevant item in this list.
461 Type-Collision: overlapping notation.
464 Type-Defect: a problem in the core program. (the @code{lilypond}
465 binary, scm files, fonts, etc).
468 Type-Documentation: inaccurate, missing, confusing, or desired
469 additional info. Must be fixable by editing a texinfo, ly, or scm
473 Type-Build: problem or desired features in the build system. This
474 includes the makefiles, stepmake, python scripts, and GUB.
477 Type-Scripts: problem or desired feature in the non-build-system
478 scripts. Mostly used for convert-ly, lilypond-book, etc.
481 Type-Enhancement: a feature request for the core program. The
482 distinction between enhancement and defect isn't extremely clear;
483 when in doubt, mark it as enhancement.
486 Type-Other: anything else.
491 @subheading Priority (mandatory)
493 Currently, only Critical items will block a stable release.
498 Priority-Critical: LilyPond segfaults, a regression (see below)
499 against a previous stable version or a regression against a fix
500 developed for this version. This does not apply where the
501 @qq{regression} occurred because a feature was removed
502 deliberately - this is not a bug.
505 Priority-High: An issue which produces output which does not
506 accurately reflect the input (e.g. where the user would expect
507 an accidental, but none is shown) or which produces aesthetically
508 poor output in a situation which could be expected to crop up
509 frequently in real-world music. It should not be used where the
510 problem can be avoided with a simple workaround. It can also
511 be used to flag where new code in a development version is not
512 functioning as it should. This level is also used for issues
513 which produce no output and fail to give the user a clue about
517 Priority-Medium: Normal priority - use this as the default.
520 Priority-Low: A minor problem which produces slightly undesirable
521 output, or which will only occur in contrived examples, or which
522 is very easily worked around.
525 Priority-Postponed: no fix planned. Generally used for things
526 which nobody wants to touch.
530 Note that these are initial classifications and can be subject
531 to change by others in the development team. For example, a
532 regression against an old stable version which hasn't been
533 noticed for a long time and which is unlikely to get fixed could
534 be downgraded from Priority-Critical by one of the programmers.
536 @subheading Opsys (optional)
538 Issues that only affect specific operating systems.
540 @subheading Patch (optional)
542 Normal Bug Squad members should not add or modify Patch issues;
543 leave them to the Patch Meister.
548 Patch-new: the patch has not been checked for @qq{obvious}
549 mistakes. When in doubt, use this tag.
552 Patch-review: the patch has no @qq{obvious} mistakes (as checked
553 by the Patch Meister), and is ready for review from main
556 Developers with git push ability can use this category, skipping
557 over @code{patch-new}.
560 Patch-needs_work: a developer has some concerns about the patch.
561 This does not necessarily mean that the patch must be changed; in
562 some cases, the developer's concerns can be resolved simply by
563 discussion the situation or providing notation examples.
565 If the patch is updated, the category should be changed to
566 @code{patch-new} (for normal contributors) or @code{patch-review}
567 (for developers who are very confident about their patch).
570 Patch-abandoned: the author has not responded to review comments
575 @subheading Other items (optional)
582 Regression: it used to work intentionally in an earlier
583 stable release. If the earlier output was accidental (i.e. we
584 didn't try to stop a collision, but it just so happened that two
585 grobs didn't collide), then breaking it does not count as a
588 To help decide whether the change is a regression, and therefore
589 should be Priority-Critical, please adopt the following process:
594 Are you certain the change is OK? If so, do nothing.
597 Are you certain that the change is bad? Add it to the tracker
598 as a Critical issue, regression.
601 If you're not certain either way, add it to the tracker as a
602 Critical issue, regression but be aware that it may be
603 recategorised or marked invalid.
607 In particular, anything that breaks a regression test is a
611 Frog: the fix is believed to be suitable for a new contributor
612 (does not require a great deal of knowledge about LilyPond). The
613 issue should also have an estimated time in a comment.
616 Maintainability: hinders development of LilyPond. For example,
617 improvements to the build system, or @qq{helper} python scripts.
620 Bounty: somebody is willing to pay for the fix. Only add this tag
621 if somebody has offered an exact figure in US dollars or euros.
624 Warning: graphical output is fine, but lilypond prints a
625 false/misleading warning message. Alternately, a warning should
626 be printed (such as a bar line error), but was not. Also applies
627 to warnings when compiling the source code or generating
631 Security: might potentially be used.
634 Performance: might potentially be used.
638 If you particularly want to add a label not in the list, go
639 ahead, but this is not recommended.
642 @node Adding issues to the tracker
643 @section Adding issues to the tracker
645 @warning{This should only be done by the Bug Squad or experienced
646 developers. Normal users should not do this; instead, they should
647 follow the guidelines for @rweb{Bug reports}.}
649 In order to assign labels to issues, Bug Squad members should log
650 in to their google account before adding an item.
655 Check if the issue falls into any previous category given on the
656 relevant checklists in @ref{Bug Squad checklists}. If in doubt,
657 add a new issue for a report. We would prefer to have some
658 incorrectly-added issues rather than lose information that should
662 Add the issue and classify it according to the guidelines in
663 @ref{Issue classification}. In particular, the item should have
664 @code{Status}, @code{Type-}, and @code{Priority-} labels.
666 Include output with the first applicable method:
671 If the issue has a notation example which fits in one system,
672 generate a small @file{bug.preview.png} file with:
675 lilypond -dpreview bug.ly
679 If the issue has an example which requires more than one system
680 (i.e. a spacing bug), generate a @file{bug.png} file with:
683 lilypond --png bug.ly
687 If the issue requires one or two pages of output, then generate a
688 @file{bug.png} file with the normal:
691 lilypond --png bug.ly
695 Images created as @file{bug.png} may be trimmed to a minimum size
696 by using the @code{trimtagline.sh} script, which can be found at
697 @uref{https://raw.github.com/gperciva/lilypond-extra/master/bug-squad/trimtagline.sh}
700 trimtagline.sh bug.ly
704 If the issue cannot be shown with less than three pages, then
705 generate a @file{bug.pdf} file with:
708 lilypond --pdf bug.ly
711 Note that this is likely to be extremely rare; most bugs should fit
712 into the first two categories above.
718 After adding the issue, please send a response email to the same
719 group(s) that the initial patch was sent to. If the initial email
720 was sent to multiple mailing lists (such as both @code{user} and
721 @code{bugs}), then reply to all those mailing lists as well. The
722 email should contain a link to the issue you just added.
729 @section Patch handling
731 @warning{This is not a Bug Squad responsibility; we have a
732 separate person handling this task.}
734 There is a single Patch Meister, and a number of Patch Helpers
735 (rename this?). The list of known patches awaiting review is:
738 @uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:patch&sort=patch}
742 @subheading Helpers: adding patches
744 The primary duty is to add patches to the google tracker; we have
745 a bad track record of losing patches in email. Patches generally
746 come to the @code{lilypond-devel} mailing list, but are sometimes
747 sent to @code{bug-lilypond}, @code{lilypond-users}, or
748 @code{frogs} mailing list instead.
752 Unless a patch is clearly in response to an existing issue, add a
753 new issue with the @code{Patch-new} label and a link to the patch
754 (either on the mailing list archives or the codereview url).
756 Issue numbers are cheap; losing developers because they got fed up
757 with us losing their hard work is expensive.
759 @c if we enter patches immediately, I don't think this is relevant.
762 Before adding a patch-reminder issue, do a quick check to see if
763 it was pushed without sending any email. This can be checked for
764 searching for relevant terms (from the patch subject or commit
765 message) on the webgit page:
768 @uref{http://git.savannah.gnu.org/gitweb/?p=lilypond.git}
773 If the patch is clearly in response to an existing issue, then
774 update that issue with the @code{Patch-new} label and a link to
775 the patch (either on the mailing list archives or the codereview
779 After adding the issue, please send a response email to the same
780 group(s) that the initial patch was sent to.
782 If the initial email was sent to multiple mailing lists (such as
783 both @code{bugs} and @code{devel}), then reply to all those
784 mailing lists as well. The email should contain a link to the
785 issue you just added.
789 @subheading Helpers: @code{Patch-review} label
791 The secondary duty is to do make sure that every issue in the
792 tracker with a @code{Patch-review} label has passed these
797 Applies automatically to git master.
799 It's ok to have offsets, but not conflicts.
802 Regtest comparison looks ok; no unexpected changes.
805 Descriptive subject line.
807 Avoid subjects like @qq{fixes 123}; instead write @qq{Doc: discuss
808 stacking-dir for BassFigureAlignment (fix 123)}.
811 Compiles docs from scratch. Only check this if you have reason to
812 suspect it might not work.
817 Check code indentation and style. This should be easier post-GOP
818 when we have a better-defined code style.
823 @subheading Patch Meister
825 The Patch Meister will:
830 send @qq{countdown} emails to
831 @code{lilypond-devel} when patches appear to be ready.
834 send general requests to review patches, or even nasty requests to
838 downgrade patches from @code{Patch-review} to
839 @code{Patch-needs_work} as appropriate.
842 downgrade patches from @code{Patch-needs_work} to
843 @code{Patch-abandoned} if no actions have been taken in four
851 @node Summary of project status
852 @section Summary of project status
854 @subsubheading Project overview
856 Grid view provides the best overview:
859 @uref{http://code.google.com/p/lilypond/issues/list?mode=grid&y=Priority&x=Type&cells=ids}
862 @subsubheading Hindering development
864 These issues stop or slow development work:
867 @uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:Maintainability}
870 @subsubheading Easy tasks
872 Issues tagged with @code{Frog} indicates a task suitable for a
873 relatively new contributor. The time given is a quick
874 (inaccurate) estimate of the time required for somebody who is
875 familiar with material in this manual, but does not know anything
876 else about LilyPond development.
879 @uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:Frog}
882 @subsubheading Patches to review
884 Patches which have no @qq{obvious} problems:
887 @uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:patch-review}