1 @c -*- coding: utf-8; mode: texinfo; -*-
5 This chapter deals with defects, feature requests, and
6 miscellaneous development tasks.
9 * Introduction to issues::
11 * Bug Squad checklists::
12 * Issue classification::
13 * Adding issues to the tracker::
15 * Summary of project status::
19 @node Introduction to issues
20 @section Introduction to issues
22 @warning{Unless otherwise specified, all the tasks in this chapter
23 are @qq{simple} tasks: they can be done by a normal user with
24 nothing more than a web browser, email, and lilypond.}
26 @qq{Issues} isn't just a politically-correct term for @qq{bug}.
27 We use the same tracker for feature requests and code TODOs, so
28 the term @qq{bug} wouldn't be accurate. Despite the difference
29 between @qq{issue} and @qq{bug}, we call our team of contributors
30 who organize issues the @emph{Bug Squad}.
32 The Bug Squad is mainly composed of non-programmers -- their job
33 is to @emph{organize} issues, not solve them. Their duties
34 include removing false bug reports, ensuring that any real bug
35 report contains enough information for developers, and checking
36 that a developer's fix actually resolves the problem.
38 New volunteers for the Bug Squad should contact the
39 @ref{Meisters, Bug Meister}.
43 @section Bug Squad setup
45 We highly recommend that you configure your email to use effective
46 sorting; this can reduce your workload @emph{immensely}. The
47 email folders names were chosen specifically to make them work if
48 you sort your folders alphabetically.
53 Read every section of this chapter, @ref{Issues}.
56 If you do not have one already, create a gmail account and send
57 the email address to the @ref{Meisters, Bug Meister}.
60 Subscribe your gmail account to @code{bug-lilypond}.
63 Configure your google code account:
68 Wait until your gmail account is listed in:
71 @uref{http://code.google.com/p/lilypond/people/list}
75 Sign in to google code by clicking in the top-right corner of:
78 @uref{http://code.google.com/p/lilypond/issues/list}
81 You cannot log on if you have Google Sharing enabled
82 @uref{http://www.googlesharing.net/}.
85 Go to your @qq{Profile}, and select @qq{Settings}.
88 Scroll down to @qq{Issue change notification}, and make sure that
89 you have @emph{selected} @qq{If I starred the issue}.
94 Configure your email client:
99 Any email sent with your gmail address in the @code{To:} or
100 @code{CC:} fields should go to a @code{bug-answers} folder.
102 When setting up your filtering rules, be aware that Google Code
103 might use different versions of your email address, such as ones
104 ending in @code{@@googlemail.com} or @code{@@gmail.com}.
107 Any other email either from, or CC'd to,
110 lilypond@@googlecode.com
114 should go into a separate @code{bug-ignore} folder. Alternately,
115 you may automatically delete these emails.
117 You will @strong{not read} these emails as part of your Bug Squad
118 duties. If you are curious, go ahead and read them later, but it
119 does @strong{not} count as Bug Squad work.
122 Any other email sent to (or CC'd to):
129 should go into a separate @code{bug-current} folder.
136 @node Bug Squad checklists
137 @section Bug Squad checklists
139 When you do Bug Squad work, start at the top of this page and work
140 your way down. Stop when you've done 20 minutes.
142 Please use the email sorting described in @ref{Bug Squad setup}.
143 This means that (as Bug Squad members) you will only ever respond
144 to emails sent or CC'd to the @code{bug-lilypond} mailing list.
147 @subsubheading Emails to you personally
149 You are not expected to work on Bug Squad matters outside of your
150 20 minutes, but sometimes a confused user will send a bug report
151 (or an update to a report) to you personally. If that happens,
152 please forward such emails to the @code{bug-lilypond} list so that
153 the currently-active Bug Squad member(s) can handle the message.
156 @subsubheading Daily schedule
162 Thursday: Colin Hall (disambiguation here)
169 @subsubheading Emails to @code{bug-answers}
171 Some of these emails will be comments on issues that you added to
175 If they are asking for more information, give the additional
179 If the email says that the issue was classified in some other
180 manner, read the rationale given and take that into account for
181 the next issue you add.
184 Otherwise, move them to your @code{bug-ignore} folder.
188 Some of these emails will be discussions about Bug Squad work;
192 @subsubheading Emails to @code{bug-current}
194 Dealing with these emails is your main task. Your job is to get
195 rid of these emails in the first method which is applicable:
199 If the email has already been handled by a Bug Squad member (i.e.
200 check to see who else has replied to it), delete it.
203 If the email is a question about how to use LilyPond, reply with
207 For questions about how to use LilyPond, please read our
208 documentation available from:
209 @uref{http://lilypond.org/website/manuals.html}
210 or ask the lilypond-user mailing list.
214 If the email mentions @qq{the latest git}, or any version number
215 that has not yet been officially released, forward it to
216 @code{lilypond-devel}.
219 If a bug report is not in the form of a Tiny example, direct the
220 user to resubmit the report with this response:
223 I'm sorry, but due to our limited resources for handling bugs, we
224 can only accept reports in the form of Tiny examples. Please see
225 step 2 in our bug reporting guidelines:
226 @uref{http://lilypond.org/website/bug-reports.html}
230 If anything is unclear, ask the user for more information.
232 How does the graphical output differ from what the user expected?
233 What version of lilypond was used (if not given) and operating
234 system (if this is a suspected cause of the problem)? In short,
235 if you cannot understand what the problem is, ask the user to
236 explain more. It is the user's responsibility to explain the
237 problem, not your responsibility to understand it.
240 If the behavior is expected, the user should be told to read the
244 I believe that this is the expected behaviour -- please read our
245 documentation about this topic. If you think that it really is a
246 mistake, please explain in more detail. If you think that the
247 docs are unclear, please suggest an improvement as described by
248 @qq{Simple tasks -- Documentation} on:
249 @uref{http://lilypond.org/website/help-us.html}
253 If the issue already exists in the tracker, send an email to that
257 This issue has already been reported; you can follow the
258 discussion and be notified about fixes here:
262 (copy+paste the google code issue URL)
265 Accept the report as described in
266 @ref{Adding issues to the tracker}.
270 All emails should be CC'd to the @code{bug-lilypond} list so that
271 other Bug Squad members know that you have processed the email.
273 @warning{There is no option for @qq{ignore the bug report} -- if
274 you cannot find a reason to reject the report, you must accept
279 @c Try omitting this from Bug Squad duties
281 @subheading Updates / discussion about issues
283 We try to keep discussions about issues on the tracker, but
284 sometimes it spills over onto email. If discussion has ended with
285 no patch / resolution and at least @strong{3 days} have passed,
291 Summarize the recent discussion on the tracker, and add a link to
292 the original discussion.
295 Add the comment @qq{there was some technical discussion which I
296 could not understand}, and include a link to the original
299 We do not expect Bug Squad members to be programmers, or even to
300 be moderately-skilled users. Your job is to keep track of issue
301 reports; it is @emph{perfectly acceptable} to not understand
302 discussions between advanced users and/or developers.
308 @subheading Regular maintenance
310 After @strong{every release} (both stable and unstable):
315 Regression test comparison: if anything has changed suspiciously,
316 ask if it was deliberate. If the text output from LilyPond (the
317 logfile) changes, the differences will be displayed with a +
318 before text added to the logfile and - before any text removed
319 from the logfile. This may or may not be suspicious.
321 There is one test designed to produce output every time the
322 regtests are created. @code{test-output-distance.ly} creates
323 randomly spaced notes and will always have different output if the
324 regtest checker is working.
326 The official comparison is online, at:
328 @c NOTE: leave this here. In this case, it's worth duplicating
331 @uref{http://lilypond.org/test/}
334 More information is available from in
335 @ref{Precompiled regression tests}.
339 Issues to verify: try to reproduce the bug with the latest
340 official GUB version; if you cannot reproduce the bug, mark the
341 item @qq{Verified} (i.e. @qq{the fix has been verified to work}).
344 @uref{http://code.google.com/p/lilypond/issues/list?can=7}
347 A few (approximately 10%) of these fixed issues relate to the
348 build system or fundamental architecture changes; there is no way
349 for you to verify these. Leave those issues alone; somebody else
353 Check for any incorrectly-classified items in the tracker. This
354 generally just means looking at the grid to see any items without
361 @c try omitting from daily tasks for now. -gp
363 @subheading Irregular maintenance
365 @warning{These tasks are a lot of work; gathering more volunteers
366 to help is definitely recommended. However, the Bug Squad should
367 handle the organization and training of new volunteers.}
369 Once every year or two:
374 Checking all regtests: although we have a system for checking the
375 regtests between two versions, occasionally a bug will slip
376 through the cracks. It is therefore good to manually examine all
377 the regtests (compare the images to the text description). More
378 information is available from in @ref{Regression tests}.
382 Checking all issues: we try to mark each Issue @q{fixed} when we
383 fix it, but occasionally one or two issues will slip through the
384 cracks. It is therefore good to check all Issues. If you see the
385 same (broken) output as the initial report, then simply post a
386 @qq{Problem still exists in 2.x.y} message to the issue.
393 @node Issue classification
394 @section Issue classification
396 The Bug Squad should classify issues according to the guidelines
397 given by developers. Every issue should have a Status, Type, and
398 Priority; the other fields are optional.
400 @subheading Status (mandatory)
407 New: the item was added by a non-member, despite numerous warnings
408 not to do this. Should be reviewed by a member of the Bug Squad.
411 Accepted: the Bug Squad added it, or reviewed the item.
414 Started: a contributor is working on a fix. Owner should change
415 to be this contributor.
425 Invalid: issue should not have been added in the current state.
428 Duplicate: issue already exists in the tracker.
431 Fixed: a contributor claims to have fixed the bug. The Bug
432 Squad should check the fix with the next official binary release
433 (not by compiling the source from git). Owner should be set to
437 Verified: Bug Squad has confirmed that the issue is closed. This
438 means that nobody should ever need look at the report again -- if
439 there is any information in the issue that should be kept, open a
440 new issue for that info.
445 @subheading Owner (optional)
447 Newly-added issues should have @emph{no owner}. When a
448 contributor indicates that he has Started or Fixed an item, he
449 should become the owner.
452 @subheading Type (mandatory)
454 The issue's Type should be the first relevant item in this list.
459 Type-Collision: overlapping notation.
462 Type-Defect: a problem in the core program. (the @code{lilypond}
463 binary, scm files, fonts, etc).
466 Type-Documentation: inaccurate, missing, confusing, or desired
467 additional info. Must be fixable by editing a texinfo, ly, or scm
471 Type-Build: problem or desired features in the build system. This
472 includes the makefiles, stepmake, python scripts, and GUB.
475 Type-Scripts: problem or desired feature in the non-build-system
476 scripts. Mostly used for convert-ly, lilypond-book, etc.
479 Type-Enhancement: a feature request for the core program. The
480 distinction between enhancement and defect isn't extremely clear;
481 when in doubt, mark it as enhancement.
484 Type-Other: anything else.
489 @subheading Priority (mandatory)
491 Currently, only Critical items will block a stable release.
496 Priority-Critical: LilyPond segfaults, a regression (see below)
497 against a previous stable version or a regression against a fix
498 developed for this version. This does not apply where the
499 @qq{regression} occurred because a feature was removed
500 deliberately - this is not a bug.
503 Priority-High: An issue which produces output which does not
504 accurately reflect the input (e.g. where the user would expect
505 an accidental, but none is shown) or which produces aesthetically
506 poor output in a situation which could be expected to crop up
507 frequently in real-world music. It should not be used where the
508 problem can be avoided with a simple workaround. It can also
509 be used to flag where new code in a development version is not
510 functioning as it should. This level is also used for issues
511 which produce no output and fail to give the user a clue about
515 Priority-Medium: Normal priority - use this as the default.
518 Priority-Low: A minor problem which produces slightly undesirable
519 output, or which will only occur in contrived examples, or which
520 is very easily worked around.
523 Priority-Postponed: no fix planned. Generally used for things
524 which nobody wants to touch.
528 Note that these are initial classifications and can be subject
529 to change by others in the development team. For example, a
530 regression against an old stable version which hasn't been
531 noticed for a long time and which is unlikely to get fixed could
532 be downgraded from Priority-Critical by one of the programmers.
534 @subheading Opsys (optional)
536 Issues that only affect specific operating systems.
538 @subheading Patch (optional)
540 Normal Bug Squad members should not add or modify Patch issues;
541 leave them to the Patch Meister.
546 Patch-new: the patch has not been checked for @qq{obvious}
547 mistakes. When in doubt, use this tag.
550 Patch-review: the patch has no @qq{obvious} mistakes (as checked
551 by the Patch Meister), and is ready for review from main
554 Developers with git push ability can use this category, skipping
555 over @code{patch-new}.
558 Patch-needs_work: a developer has some concerns about the patch.
559 This does not necessarily mean that the patch must be changed; in
560 some cases, the developer's concerns can be resolved simply by
561 discussion the situation or providing notation examples.
563 If the patch is updated, the category should be changed to
564 @code{patch-new} (for normal contributors) or @code{patch-review}
565 (for developers who are very confident about their patch).
568 Patch-abandoned: the author has not responded to review comments
573 @subheading Other items (optional)
580 Regression: it used to work intentionally in an earlier
581 stable release. If the earlier output was accidental (i.e. we
582 didn't try to stop a collision, but it just so happened that two
583 grobs didn't collide), then breaking it does not count as a
586 To help decide whether the change is a regression, and therefore
587 should be Priority-Critical, please adopt the following process:
592 Are you certain the change is OK? If so, do nothing.
595 Are you certain that the change is bad? Add it to the tracker
596 as a Critical issue, regression.
599 If you're not certain either way, add it to the tracker as a
600 Critical issue, regression but be aware that it may be
601 recategorised or marked invalid.
605 In particular, anything that breaks a regression test is a
609 Frog: the fix is believed to be suitable for a new contributor
610 (does not require a great deal of knowledge about LilyPond). The
611 issue should also have an estimated time in a comment.
614 Maintainability: hinders development of LilyPond. For example,
615 improvements to the build system, or @qq{helper} python scripts.
618 Bounty: somebody is willing to pay for the fix. Only add this tag
619 if somebody has offered an exact figure in US dollars or euros.
622 Warning: graphical output is fine, but lilypond prints a
623 false/misleading warning message. Alternately, a warning should
624 be printed (such as a bar line error), but was not. Also applies
625 to warnings when compiling the source code or generating
629 Security: might potentially be used.
632 Performance: might potentially be used.
636 If you particularly want to add a label not in the list, go
637 ahead, but this is not recommended.
640 @node Adding issues to the tracker
641 @section Adding issues to the tracker
643 @warning{This should only be done by the Bug Squad or experienced
644 developers. Normal users should not do this; instead, they should
645 follow the guidelines for @rweb{Bug reports}.}
647 In order to assign labels to issues, Bug Squad members should log
648 in to their google account before adding an item.
653 Check if the issue falls into any previous category given on the
654 relevant checklists in @ref{Bug Squad checklists}. If in doubt,
655 add a new issue for a report. We would prefer to have some
656 incorrectly-added issues rather than lose information that should
660 Add the issue and classify it according to the guidelines in
661 @ref{Issue classification}. In particular, the item should have
662 @code{Status}, @code{Type-}, and @code{Priority-} labels.
664 Include output with the first applicable method:
669 If the issue has a notation example which fits in one system,
670 generate a small @file{bug.preview.png} file with:
673 lilypond -dpreview bug.ly
677 If the issue has an example which requires more than one system
678 (i.e. a spacing bug), generate a @file{bug.png} file with:
681 lilypond --png bug.ly
685 If the issue requires one or two pages of output, then generate a
686 @file{bug.png} file with the normal:
689 lilypond --png bug.ly
693 Images created as @file{bug.png} may be trimmed to a minimum size
694 by using the @code{trimtagline.sh} script, which can be found at
695 @uref{https://raw.github.com/gperciva/lilypond-extra/master/bug-squad/trimtagline.sh}
698 trimtagline.sh bug.ly
702 If the issue cannot be shown with less than three pages, then
703 generate a @file{bug.pdf} file with:
706 lilypond --pdf bug.ly
709 Note that this is likely to be extremely rare; most bugs should fit
710 into the first two categories above.
716 After adding the issue, please send a response email to the same
717 group(s) that the initial patch was sent to. If the initial email
718 was sent to multiple mailing lists (such as both @code{user} and
719 @code{bugs}), then reply to all those mailing lists as well. The
720 email should contain a link to the issue you just added.
727 @section Patch handling
729 @warning{This is not a Bug Squad responsibility; we have a
730 separate person handling this task.}
732 There is a single Patch Meister, and a number of Patch Helpers
733 (rename this?). The list of known patches awaiting review is:
736 @uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:patch&sort=patch}
740 @subheading Helpers: adding patches
742 The primary duty is to add patches to the google tracker; we have
743 a bad track record of losing patches in email. Patches generally
744 come to the @code{lilypond-devel} mailing list, but are sometimes
745 sent to @code{bug-lilypond}, @code{lilypond-users}, or
746 @code{frogs} mailing list instead.
750 Unless a patch is clearly in response to an existing issue, add a
751 new issue with the @code{Patch-new} label and a link to the patch
752 (either on the mailing list archives or the codereview url).
754 Issue numbers are cheap; losing developers because they got fed up
755 with us losing their hard work is expensive.
757 @c if we enter patches immediately, I don't think this is relevant.
760 Before adding a patch-reminder issue, do a quick check to see if
761 it was pushed without sending any email. This can be checked for
762 searching for relevant terms (from the patch subject or commit
763 message) on the webgit page:
766 @uref{http://git.savannah.gnu.org/gitweb/?p=lilypond.git}
771 If the patch is clearly in response to an existing issue, then
772 update that issue with the @code{Patch-new} label and a link to
773 the patch (either on the mailing list archives or the codereview
777 After adding the issue, please send a response email to the same
778 group(s) that the initial patch was sent to.
780 If the initial email was sent to multiple mailing lists (such as
781 both @code{bugs} and @code{devel}), then reply to all those
782 mailing lists as well. The email should contain a link to the
783 issue you just added.
787 @subheading Helpers: @code{Patch-review} label
789 The secondary duty is to do make sure that every issue in the
790 tracker with a @code{Patch-review} label has passed these
795 Applies automatically to git master.
797 It's ok to have offsets, but not conflicts.
800 Regtest comparison looks ok; no unexpected changes.
803 Descriptive subject line.
805 Avoid subjects like @qq{fixes 123}; instead write @qq{Doc: discuss
806 stacking-dir for BassFigureAlignment (fix 123)}.
809 Compiles docs from scratch. Only check this if you have reason to
810 suspect it might not work.
815 Check code indentation and style. This should be easier post-GOP
816 when we have a better-defined code style.
821 @subheading Patch Meister
823 The Patch Meister will:
828 send @qq{countdown} emails to
829 @code{lilypond-devel} when patches appear to be ready.
832 send general requests to review patches, or even nasty requests to
836 downgrade patches from @code{Patch-review} to
837 @code{Patch-needs_work} as appropriate.
840 downgrade patches from @code{Patch-needs_work} to
841 @code{Patch-abandoned} if no actions have been taken in four
849 @node Summary of project status
850 @section Summary of project status
852 @subsubheading Project overview
854 Grid view provides the best overview:
857 @uref{http://code.google.com/p/lilypond/issues/list?mode=grid&y=Priority&x=Type&cells=ids}
860 @subsubheading Hindering development
862 These issues stop or slow development work:
865 @uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:Maintainability}
868 @subsubheading Easy tasks
870 Issues tagged with @code{Frog} indicates a task suitable for a
871 relatively new contributor. The time given is a quick
872 (inaccurate) estimate of the time required for somebody who is
873 familiar with material in this manual, but does not know anything
874 else about LilyPond development.
877 @uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:Frog}
880 @subsubheading Patches to review
882 Patches which have no @qq{obvious} problems:
885 @uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:patch-review}