1 @c -*- coding: utf-8; mode: texinfo; -*-
5 This chapter deals with defects, feature requests, and
6 miscellaneous development tasks.
9 * Introduction to issues::
11 * Bug Squad checklists::
12 * Issue classification::
13 * Adding issues to the tracker::
14 * Summary of project status::
15 * Finding the cause of a regression::
19 @node Introduction to issues
20 @section Introduction to issues
22 @warning{Unless otherwise specified, all the tasks in this chapter
23 are @qq{simple} tasks: they can be done by a normal user with
24 nothing more than a web browser, email, and lilypond.}
26 @qq{Issues} isn't just a politically-correct term for @qq{bug}.
27 We use the same tracker for feature requests and code TODOs, so
28 the term @qq{bug} wouldn't be accurate. Despite the difference
29 between @qq{issue} and @qq{bug}, we call our team of contributors
30 who organize issues the @emph{Bug Squad}.
32 The Bug Squad is mainly composed of non-programmers -- their job
33 is to @emph{organize} issues, not solve them. Their duties
34 include removing false bug reports, ensuring that any real bug
35 report contains enough information for developers, and checking
36 that a developer's fix actually resolves the problem.
38 New volunteers for the Bug Squad should contact the
39 @ref{Meisters, Bug Meister}.
43 @section Bug Squad setup
45 We highly recommend that you configure your email to use effective
46 sorting; this can reduce your workload @emph{immensely}. The
47 email folders names were chosen specifically to make them work if
48 you sort your folders alphabetically.
53 Skim through every section of this chapter, @ref{Issues}. Read in
54 detail any sections called @qq{Bug Squad...}, or any page linked
55 from @ref{Bug Squad checklists}.
58 If you do not have one already, create a gmail account and send
59 the email address to the @ref{Meisters, Bug Meister}.
62 Subscribe your gmail account to @code{bug-lilypond}.
65 Configure your google code account:
70 Wait until your gmail account is listed in:
73 @uref{http://code.google.com/p/lilypond/people/list}
77 Sign in to google code by clicking in the top-right corner of:
80 @uref{http://code.google.com/p/lilypond/issues/list}
83 You cannot log if you have Google Sharing
84 @uref{http://www.googlesharing.net/} enabled.
87 Go to your @qq{Profile}, and select @qq{Settings}.
90 Scroll down to @qq{Issue change notification}, and make sure that
91 you have @emph{selected} @qq{If I starred the issue}.
96 Configure your email client:
101 Any email sent with your gmail address in the @code{To:} or
102 @code{CC:} fields should go to a @code{bug-answers} folder.
104 When setting up your filtering rules, be aware that Google Code
105 might use different versions of your email address, such as ones
106 ending in @code{@@googlemail.com} or @code{@@gmail.com}.
109 Any other email either from, or CC'd to,
112 lilypond@@googlecode.com
116 should go into a separate @code{bug-ignore} folder. Alternately,
117 you may automatically delete these emails.
119 You will @strong{not read} these emails as part of your Bug Squad
120 duties. If you are curious, go ahead and read them later, but it
121 does @strong{not} count as Bug Squad work.
124 Any other email sent to (or CC'd to):
131 should go into a separate @code{bug-current} folder.
138 @node Bug Squad checklists
139 @section Bug Squad checklists
141 When you do Bug Squad work, start at the top of this page and work
142 your way down. Stop when you've done 15 minutes.
144 Please use the email sorting described in @ref{Bug Squad setup}.
145 This means that (as Bug Squad members) you will only ever respond
146 to emails sent or CC'd to the @code{bug-lilypond} mailing list.
149 @subsubheading Emails to you personally
151 You are not expected to work on Bug Squad matters outside of your
152 15 minutes, but sometimes a confused user will send a bug report
153 (or an update to a report) to you personally. If that happens,
154 please forward such emails to the @code{bug-lilypond} list so that
155 the currently-active Bug Squad member(s) can handle the message.
158 @subsubheading Daily schedule
160 The Bug Meister is omitted from the daily schedule.
165 Tuesday: James Bailey
173 @subsubheading Emails to @code{bug-answers}
175 Some of these emails will be comments on issues that you added to
179 If they are asking for more information, give the additional
183 If the email says that the issue was classified in some other
184 manner, read the rationale given and take that into account for
185 the next issue you add.
188 Otherwise, move them to your @code{bug-ignore} folder.
192 Some of these emails will be discussions about Bug Squad work;
196 @subsubheading Emails to @code{bug-current}
198 Dealing with these emails is your main task. Your job is to get
199 rid of these emails in the first method which is applicable:
203 If the email has already been handled by a Bug Squad member (i.e.
204 check to see who else has replied to it), delete it.
207 If the email is a question about how to use LilyPond, reply with
211 For questions about how to use LilyPond, please read our
212 documentation available from:
213 @uref{http://lilypond.org/website/manuals.html}
214 or ask the lilypond-user mailing list.
218 If the email mentions @qq{the latest git}, or any version number
219 that has not yet been officially released, forward it to
220 @code{lilypond-devel}.
223 If a bug report is not in the form of a Tiny example, direct the
224 user to resubmit the report with this response:
227 I'm sorry, but due to our limited resources for handling bugs, we
228 can only accept reports in the form of Tiny examples. Please see
229 step 2 in our bug reporting guidelines:
230 @uref{http://lilypond.org/website/bug-reports.html}
234 If anything is unclear, ask the user for more information.
236 How does the graphical output differ from what the user expected?
237 What version of lilypond was used (if not given) and operating
238 system (if this is a suspected cause of the problem)? In short,
239 if you cannot understand what the problem is, ask the user to
240 explain more. It is the user's responsibility to explain the
241 problem, not your responsibility to understand it.
244 If the behavior is expected, the user should be told to read the
248 I believe that this is the expected behaviour -- please read our
249 documentation about this topic. If you think that it really is a
250 mistake, please explain in more detail. If you think that the
251 docs are unclear, please suggest an improvement as described by
252 @qq{Simple tasks -- Documentation} on:
253 @uref{http://lilypond.org/website/help-us.html}
257 If the issue already exists in the tracker, send an email to that
261 This issue has already been reported; you can follow the
262 discussion and be notified about fixes here:
266 (copy+paste the google code issue URL)
269 Accept the report as described in
270 @ref{Adding issues to the tracker}.
274 All emails should be CC'd to the @code{bug-lilypond} list so that
275 other Bug Squad members know that you have processed the email.
277 @warning{There is no option for @qq{ignore the bug report} -- if
278 you cannot find a reason to reject the report, you must accept
283 @c Try omitting this from Bug Squad duties
285 @subheading Updates / discussion about issues
287 We try to keep discussions about issues on the tracker, but
288 sometimes it spills over onto email. If discussion has ended with
289 no patch / resolution and at least @strong{3 days} have passed,
295 Summarize the recent discussion on the tracker, and add a link to
296 the original discussion.
299 Add the comment @qq{there was some technical discussion which I
300 could not understand}, and include a link to the original
303 We do not expect Bug Squad members to be programmers, or even to
304 be moderately-skilled users. Your job is to keep track of issue
305 reports; it is @emph{perfectly acceptable} to not understand
306 discussions between advanced users and/or developers.
312 @subheading Regular maintenance
314 After @strong{every release} (both stable and unstable):
319 Regression test comparison: if anything has changed suspiciously,
320 ask if it was deliberate. The official comparison is online, at:
322 @c NOTE: leave this here. In this case, it's worth duplicating
325 @uref{http://lilypond.org/test/}
328 More information is available from in
329 @ref{Precompiled regression tests}.
333 Issues to verify: try to reproduce the bug with the latest
334 version; if you cannot reproduce the bug, mark the item
335 @qq{Verified} (i.e. @qq{the fix has been verified to work}).
338 @uref{http://code.google.com/p/lilypond/issues/list?can=7}
341 A few (approximately 10%) of these fixed issues relate to the
342 build system or fundamental architecture changes; there is no way
343 for you to verify these. Leave those issues alone; somebody else
350 @c try omitting from daily tasks for now. -gp
352 Once every @strong{two weeks} or so:
357 Check for any incorrectly-classified items in the tracker. This
358 generally just means looking at the grid to see any items without
362 Check for any items with @code{label:patch}. If it's been more
363 than a week since the last action on the issue, send an email to
364 -devel to remind them about it. If the patch was withdrawn for
365 more work, then remove the @code{patch} label.
368 @uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:patch}
374 @subheading Irregular maintenance
376 @warning{These tasks are a lot of work; gathering more volunteers
377 to help is definitely recommended. However, the Bug Squad should
378 handle the organization and training of new volunteers.}
380 Once every year or two:
385 Checking all regtests: although we have a system for checking the
386 regtests between two versions, occasionally a bug will slip
387 through the cracks. It is therefore good to manually examine all
388 the regtests (compare the images to the text description). More
389 information is available from in @ref{Regression tests}.
393 Checking all issues: we try to mark each Issue @q{fixed} when we
394 fix it, but occasionally one or two issues will slip through the
395 cracks. It is therefore good to check all Issues. If you see the
396 same (broken) output as the initial report, then simply post a
397 @qq{Problem still exists in 2.x.y} message to the issue.
404 @node Issue classification
405 @section Issue classification
407 The Bug Squad should classify issues according to the guidelines
408 given by developers. Every issue should have a Status, Type, and
409 Priority; the other fields are optional.
411 @subheading Status (mandatory)
418 New: the item was added by a non-member, despite numerous warnings
419 not to do this. Should be reviewed by a member of the Bug Squad.
422 Accepted: the Bug Squad added it, or reviewed the item.
425 Started: a contributor is working on a fix. Owner should change
426 to be this contributor.
436 Invalid: issue should not have been added in the current state.
439 Duplicate: issue already exists in the tracker.
442 Fixed: a contributor claims to have fixed the bug. The Bug
443 Squad should check the fix with the next official binary release
444 (not by compiling the source from git). Owner should be set to
448 Verified: Bug Squad has confirmed that the issue is closed. This
449 means that nobody should ever need look at the report again -- if
450 there is any information in the issue that should be kept, open a
451 new issue for that info.
456 @subheading Owner (optional)
458 Newly-added issues should have @emph{no owner}. When a
459 contributor indicates that he has Started or Fixed an item, he
460 should become the owner.
463 @subheading Type (mandatory)
465 The issue's Type should be the first relevant item in this list.
470 Type-Collision: overlapping notation.
473 Type-Defect: a problem in the core program. (the @code{lilypond}
474 binary, scm files, fonts, etc).
477 Type-Documentation: inaccurate, missing, confusing, or desired
478 additional info. Must be fixable by editing a texinfo, ly, or scm
482 Type-Build: problem or desired features in the build system. This
483 includes the makefiles, stepmake, python scripts, and GUB.
486 Type-Scripts: problem or desired feature in the non-build-system
487 scripts. Mostly used for convert-ly, lilypond-book, etc.
490 Type-Enhancement: a feature request for the core program. The
491 distinction between enhancement and defect isn't extremely clear;
492 when in doubt, mark it as enhancement.
495 Type-Other: anything else.
500 @subheading Priority (mandatory)
502 Currently, only Critical items will block a stable release.
507 Priority-Critical: LilyPond segfaults, a regression (see below)
508 against a previous stable version or a regression against a fix
509 developed for this version. This does not apply where the
510 @qq{regression} occurred because a feature was removed
511 deliberately - this is not a bug.
514 Priority-High: An issue which produces output which does not
515 accurately reflect the input (e.g. where the user would expect
516 an accidental, but none is shown) or which produces aesthetically
517 poor output in a situation which could be expected to crop up
518 frequently in real-world music. It should not be used where the
519 problem can be avoided with a simple workaround. It can also
520 be used to flag where new code in a development version is not
521 functioning as it should. This level is also used for issues
522 which produce no output and fail to give the user a clue about
526 Priority-Medium: Normal priority - use this as the default.
529 Priority-Low: A minor problem which produces slightly undesirable
530 output, or which will only occur in contrived examples, or which
531 is very easily worked around.
534 Priority-Postponed: no fix planned. Generally used for things
535 which nobody wants to touch.
539 Note that these are initial classifications and can be subject
540 to change by others in the development team. For example, a
541 regression against an old stable version which hasn't been
542 noticed for a long time and which is unlikely to get fixed could
543 be downgraded from Priority-Critical by one of the programmers.
545 @subheading Opsys (optional)
547 Issues that only affect specific operating systems.
550 @subheading Other items (optional)
557 Regression: it used to work intentionally in an earlier
558 stable release. If the earlier output was accidental (i.e. we
559 didn't try to stop a collision, but it just so happened that two
560 grobs didn't collide), then breaking it does not count as a
563 To help decide whether the change is a regression, and therefore
564 should be Priority-Critical, please adopt the following process:
569 Are you certain the change is OK? If so, do nothing.
572 Are you certain that the change is bad? Add it to the tracker
573 as a Critical issue, regression.
576 If you're not certain either way, add it to the tracker as a
577 Critical issue, regression but be aware that it may be
578 recategorised or marked invalid.
582 In particular, anything that breaks a regression test is a
586 Patch: a patch to fix an issue is attached.
589 Frog: the fix is believed to be suitable for a new contributor
590 (does not require a great deal of knowledge about LilyPond). The
591 issue should also have an estimated time in a comment.
594 Maintainability: hinders development of LilyPond. For example,
595 improvements to the build system, or @qq{helper} python scripts.
598 Bounty: somebody is willing to pay for the fix. Only add this tag
599 if somebody has offered an exact figure in US dollars or euros.
602 Warning: graphical output is fine, but lilypond prints a
603 false/misleading warning message. Alternately, a warning should
604 be printed (such as a bar line error), but was not. Also applies
605 to warnings when compiling the source code or generating
609 Security: might potentially be used.
612 Performance: might potentially be used.
616 If you particularly want to add a label not in the list, go
617 ahead, but this is not recommended.
620 @node Adding issues to the tracker
621 @section Adding issues to the tracker
623 @warning{This should only be done by the Bug Squad or experienced
624 developers. Normal users should not do this; instead, they should
625 follow the guidelines for @rweb{Bug reports}.}
627 In order to assign labels to issues, Bug Squad members should log
628 in to their google account before adding an item.
630 @subsubheading Normal issues
635 Check if the issue falls into any previous category given on the
636 relevant checklists in @ref{Bug Squad checklists}. If in doubt,
637 add a new issue for a report. We would prefer to have some
638 incorrectly-added issues rather than lose information that should
642 Add the issue and classify it according to the guidelines in
643 @ref{Issue classification}. In particular, the item should have
644 @code{Status}, @code{Type-}, and @code{Priority-} labels.
646 Include output with the first applicable method:
651 If the issue has a notation example which fits in one system,
652 generate a small @file{bug.preview.png} file with:
655 lilypond -dpreview bug.ly
659 If the issue has an example which requires more than one system
660 (i.e. a spacing bug), generate a @file{bug.png} file with:
663 lilypond --png bug.ly
667 If the issue requires one or two pages of output, then generate a
668 @file{bug.png} file with the normal:
671 lilypond --png bug.ly
675 If the issue cannot be shown with less than three pages, then
676 generate a @file{bug.pdf} file with:
679 lilypond --pdf bug.ly
682 Note that this is likely to be extremely rare; most bugs should fit
683 into the first two categories above.
689 After adding the issue, please send a response email to the same
690 group(s) that the initial patch was sent to. If the initial email
691 was sent to multiple mailing lists (such as both @code{user} and
692 @code{bugs}), then reply to all those mailing lists as well. The
693 email should contain a link to the issue you just added.
698 @subsubheading Patch reminders
700 @warning{This is not a Bug Squad responsibility; we have a
701 separate person handling this task.}
703 There is a special category of issues: reminders of an existing
704 patch. These should be added if a patch has been sent to a
705 lilypond mailing list (generally @code{lilypond-devel}, but they
706 sometimes appear on @code{bug-lilypond} as well) and has had no
707 discussion for at least @strong{3 days}. Do not add issues for
708 patches under active discussion.
710 Before adding a patch-reminder issue, do a quick check to see if
711 it was pushed without sending any email. This can be checked for
712 searching for relevant terms (from the patch subject or commit
713 message) on the webgit page:
716 @uref{http://git.savannah.gnu.org/gitweb/?p=lilypond.git}
719 After adding the issue, please send a response email to the same
720 group(s) that the initial patch was sent to. If the initial email
721 was sent to multiple mailing lists (such as both @code{bugs} and
722 @code{devel}), then reply to all those mailing lists as well. The
723 email should contain a link to the issue you just added.
727 @node Summary of project status
728 @section Summary of project status
730 The best overview of our current status is given by the grid view:
733 @uref{http://code.google.com/p/lilypond/issues/list?mode=grid&y=Priority&x=Type&cells=ids}
736 Also of interest might be the issues hindering future development:
739 @uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:Maintainability&mode=grid&y=Priority&x=Type&cells=ids}
742 Finally, issues tagged with @code{Frog} indicates a task suitable
743 for a relatively new contributor. The time given is a quick
744 (inaccurate) estimate of the time required for somebody who is
745 familiar with material in this manual, but does not know anything
746 else about LilyPond development.
749 @uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:Frog&mode=grid&y=Priority&x=Type&cells=ids}
753 @node Finding the cause of a regression
754 @section Finding the cause of a regression
756 @warning{This is not a @qq{simple} task; it requires a fair amount
757 of technical knowledge.}
759 Git has special functionality to help tracking down the exact
760 commit which causes a problem. See the git manual page for
761 @code{git bisect}. This is a job that non-programmers can do,
762 although it requires familiarity with git, ability to compile
763 LilyPond, and generally a fair amount of technical knowledge. A
764 brief summary is given below, but you may need to consult other
765 documentation for in-depth explanations.
767 Even if you are not familiar with git or are not able to compile
768 LilyPond you can still help to narrow down the cause of a
769 regression simply by downloading the binary releases of different
770 LilyPond versions and testing them for the regression. Knowing
771 which version of LilyPond first exhibited the regression is
772 helpful to a developer as it shortens the @code{git bisect}
775 Once a problematic commit is identified, the programmers' job is
776 much easier. In fact, for most regression bugs, the majority of
777 the time is spent simply finding the problematic commit.
779 More information is in @ref{Regression tests}.
781 @subheading git bisect setup
783 We need to set up the bisect for each problem we want to
786 Suppose we have an input file which compiled in version 2.13.32,
787 but fails in version 2.13.38 and above.
798 Give it the earliest known bad tag:
801 git bisect bad release/2.13.38-1
804 (you can see tags with: @code{git tag} )
807 Give it the latest known good tag:
810 git bisect good release/2.13.32-1
813 You should now see something like:
815 Bisecting: 195 revisions left to test after this (roughly 8 steps)
816 [b17e2f3d7a5853a30f7d5a3cdc6b5079e77a3d2a] Web: Announcement
817 update for the new @qq{LilyPond Report}.
822 @subheading git bisect actual
834 Test your input file:
837 out/bin/lilypond test.ly
845 Does it crash, or is the output bad? If so:
852 Does your input file produce good output? If so:
861 Once the exact problem commit has been identified, git will inform
862 you with a message like:
865 6d28aebbaaab1be9961a00bf15a1ef93acb91e30 is the first bad commit
866 %%% ... blah blah blah ...
869 If there is still a range of commits, then git will automatically
870 select a new version for you to test. Go to step #1.
874 @subheading Recommendation: use two terminal windows
878 One window is open to the @code{build/} directory, and alternates
879 between these commands:
883 out/bin/lilypond test.ly
887 One window is open to the top source directory, and alternates
888 between these commands: