1 @c -*- coding: utf-8; mode: texinfo; -*-
5 This chapter deals with defects, feature requests, and
6 miscellaneous development tasks.
9 * Introduction to issues::
11 * Bug Squad checklists::
12 * Issue classification::
13 * Adding issues to the tracker::
14 * Summary of project status::
15 * Finding the cause of a regression::
19 @node Introduction to issues
20 @section Introduction to issues
22 @warning{Unless otherwise specified, all the tasks in this chapter
23 are @qq{simple} tasks: they can be done by a normal user with
24 nothing more than a web browser, email, and lilypond.}
26 @qq{Issues} isn't just a politically-correct term for @qq{bug}.
27 We use the same tracker for feature requests and code TODOs, so
28 the term @qq{bug} wouldn't be accurate. Despite the difference
29 between @qq{issue} and @qq{bug}, we call our team of contributors
30 who organize issues the @emph{Bug Squad}.
32 The Bug Squad is mainly composed of non-programmers -- their job
33 is to @emph{organize} issues, not solve them. Their duties
34 include removing false bug reports, ensuring that any real bug
35 report contains enough information for developers, and checking
36 that a developer's fix actually resolves the problem.
38 New volunteers for the Bug Squad should contact the
39 @ref{Meisters, Bug Meister}.
43 @section Bug Squad setup
45 We highly recommend that you configure your email to use effective
46 sorting; this can reduce your workload @emph{immensely}. The
47 email folders names were chosen specifically to make them work if
48 you sort your folders alphabetically.
53 Skim through every section of this chapter, @ref{Issues}. Read in
54 detail any sections called @qq{Bug Squad...}, or any page linked
55 from @ref{Bug Squad checklists}.
58 If you do not have one already, create a gmail account and send
59 the email address to the @ref{Meisters, Bug Meister}.
62 Subscribe your gmail account to @code{bug-lilypond}.
65 Configure your google code account:
70 Sign in to google code by clicking in the top-right corner of:
73 @uref{http://code.google.com/p/lilypond/issues/list}
77 Go to your @qq{Profile}, and select @qq{Settings}.
80 Scroll down to @qq{Issue change notification}, and make sure that
81 you have @emph{selected} @qq{If I starred the issue}.
86 Configure your email client:
91 Any email sent with your gmail address in the @code{To:} or
92 @code{CC:} fields should go to a @code{bug-answers} folder.
94 When setting up your filtering rules, be aware that Google Code
95 might use different versions of your email address, such as ones
96 ending in @code{@@googlemail.com} or @code{@@gmail.com}.
99 Any other email either from, or CC'd to,
102 lilypond@@googlecode.com
106 should go into a separate @code{bug-ignore} folder. Alternately,
107 you may automatically delete these emails.
109 You will @strong{not read} these emails as part of your Bug Squad
110 duties. If you are curious, go ahead and read them later, but it
111 does @strong{not} count as Bug Squad work.
114 Any other email sent to (or CC'd to):
121 should go into a separate @code{bug-current} folder.
128 @node Bug Squad checklists
129 @section Bug Squad checklists
131 When you do Bug Squad work, start at the top of this page and work
132 your way down. Stop when you've done 15 minutes.
134 Please use the email sorting described in @ref{Bug Squad setup}.
135 This means that (as Bug Squad members) you will only ever respond
136 to emails sent or CC'd to the @code{bug-lilypond} mailing list.
139 @subsubheading Emails to you personally
141 You are not expected to work on Bug Squad matters outside of your
142 15 minutes, but sometimes a confused user will send a bug report
143 (or an update to a report) to you personally. If that happens,
144 please forward such emails to the @code{bug-lilypond} list so that
145 the currently-active Bug Squad member(s) can handle the message.
148 @subsubheading Daily schedule
150 The Bug Meister is omitted from the daily schedule.
155 Tuesday: James Bailey
163 @subsubheading Emails to @code{bug-answers}
165 Some of these emails will be comments on issues that you added to
169 If they are asking for more information, give the additional
173 If the email says that the issue was classified in some other
174 manner, read the rationale given and take that into account for
175 the next issue you add.
178 Otherwise, move them to your @code{bug-ignore} folder.
182 Some of these emails will be discussions about Bug Squad work;
186 @subsubheading Emails to @code{bug-current}
188 Dealing with these emails is your main task. Your job is to get
189 rid of these emails in the first method which is applicable:
193 If the email has already been handled by a Bug Squad member (i.e.
194 check to see who else has replied to it), delete it.
197 If the email is a question about how to use LilyPond, reply with
201 For questions about how to use LilyPond, please read our
202 documentation available from:
203 @uref{http://lilypond.org/website/manuals.html}
204 or ask the lilypond-user mailing list.
208 If the email mentions @qq{the latest git}, or any version number
209 that has not yet been officially released, forward it to
210 @code{lilypond-devel}.
213 If a bug report is not in the form of a Tiny example, direct the
214 user to resubmit the report with this response:
217 I'm sorry, but due to our limited resources for handling bugs, we
218 can only accept reports in the form of Tiny examples. Please see
219 step 2 in our bug reporting guidelines:
220 @uref{http://lilypond.org/website/bug-reports.html}
224 If anything is unclear, ask the user for more information.
226 How does the graphical output differ from what the user expected?
227 What version of lilypond was used (if not given) and operating
228 system (if this is a suspected cause of the problem)? In short,
229 if you cannot understand what the problem is, ask the user to
230 explain more. It is the user's responsibility to explain the
231 problem, not your responsibility to understand it.
234 If the behavior is expected, the user should be told to read the
238 I believe that this is the expected behaviour -- please read our
239 documentation about this topic. If you think that it really is a
240 mistake, please explain in more detail. If you think that the
241 docs are unclear, please suggest an improvement as described by
242 @qq{Simple tasks -- Documentation} on:
243 @uref{http://lilypond.org/website/help-us.html}
247 If the issue already exists in the tracker, send an email to that
251 This issue has already been reported; you can follow the
252 discussion and be notified about fixes here:
256 (copy+paste the google code issue URL)
259 Accept the report as described in
260 @ref{Adding issues to the tracker}.
264 All emails should be CC'd to the @code{bug-lilypond} list so that
265 other Bug Squad members know that you have processed the email.
267 @warning{There is no option for @qq{ignore the bug report} -- if
268 you cannot find a reason to reject the report, you must accept
273 @c Try omitting this from Bug Squad duties
275 @subheading Updates / discussion about issues
277 We try to keep discussions about issues on the tracker, but
278 sometimes it spills over onto email. If discussion has ended with
279 no patch / resolution and at least @strong{3 days} have passed,
285 Summarize the recent discussion on the tracker, and add a link to
286 the original discussion.
289 Add the comment @qq{there was some technical discussion which I
290 could not understand}, and include a link to the original
293 We do not expect Bug Squad members to be programmers, or even to
294 be moderately-skilled users. Your job is to keep track of issue
295 reports; it is @emph{perfectly acceptable} to not understand
296 discussions between advanced users and/or developers.
302 @subheading Regular maintenance
304 After @strong{every release} (both stable and unstable):
309 Regression test comparison: if anything has changed suspiciously,
310 ask if it was deliberate. The official comparison is online, at:
312 @c NOTE: leave this here. In this case, it's worth duplicating
315 @uref{http://lilypond.org/test/}
318 More information is available from in
319 @ref{Precompiled regression tests}.
323 Issues to verify: try to reproduce the bug with the latest
324 version; if you cannot reproduce the bug, mark the item
325 @qq{Verified} (i.e. @qq{the fix has been verified to work}).
328 @uref{http://code.google.com/p/lilypond/issues/list?can=7}
331 A few (approximately 10%) of these fixed issues relate to the
332 build system or fundamental architecture changes; there is no way
333 for you to verify these. Leave those issues alone; somebody else
340 @c try omitting from daily tasks for now. -gp
342 Once every @strong{two weeks} or so:
347 Check for any incorrectly-classified items in the tracker. This
348 generally just means looking at the grid to see any items without
352 Check for any items with @code{label:patch}. If it's been more
353 than a week since the last action on the issue, send an email to
354 -devel to remind them about it. If the patch was withdrawn for
355 more work, then remove the @code{patch} label.
358 @uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:patch}
364 @subheading Irregular maintenance
366 @warning{These tasks are a lot of work; gathering more volunteers
367 to help is definitely recommended. However, the Bug Squad should
368 handle the organization and training of new volunteers.}
370 Once every year or two:
375 Checking all regtests: although we have a system for checking the
376 regtests between two versions, occasionally a bug will slip
377 through the cracks. It is therefore good to manually examine all
378 the regtests (compare the images to the text description). More
379 information is available from in @ref{Regression tests}.
383 Checking all issues: we try to mark each Issue @q{fixed} when we
384 fix it, but occasionally one or two issues will slip through the
385 cracks. It is therefore good to check all Issues. If you see the
386 same (broken) output as the initial report, then simply post a
387 @qq{Problem still exists in 2.x.y} message to the issue.
394 @node Issue classification
395 @section Issue classification
397 The Bug Squad should classify issues according to the guidelines
398 given by developers. Every issue should have a Status, Type, and
399 Priority; the other fields are optional.
401 @subheading Status (mandatory)
408 New: the item was added by a non-member, despite numerous warnings
409 not to do this. Should be reviewed by a member of the Bug Squad.
412 Accepted: the Bug Squad added it, or reviewed the item.
415 Started: a contributor is working on a fix. Owner should change
416 to be this contributor.
426 Invalid: issue should not have been added in the current state.
429 Duplicate: issue already exists in the tracker.
432 Fixed: a contributor claims to have fixed the bug. The Bug
433 Squad should check the fix with the next official binary release
434 (not by compiling the source from git). Owner should be set to
438 Verified: Bug Squad has confirmed that the issue is closed. This
439 means that nobody should ever need look at the report again -- if
440 there is any information in the issue that should be kept, open a
441 new issue for that info.
446 @subheading Owner (optional)
448 Newly-added issues should have @emph{no owner}. When a
449 contributor indicates that he has Started or Fixed an item, he
450 should become the owner.
453 @subheading Type (mandatory)
455 The issue's Type should be the first relevant item in this list.
460 Type-Collision: overlapping notation.
463 Type-Defect: a problem in the core program. (the @code{lilypond}
464 binary, scm files, fonts, etc).
467 Type-Documentation: inaccurate, missing, confusing, or desired
468 additional info. Must be fixable by editing a texinfo, ly, or scm
472 Type-Build: problem or desired features in the build system. This
473 includes the makefiles, stepmake, python scripts, and GUB.
476 Type-Scripts: problem or desired feature in the non-build-system
477 scripts. Mostly used for convert-ly, lilypond-book, etc.
479 Type-Enhancement: a feature request for the core program. The
480 distinction between enhancement and defect isn't extremely clear;
481 when in doubt, mark it as enhancement.
484 Type-Other: anything else.
489 @subheading Priority (mandatory)
491 Currently, only Critical items will block a stable release.
496 Priority-Critical: lilypond segfaults, or a regression occurred
497 within the last two stable versions. (i.e. when developing 2.13,
498 any regression against 2.12 or 2.10 counts)
501 Priority-High: highly embarrassing items, and any regression
502 against a version earlier than two stable versions (i.e. when
503 developing 2.13, any regression against 2.8 or earlier). This
504 level is also used for issues which produce no output and fail to
505 give the user a clue about what's wrong.
508 Priority-Medium: normal priority.
511 Priority-Low: less important than normal.
514 Priority-Postponed: no fix planned. Generally used for things
515 like Ancient notation, which nobody wants to touch.
519 The difference between Priority-Medium and Priority-Low is not
520 well-defined, both in this policy and in practice. The only
521 answer we can give at the moment is @qq{look at existing items in
522 of the same type, and try to guess whether the priority is closer
523 to the Medium items or Low items}. We're aware of the ambiguity,
524 and won't complain if somebody picks a @q{wrong} value for
528 @subheading Opsys (optional)
530 Issues that only affect specific operating systems.
533 @subheading Other items (optional)
540 Regression: it used to @strong{deliberately} work in an earlier
541 stable release. If the earlier output was accidental (i.e. we
542 didn't try to stop a collision, but it just so happened that two
543 grobs didn't collide), then breaking it does not count as a
547 Patch: a patch to fix an issue is attached.
550 Frog: the fix is believed to be suitable for a new contributor
551 (does not require a great deal of knowledge about LilyPond). The
552 issue should also have an estimated time in a comment.
555 Maintainability: hinders development of LilyPond. For example,
556 improvements to the build system, or @qq{helper} python scripts.
559 Bounty: somebody is willing to pay for the fix. Only add this tag
560 if somebody has offered an exact figure in US dollars or euros.
563 Warning: graphical output is fine, but lilypond prints a
564 false/misleading warning message. Alternately, a warning should
565 be printed (such as a bar line error), but was not. Also applies
566 to warnings when compiling the source code or generating
570 Security: might potentially be used.
573 Performance: might potentially be used.
577 If you particularly want to add an label not in the list, go
578 ahead, but this is not recommended.
581 @node Adding issues to the tracker
582 @section Adding issues to the tracker
584 @warning{This should only be done by the Bug Squad or experienced
585 developers. Normal users should not do this; instead, they should
586 follow the guidelines for @rweb{Bug reports}.}
588 In order to assign labels to issues, Bug Squad members should log
589 in to their google account before adding an item.
591 @subsubheading Normal issues
596 Check if the issue falls into any previous category given on the
597 relevant checklists in @ref{Bug Squad checklists}. If in doubt,
598 add a new issue for a report. We would prefer to have some
599 incorrectly-added issues rather than lose information that should
603 Add the issue and classify it according to the guidelines in
604 @ref{Issue classification}. In particular, the item should have
605 @code{Status}, @code{Type-}, and @code{Priority-} labels.
607 Include output with the first applicable method:
612 If the issue has a notation example which fits in one system,
613 generate a small @file{bug.preview.png} file with:
616 lilypond -dpreview bug.ly
620 If the issue has an example which requires more than one system
621 (i.e. a spacing bug), generate a @file{bug.png} file with:
624 lilypond --png bug.ly
628 If the issue requires one or two pages of output, then generate a
629 @file{bug.png} file with the normal:
632 lilypond --png bug.ly
636 If the issue cannot be shown with less than three pages, then
637 generate a @file{bug.pdf} file with:
640 lilypond --pdf bug.ly
643 Note that this is likely to be extremely rare; most bugs should fit
644 into the first two categories above.
650 After adding the issue, please send a response email to the same
651 group(s) that the initial patch was sent to. If the initial email
652 was sent to multiple mailing lists (such as both @code{user} and
653 @code{bugs}), then reply to all those mailing lists as well. The
654 email should contain a link to the issue you just added.
659 @subsubheading Patch reminders
661 @warning{This is not a Bug Squad responsibility; we have a
662 separate person handling this task.}
664 There is a special category of issues: reminders of an existing
665 patch. These should be added if a patch has been sent to a
666 lilypond mailing list (generally @code{lilypond-devel}, but they
667 sometimes appear on @code{bug-lilypond} as well) and has had no
668 discussion for at least @strong{3 days}. Do not add issues for
669 patches under active discussion.
671 Before adding a patch-reminder issue, do a quick check to see if
672 it was pushed without sending any email. This can be checked for
673 searching for relevant terms (from the patch subject or commit
674 message) on the webgit page:
677 @uref{http://git.savannah.gnu.org/gitweb/?p=lilypond.git}
680 After adding the issue, please send a response email to the same
681 group(s) that the initial patch was sent to. If the initial email
682 was sent to multiple mailing lists (such as both @code{bugs} and
683 @code{devel}), then reply to all those mailing lists as well. The
684 email should contain a link to the issue you just added.
688 @node Summary of project status
689 @section Summary of project status
691 The best overview of our current status is given by the grid view:
694 @uref{http://code.google.com/p/lilypond/issues/list?mode=grid&y=Priority&x=Type&cells=ids}
697 Also of interest might be the issues hindering future development:
700 @uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:Maintainability&mode=grid&y=Priority&x=Type&cells=ids}
703 Finally, issues tagged with @code{Frog} indicates a task suitable
704 for a relatively new contributor. The time given is a quick
705 (inaccurate) estimate of the time required for somebody who is
706 familiar with material in this manual, but does not know anything
707 else about LilyPond development.
710 @uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:Frog&mode=grid&y=Priority&x=Type&cells=ids}
714 @node Finding the cause of a regression
715 @section Finding the cause of a regression
717 @warning{This is not a @qq{simple} task; it requires a fair amount
718 of technical knowledge.}
720 Git has special functionality to help tracking down the exact
721 commit which causes a problem. See the git manual page for
722 @code{git bisect}. This is a job that non-programmers can do,
723 although it requires familiarity with git, ability to compile
724 LilyPond, and generally a fair amount of technical knowledge. An
725 in-depth explanation of this process will not be given here.
727 Even if you are not familiar with git or are not able to compile
728 LilyPond you can still help to narrow down the cause of a
729 regression simply by downloading the binary releases of different
730 LilyPond versions and testing them for the regression. Knowing
731 which version of LilyPond first exhibited the regression is
732 helpful to a developer as it shortens the @code{git bisect}
733 procedure described above.
735 Once a problematic commit is identified, the programmers' job is
736 much easier. In fact, for most regression bugs, the majority of
737 the time is spent simply finding the problematic commit.
739 More information is in @ref{Regression tests}.