1 Elu@c -*- coding: utf-8; mode: texinfo; -*-
5 This chapter deals with defects, feature requests, and
6 miscellaneous development tasks.
9 * Introduction to issues::
11 * Issue classification::
12 * Adding issues to the tracker::
14 * Summary of project status::
18 @node Introduction to issues
19 @section Introduction to issues
21 @warning{All the tasks in this chapter require no programming skills and
22 can be done by anyone with a web browser, an email client and the
23 ability to run LilyPond.}
25 The term @q{issues} refers not just to software bugs but also includes
26 feature requests, documentation additions and corrections as well as any
27 other general code @q{TODOs} that need to be kept track of.
31 @section The Bug Squad
35 * Bug Squad checklists::
38 To help keep track and organize all issues are a group of tireless
39 volunteers collectively known as the @emph{Bug Squad}. Composed mainly
40 of non-programmers, the Bug Squad's responsibilities include:
45 Monitoring the LilyPond Bugs mailing list looking for any issues
46 reported by other users ensuring that they are accurate and contain
47 enough information for the developers to work with, preferably with
48 @rweb{Tiny examples} and if applicable, screenshots.
51 Adding new issues to the @emph{issue tracker} or updating existing
52 issues with new information.
55 Verifying issues in the @emph{issue tracker} that have been marked
56 as @q{fixed}; making sure either that the fix works or (in the case of
57 Documentation for example) has at least been commited to the code base.
61 The @ref{Meisters, Bug Meister} also helps check the current
62 @ref{Regression tests} and highlights any significant changes (or
63 problems) since the previous LilyPond release.
65 If you would like to be part of the Bug Squad, please contact the
66 @ref{Meisters, Bug Meister}.
70 @subsection Bug Squad setup
72 We highly recommend that you configure your email client to use some
73 kind of sorting and filtering as this will significantly reduce and
74 simplify your workload. Suggested email folder names are mentioned
75 below to work when sorted alphabetically.
80 Read every section of the @ref{Issues} chapter in this guide.
83 Subscribe your email account to @code{bug-lilypond}. See
84 @uref{https://lists.gnu.org/mailman/listinfo/bug-lilypond}.
87 Send your email address to the @ref{Meisters, Bug Meister}.
90 Create your own Sourceforge login (required for the Allura issue
96 Go to @uref{https://sourceforge.net/p/testlilyissues/issues/}
99 Click on 'Join' in the top-right corner.
102 Fill in your details as required and click the @emph{Register} button to
103 complete the registration.
108 Send your Sourceforge @emph{username} (not your email address) to
109 @email{bug-lilypond@@gnu.org} asking to be given appropriate permissions
110 to either create, edit and comment on tracker issues.
113 Configure your email client:
118 Any email sent with your address in the @code{To:} or @code{CC:} fields
119 should be configured to go into a @code{bug-answers} folder.
122 Any email either @code{From:} or @code{CC:} to,
125 testlilyissues-auto@@lists.sourceforge.net
129 should be configured to go into a @code{bug-ignore} folder or,
130 alternately, configure your email client to delete these automatically.
131 You do @emph{not} need to read mails in the @code{bug-ignore} folder.
132 If you are curious (and have time) then read them, but they are not
133 necessary for Bug Squad work.
136 Any email sent @code{To:} or @code{CC:} to,
143 should be configured to go into a @code{bug-current} folder.
150 @node Bug Squad checklists
151 @subsection Bug Squad checklists
153 When you do Bug Squad work, start at the top of this page and work
154 your way down. Stop when you've done 20 minutes.
156 Please use the email sorting described in @ref{Bug Squad setup}.
157 This means that (as Bug Squad members) you will only ever respond
158 to emails sent or CC'd to the @code{bug-lilypond} mailing list.
161 @subsubheading Emails to you personally
163 You are not expected to work on Bug Squad matters outside of your
164 20 minutes, but sometimes a confused user will send a bug report
165 (or an update to a report) to you personally. If that happens,
166 please forward such emails to the @code{bug-lilypond} list so that
167 the currently-active Bug Squad member(s) can handle the message.
170 @subsubheading Daily schedule as of July 2015
173 Monday: Federico Bruni
174 Tuesday: Graham Percival
175 Wednesday: Simon Albrecht
176 Thursday: Colin Campbell
178 Saturday: Colin Campbell
179 Sunday: Graham Percival
183 @subsubheading Emails to @code{bug-answers}
185 Some of these emails will be comments on issues that you added to
189 If they are asking for more information, give the additional
193 If the email says that the issue was classified in some other
194 manner, read the rationale given and take that into account for
195 the next issue you add.
198 Otherwise, move them to your @code{bug-ignore} folder.
202 Some of these emails will be discussions about Bug Squad work;
206 @subsubheading Emails to @code{bug-current}
208 Dealing with these emails is your main task. Your job is to get
209 rid of these emails in the first method which is applicable:
213 If the email has already been handled by a Bug Squad member (i.e.
214 check to see who else has replied to it), delete it.
217 If the email is a question about how to use LilyPond, reply with
221 For questions about how to use LilyPond, please read our
222 documentation available from:
223 @uref{http://lilypond.org/website/manuals.html}
224 or ask the lilypond-user mailing list.
228 If the email mentions @qq{the latest git}, or any version number
229 that has not yet been officially released, forward it to
230 @code{lilypond-devel}.
233 If a bug report is not in the form of a Tiny example, direct the
234 user to resubmit the report with this response:
237 I'm sorry, but due to our limited resources for handling bugs, we
238 can only accept reports in the form of Tiny examples. Please see
239 step 2 in our bug reporting guidelines:
240 @uref{http://lilypond.org/website/bug-reports.html}
244 If anything is unclear, ask the user for more information.
246 How does the graphical output differ from what the user expected?
247 What version of lilypond was used (if not given) and operating
248 system (if this is a suspected cause of the problem)? In short,
249 if you cannot understand what the problem is, ask the user to
250 explain more. It is the user's responsibility to explain the
251 problem, not your responsibility to understand it.
254 If the behavior is expected, the user should be told to read the
258 I believe that this is the expected behaviour -- please read our
259 documentation about this topic. If you think that it really is a
260 mistake, please explain in more detail. If you think that the
261 docs are unclear, please suggest an improvement as described by
262 @qq{Simple tasks -- Documentation} on:
263 @uref{http://lilypond.org/website/help-us.html}
267 If the issue already exists in the tracker, send an email to that
271 This issue has already been reported; you can follow the
272 discussion and be notified about fixes here:
276 (copy+paste the google code issue URL)
279 Accept the report as described in
280 @ref{Adding issues to the tracker}.
284 All emails should be CC'd to the @code{bug-lilypond} list so that
285 other Bug Squad members know that you have processed the email.
287 @warning{There is no option for @qq{ignore the bug report} -- if
288 you cannot find a reason to reject the report, you must accept
293 @c Try omitting this from Bug Squad duties
295 @subheading Updates / discussion about issues
297 We try to keep discussions about issues on the tracker, but
298 sometimes it spills over onto email. If discussion has ended with
299 no patch / resolution and at least @strong{3 days} have passed,
305 Summarize the recent discussion on the tracker, and add a link to
306 the original discussion.
309 Add the comment @qq{there was some technical discussion which I
310 could not understand}, and include a link to the original
313 We do not expect Bug Squad members to be programmers, or even to
314 be moderately-skilled users. Your job is to keep track of issue
315 reports; it is @emph{perfectly acceptable} to not understand
316 discussions between advanced users and/or developers.
322 @subheading Regular maintenance
324 After @strong{every release} (both stable and unstable):
329 Issues to verify: go to
332 @uref{https://sourceforge.net/p/testlilyissues/issues/search/?q=status%3AFixed}
335 (You can also generate this list by selecting the @qq{Open (Fixed)}
336 button down the left-hand frame)
338 You should see a list of Issues that have been marked as 'Fixed' by a
339 developer. If the developer has done their job properly, the
340 Issue should have the @qq{Labels} field filled in with @qq{Fixed_x_y_z},
341 where X is the major version, y the minor version and z the current
348 This will help you work out which you can verify - do not verify any
349 Issues where the claimed fixed build is not yet released. Work your
350 way through these as follows:
352 If the Issue refers to a bug, try to reproduce the bug with the latest
353 officially released version (not one you've built yourself from
354 source); if the bug is no longer there, mark the
355 issue @qq{Verified} (i.e. @qq{the fix has been verified to work}).
357 Quite a few of these will be issues tracking patches. @strong{You
358 do not have to prove these patches work - simply that they have
359 been pushed into the code base.} The developer should have put
360 information similar to @qq{Pushed as as
361 d8fce1e1ea2aca1a82e25e47805aef0f70f511b9} in the tracker. The
362 long list of letters and numbers is called the @qq{committish}.
363 Providing you can find this at the git tracker:
366 @uref{http://git.savannah.gnu.org/gitweb/?p=lilypond.git}
369 then you should mark the issue as verified. A quick way of
370 finding these is to enter the committish at the following address:
373 @uref{http://philholmes.net/lilypond/git/}
376 The Issue tracker also requires that any issues labelled as
377 @qq{Duplicate} are also verified. Check that the linked issue is
378 a duplicate and verify the issue.
380 A few (approximately 10%) of the fixed issues relate to the
381 build system or fundamental architecture changes; there is no way
382 for you to verify these. Leave those issues alone; somebody else
386 The official regression test comparison is online at:
388 @c NOTE: leave this here. In this case, it's worth duplicating
391 @uref{http://lilypond.org/test/}
394 If anything has changed suspiciously,
395 ask if it was deliberate. If the text output from LilyPond (the
396 logfile) changes, the differences will be displayed with a +
397 before text added to the logfile and - before any text removed
398 from the logfile. This may or may not be suspicious.
400 There is one test designed to produce output every time the
401 regtests are created. @code{test-output-distance.ly} creates
402 randomly spaced notes and will always have different output if the
403 regtest checker is working.
405 More information is available from in
406 @ref{Precompiled regression tests}.
409 Check for any incorrectly-classified items in the tracker. This
410 generally just means looking at the grid to see any items without
417 @c try omitting from daily tasks for now. -gp
419 @subheading Irregular maintenance
421 @warning{These tasks are a lot of work; gathering more volunteers
422 to help is definitely recommended. However, the Bug Squad should
423 handle the organization and training of new volunteers.}
425 Once every year or two:
430 Checking all regtests: although we have a system for checking the
431 regtests between two versions, occasionally a bug will slip
432 through the cracks. It is therefore good to manually examine all
433 the regtests (compare the images to the text description). More
434 information is available from in @ref{Regression tests}.
438 Checking all issues: we try to mark each Issue @q{fixed} when we
439 fix it, but occasionally one or two issues will slip through the
440 cracks. It is therefore good to check all Issues. If you see the
441 same (broken) output as the initial report, then simply post a
442 @qq{Problem still exists in 2.x.y} message to the issue.
449 @node Issue classification
450 @section Issue classification
452 The Bug Squad should classify issues according to the guidelines
453 given by developers. Every issue should have a Status and Type;
454 the other fields are optional.
456 @subheading Status (mandatory)
463 New: the item was added by a non-member, despite numerous warnings
464 not to do this. Should be reviewed by a member of the Bug Squad.
467 Accepted: the Bug Squad added it, or reviewed the item.
470 Started: a contributor is working on a fix. Owner should change
471 to be this contributor.
481 Invalid: issue should not have been added in the current state.
484 Duplicate: issue already exists in the tracker.
487 Fixed: a contributor claims to have fixed the bug. The Bug
488 Squad should check the fix with the next official binary release
489 (not by compiling the source from git). Owner should be set to
493 Verified: Bug Squad has confirmed that the issue is closed. This
494 means that nobody should ever need look at the report again -- if
495 there is any information in the issue that should be kept, open a
496 new issue for that info.
501 @subheading Owner (optional)
503 Newly-added issues should have @emph{no owner}. When a
504 contributor indicates that he has Started or Fixed an item, he
505 should become the owner.
508 @subheading Type (mandatory)
510 The issue's Type should be the first relevant item in this list.
515 Type-Critical: normally a regression
516 against the current stable version or the previous stable version.
517 Alternatively, a regression against a fix developed for the
518 current version. This does not apply where the
519 @qq{regression} occurred because a feature was removed
520 deliberately - this is not a bug.
522 Currently, only Critical items will block a stable release.
525 Type-Maintainability: hinders future development.
528 Type-Crash: any input which produces a crash.
531 Type-Ugly: overlapping or other ugly notation in graphical output.
534 Type-Defect: a problem in the core program. (the @code{lilypond}
535 binary, scm files, fonts, etc).
538 Type-Documentation: inaccurate, missing, confusing, or desired
539 additional info. Must be fixable by editing a texinfo, ly, or scm
543 Type-Build: problem or desired features in the build system. This
544 includes the makefiles, stepmake, python scripts, and GUB.
547 Type-Scripts: problem or desired feature in the non-build-system
548 scripts. Mostly used for convert-ly, lilypond-book, etc.
551 Type-Enhancement: a feature request for the core program. The
552 distinction between enhancement and defect isn't extremely clear;
553 when in doubt, mark it as enhancement.
556 Type-Patch: tracking a patch on Rietveld. Bug squad should not
557 need to use this label.
560 Type-Other: anything else.
565 @subheading Priority (mandatory)
567 Currently, only Critical items will block a stable release.
572 Priority-Critical: LilyPond segfaults, a regression (see below)
573 against a previous stable version or a regression against a fix
574 developed for this version. This does not apply where the
575 @qq{regression} occurred because a feature was removed
576 deliberately - this is not a bug.
579 Priority-High: An issue which produces output which does not
580 accurately reflect the input (e.g. where the user would expect
581 an accidental, but none is shown) or which produces aesthetically
582 poor output in a situation which could be expected to crop up
583 frequently in real-world music. It should not be used where the
584 problem can be avoided with a simple workaround. It can also
585 be used to flag where new code in a development version is not
586 functioning as it should. This level is also used for issues
587 which produce no output and fail to give the user a clue about
591 Priority-Medium: Normal priority - use this as the default.
594 Priority-Low: A minor problem which produces slightly undesirable
595 output, or which will only occur in contrived examples, or which
596 is very easily worked around.
599 Priority-Postponed: no fix planned. Generally used for things
600 which nobody wants to touch.
604 Note that these are initial classifications and can be subject
605 to change by others in the development team. For example, a
606 regression against an old stable version which hasn't been
607 noticed for a long time and which is unlikely to get fixed could
608 be downgraded from Priority-Critical by one of the programmers.
612 @subheading Opsys (optional)
614 Issues that only affect specific operating systems.
616 @subheading Patch label (optional)
618 Normal Bug Squad members should not add or modify Patch issues
619 except to verify them; for all other Patch work, leave them to the
625 Patch-new: the patch has not been checked for @qq{obvious}
626 mistakes. When in doubt, use this tag.
629 Patch-review: the patch has no @qq{obvious} mistakes (as checked
630 by the Patch Meister), and is ready for review from main
633 Developers with git push ability can use this category, skipping
634 over @code{patch-new}.
637 Patch-needs_work: a developer has some concerns about the patch.
638 This does not necessarily mean that the patch must be changed; in
639 some cases, the developer's concerns can be resolved simply by
640 discussion the situation or providing notation examples.
642 If the patch is updated, the category should be changed to
643 @code{patch-new} (for normal contributors) or @code{patch-review}
644 (for developers who are very confident about their patch).
647 Patch-countdown: final call for any patch problems
650 Patch-push: patch has passed the countdown and should be pushed.
653 Patch-abandoned: the author has not responded to review comments
658 @subheading Other items (optional)
665 Regression: it used to work intentionally in the current
666 stable release or the previous stable release. If the earlier
667 output was accidental (i.e. we didn't try to stop a collision,
668 but it just so happened that two grobs didn't collide), then
669 breaking it does not count as a regression.
671 To help decide whether the change is a regression, please adopt
672 the following process:
677 Are you certain the change is OK? If so, do nothing.
680 Are you certain that the change is bad? Add it to the tracker
684 If you're not certain either way, add it to the tracker as a
685 regression but be aware that it may be recategorised or marked
690 In particular, anything that breaks a regression test is a
694 Frog: the fix is believed to be suitable for a new contributor
695 (does not require a great deal of knowledge about LilyPond). The
696 issue should also have an estimated time in a comment.
699 Bounty: somebody is willing to pay for the fix. Only add this tag
700 if somebody has offered an exact figure in US dollars or euros.
703 Warning: graphical output is fine, but lilypond prints a
704 false/misleading warning message. Alternately, a warning should
705 be printed (such as a bar line error), but was not. Also applies
706 to warnings when compiling the source code or generating
710 Security: security risk.
713 Performance: performance issue.
717 If you particularly want to add a label not in the list, go
718 ahead, but this is not recommended, except when an issue is marked
719 as fixed. In this case it should be labeled Fixed_mm_MM_ss,
720 where mm is major version, MM minor version and ss current
724 @node Adding issues to the tracker
725 @section Adding issues to the tracker
727 @warning{This should only be done by the Bug Squad or experienced
728 developers. Normal users should not do this; instead, they should
729 follow the guidelines for @rweb{Bug reports}.}
731 In order to assign labels to issues, Bug Squad members should log
732 in to their google account before adding an item.
737 Check if the issue falls into any previous category given on the
738 relevant checklists in @ref{Bug Squad checklists}. If in doubt,
739 add a new issue for a report. We would prefer to have some
740 incorrectly-added issues rather than lose information that should
744 Add the issue and classify it according to the guidelines in
745 @ref{Issue classification}. In particular, the item should have
746 @code{Status} and @code{Type-} labels.
748 Include output with the first applicable method:
753 If the issue has a notation example which fits in one system,
754 generate a small @file{bug.preview.png} file with:
757 lilypond -dpreview bug.ly
761 If the issue has an example which requires more than one system
762 (i.e. a spacing bug), generate a @file{bug.png} file with:
765 lilypond --png bug.ly
769 If the issue requires one or two pages of output, then generate a
770 @file{bug.png} file with the normal:
773 lilypond --png bug.ly
777 Images created as @file{bug.png} may be trimmed to a minimum size
778 by using the @code{trimtagline.sh} script, which can be found at
781 @uref{https://raw.github.com/gperciva/lilypond-extra/master/bug-squad/trimtagline.sh}
785 trimtagline.sh bug.ly
789 If the issue cannot be shown with less than three pages, then
790 generate a @file{bug.pdf} file with:
793 lilypond --pdf bug.ly
796 Note that this is likely to be extremely rare; most bugs should
797 fit into the first two categories above.
803 After adding the issue, please send a response email to the same
804 group(s) that the initial patch was sent to. If the initial email
805 was sent to multiple mailing lists (such as both @code{user} and
806 @code{bugs}), then reply to all those mailing lists as well. The
807 email should contain a link to the issue you just added.
813 @section Patch handling
815 @warning{This is not a Bug Squad responsibility; we have a
816 separate person handling this task.}
818 For contributors/developers: follow the steps in
819 @ref{Patches}, and @ref{Pushing to staging}.
822 For people doing maintenance tasks: git-cl is adding issues, James
823 is testing patches and managing the Patch countdown. He also generally
824 runs the scripts that merging to Staging (although other developers are
825 available to do this task if required).
829 @node Summary of project status
830 @section Summary of project status
832 @subsubheading Project overview
837 @uref{https://sourceforge.net/projects/testlilyissues/}
840 @subsubheading Hindering development
842 These issues stop or slow development work:
845 @uref{https://sourceforge.net/p/testlilyissues/issues/search/?q=status:Accepted%20AND%20_type:Maintainability}
848 @subsubheading Easy tasks
850 Issues tagged with @code{Frog} indicates a task suitable for a
851 relatively new contributor. The time given is a quick (and probably
852 inaccurate) estimate of the time required for somebody who is familiar
853 with material in this manual, but does not know anything else about
854 LilyPond development.
857 @uref{https://sourceforge.net/p/testlilyissues/issues/search/?q=status:Accepted%20AND%20labels:Frog}
860 @subsubheading Patches currently in the Patch Review cycle
864 @c The following URL is provided by one of the Developers giving a much
865 @c easier way to see all patches at all stages of the Review cycle in a
869 http://philholmes.net/lilypond/allura/
877 @uref{https://sourceforge.net/p/testlilyissues/issues/search/?q=status%3AStarted+AND+_patch%3Anew}
885 @uref{https://sourceforge.net/p/testlilyissues/issues/search/?q=status%3AStarted+AND+_patch%3Areview}
890 Patches on final Countdown
893 @uref{https://sourceforge.net/p/testlilyissues/issues/search/?q=status%3AStarted+AND+_patch%3Acountdown}
898 Patches that can be pushed
901 @uref{https://sourceforge.net/p/testlilyissues/issues/search/?q=status%3AStarted+AND+_patch%3Apush}