1 @c -*- coding: utf-8; mode: texinfo; -*-
2 @node Administrative policies
3 @chapter Administrative policies
5 This chapter discusses miscellaneous administrative issues which
6 don't fit anywhere else.
9 * Meta-policy for this document::
10 * Environment variables::
13 * Administrative mailing list::
14 * Grand Organization Project (GOP)::
15 * Grand LilyPond Input Syntax Standardization (GLISS)::
19 @node Meta-policy for this document
20 @section Meta-policy for this document
22 The Contributor's Guide as a whole is still a work in progress,
23 but some chapters are much more complete than others. Chapters
24 which are @qq{almost finished} should not have major changes
25 without a discussion on @w{@code{-devel}}; in other chapters, a
26 disorganized @qq{wiki-style dump} of information is encouraged.
28 Do not change (other than spelling mistakes) without discussion:
33 @ref{Introduction to contributing}
36 @ref{Working with source code}
40 Please dump info in an appropriate @@section within these manuals,
41 but discuss any large-scale reorganization:
49 @ref{Documentation work}
55 @ref{Regression tests}
58 @ref{Programming work}
63 Totally disorganized; do whatever the mao you want:
77 @ref{Administrative policies}
82 @node Environment variables
83 @section Environment variables
85 Some maintenance scripts and instructions in this guide rely on
86 the following environment variables. They should be predefined in
87 LilyDev distribution (see @ref{LilyDev}); if you set up your own
88 development environment, you can set them by appending these settings to
89 your @file{~/.bashrc} (or whatever defines your default environment
90 variables for the user account for LilyPond development), then logging
91 out and in (adapt directories to your setup):
94 LILYPOND_GIT=~/lilypond-git
96 LILYPOND_BUILD_DIR=~/lilypond-git/build
97 export LILYPOND_BUILD_DIR
100 The standard build and install procedure (with @code{autogen.sh},
101 @code{configure}, @code{make}, @code{make install}, @code{make doc}
102 @dots{}) does not rely on them.
104 In addition, for working on the website, @code{LILYPOND_WEB_MEDIA_GIT}
105 should be set to the repository lilypond-extra, see
106 @ref{lilypond-extra}.
112 We have four jobs for organizing a team of contributors:
117 Bug Meister: trains new Bug Squad volunteers, organizes who works
118 on which part of their job, checks to make sure that everything is
119 running smoothly, and has final say on our policy for Issues.
124 Doc Meister: trains new doc editors/writers, organizes who works
125 on which part of the job, checks to make sure that everything is
126 running smoothly, and has final say on our policy for
127 Documentation. Also includes LSR work.
132 Translation Meister: trains new translators, updates the
133 translation priority list, and handles merging branches (in both
143 @subheading Introduction
145 Patchy is a set of Python scripts to automate two administrative
150 @code{lilypond-patchy-staging.py}: checks that new commits in
151 @code{staging} can compile the regtests and documentation before
152 merging @code{staging} into @code{master}.
154 (completely automatic)
157 @code{test-patches.py}: checks that patches apply to Git @code{master},
158 compile, and lets a human check that there are no big unintended
159 changes to the regtests.
161 (requires some human input)
165 @subheading Installing Patchy
167 To install Patchy, you should do the following:
171 Create a new user on your box to run Patchy; this is a security
172 step for your own protection. It is recommended that this should
173 not be an administrator. New users are created from System;
174 Administration; Users and Groups.
177 Get the Patchy scripts from
179 @uref{https://github.com/gperciva/lilypond-extra/}
181 Patchy is in the @file{patches/} directory.
184 Put the scripts and Python libraries contained in @file{patches} in a
185 sensible place on your system; this can be done by appending
186 @file{patches/} full path to the @var{PATH} of the user that runs
190 Create a new git repository with
192 git clone git://git.sv.gnu.org/lilypond.git
194 This will create a directory called lilypond with the repo in it.
195 Make sure it's where you want it and name it lilypond-git
196 (assuming you want to follow the standard naming conventions).
199 Create environment variables @var{LILYPOND_GIT} and
200 @var{LILYPOND_BUILD_DIR}, see @ref{Environment variables}.
203 Run Patchy once to set up config files, answer @q{@code{n}} when it
204 asks for going on, unless the default config file happens to suit your
207 lilypond-patchy-staging.py
211 Edit @file{$HOME/.lilypond-patchy-config} to provide the location of
212 your local lilypond Git repository, working directories for your build
213 directory, your results directory, compiler options and notification
214 method. If you don't want to use email notification, then delete
215 everything after @code{smtp_command:}.
218 Ensure that your new user has git push access. Follow the
219 instructions in the CG at @ref{Commit access}. Do not set
220 password protection for the key --- if you do you will not be able
221 to run patchy unattended.
225 @subheading lilypond-patchy-staging.py
227 @code{lilypond-patchy-staging.py} is run with
229 python lilypond-patchy-staging.py
231 Not much appears to happen except you can see a lot of CPU gets used
232 if you open System Monitor. There's not much point running
233 @code{lilypond-patchy-staging.py} unless there is something in
234 @code{staging} to be merged to @code{master}, however, if there's
235 nothing new in @code{staging} then the script won't waste resources by
238 The script fetches the current patches in staging and runs
239 @code{make}, @code{make test} and @code{make doc} to ensure that all of
240 these complete error-free. If you have set Patchy up to use email,
241 it emails its results to you. If you haven't, then you can view
242 them in a logfile. It also merges @code{staging} into @code{master}.
244 @warning{in case the build fails, do not try to push fixes on top of
245 staging branch, for details see @ref{Pushing to staging}.}
247 When you have run Patchy a few successful times with email sending,
248 you are ready for running it as a cron job. First, make sure you have
249 the following in @file{$HOME/.lilypond-patchy-config} to avoid
254 notify_non_action = no
257 Then, assuming Patchy run with user account @code{patchy}, write the
258 following to @file{$HOME/lilypond-patchy.cron}, adapting it as
259 necessary (the @code{/2} means @qq{run this every 2 hours}):
261 02 0-23/2 * * * /home/patchy/git/lilypond-extra/patches/lilypond-patchy-staging.py
264 @warning{@code{cron} will not inherit environment variables from
265 your main setup, so you must re-define any variables inside
266 @file{$HOME/lilypond-patchy.cron}. For instance, @var{LILYPOND_GIT}
267 may need to be defined if @var{git_repository_dir} is not correctly
268 set in @file{$HOME/.lilypond-patchy-config}.}
270 Finally, install the cron job (you may need superuser privileges for
273 crontab -u patchy /home/patchy/lilypond-patchy.cron
276 @subheading test-patches.py
277 @code{test-patches.py} prepares a regtest comparison for a human to
278 quickly glance at, to determine if the patch is ready for a review.
279 After looking at the comparison (or the lack of a comparison in the
280 case of problems), run @code{accept-patch.py} or
281 @code{reject-patch.py}.
283 Once a patch has gotten a "LGTM" from Patchy, it should be
284 reviewed by relevant developers, and if it passes this, it can be
285 considered for countdown (see @ref{Commits and patches}) and
286 pushing to staging (see @ref{Pushing to staging}).
289 @node Administrative mailing list
290 @section Administrative mailing list
292 A mailing list for administrative issues is maintained at
293 @code{lilypond-hackers@@gnu.org}.
295 This list is intended to be used for discussions that should be kept
296 private. Therefore, the archives are closed to the public.
298 Subscription to this list is limited to certain senior developers.
300 At the present time, the list is dormant.
302 Details about the criteria for membership, the types of discussion
303 to take place on the list, and other policies for the hackers list
304 will be finalized during the
305 @ref{Grand Organization Project (GOP)}.
309 @node Grand Organization Project (GOP)
310 @section Grand Organization Project (GOP)
316 Clarify the various development tasks by writing down the policies
317 and techniques and/or simplifying the tasks directly.
320 Get more people involved in development: specifically, find people
321 to do easy tasks to allow advanced developers to concentrate on
330 * Policy decisions (finished)::
334 @subsection Motivation
336 Most readers are probably familiar with the LilyPond Grand
337 Documentation Project, which ran from Aug 2007 to Aug 2008. This
338 project involved over 20 people and resulted in an almost complete
339 rewrite of the documentation. Most of those contributors were
340 normal users who decided to volunteer their time and effort to
341 improve lilypond for everybody. By any measure, it was a great
344 The Grand Organization Project aims to do the same thing with a
345 larger scope -- instead of focusing purely on documentation, the
346 project aims to improve all parts of LilyPond and its community.
347 Just as with GDP, the main goal is to encourage and train users to
348 become more involved.
350 If you have never contributed to an open-source project before --
351 especially if you use Windows or OSX and do not know how to
352 program or compile programs -- you may be wondering if there's
353 anything you can do. Rest assured that you @emph{can} help.
355 @subheading "Trickle-up" development
357 One of the reasons I'm organizing GOP is "trickle-up"
358 development. The idea is this: doing easy tasks frees up advanced
359 developers to do harder tasks. Don't ask "am I the @emph{best}
360 person for this job"; instead, ask "am I @emph{capable} of doing
361 this job, so that the current person can do stuff I @emph{can't}
364 For example, consider lilypond's poor handling of grace notes in
365 conjunction with clef and tempo changes. Fixing this will require
366 a fair amount of code rewriting, and would take an advanced
367 developer a few weeks to do. It's clearly beyond the scope of a
368 normal user, so we might as well sit back and do nothing, right?
370 No; we @emph{can} help, indirectly. Suppose that our normal user
371 starts answering more emails on lilypond-user. This in turn means
372 that documentation writers don't need to answer those emails, so
373 they can spend more time improving the docs. I've noticed that all
374 doc writers tackle harder and harder subjects, and when they start
375 writing docs on scheme programming and advanced tweaks, they start
376 contributing bug fixes to lilypond. Having people performing these
377 easy-to-moderate bug fixes frees up the advanced developers to
378 work on the really hard stuff... like rewriting the grace note
381 Having 1 more normal user answering emails on lilypond-user won't
382 have a dramatic @q{trickle-up} effect all by itself, of course. But if
383 we had 8 users volunteering to answer emails, 6 users starting to
384 write documentation, and 2 users editing LSR... well, that would
385 free up a lot of current bug-fixing-capable contributors to focus
386 on that, and we could start to make a real dent in the number of
387 bugs in lilypond. Quite apart from the eased workload, having that
388 many new helpers will provide a great moral boost!
391 @subsection Ongoing jobs
393 Although GOP is a short-term project, the main goal is to train
394 more people to handle ongoing jobs. The more people doing these
395 jobs, the lighter the work will be, and the more we can get done
398 Also, it would be nice if we had at least one "replacement" /
399 "understudy" for each role -- too many tasks are only being done
400 by one person, so if that person goes on vacation or gets very
401 busy with other matters, work in that area grinds to a halt.
403 @subheading Jobs for normal users
407 LilyPond is sometimes critized for not listening to users, but
408 whenever we ask for opinions about specific issues, we never get
409 enough feedback. This is somewhat aggravating.
410 We need a group of users to make a dedicated effort to test and
411 give feedback. If there's new documentation, read it. If there's
412 an experimental binary, download it and try compiling a score with
413 it. If we're trying to name a new command, think about it and give
416 @item lilypond-user support:
417 I think it would be nice if we had an official team of users
420 @item LilyPond Report:
421 Keeping a monthly newsletter running is a non-trivial task. A lot
422 of work is needed to organize it; it would be great if we could
423 split up the work. One person could write the Snippet of the
424 Month, another person could do Quotes of the Month, another person
425 could do interviews, etc.
428 Although GDP (the Grand Documentation Project) did great work,
429 there's still many tasks remaining.
432 Keeping the documentation translations is a monumental task; we
433 need all the help we can get!
437 @subheading Jobs for advanced users for developers
440 @item Git help for writers:
441 We often receive reports of typos and minor text updates to the
442 documentation. It would be great if somebody could create
443 properly-formatted patches for these corrections.
445 Technical requirements: ability to run @ref{LilyDev}.
448 LSR contains many useful examples of lilypond, but some snippets
449 are out of date and need updating. Other snippets need to be
450 advertized, and new snippets need to be sorted. We could use
451 another person to handle LSR.
453 Technical requirements: use of a web browser. LilyPond
454 requirements: you should be familiar with most of Notation
455 chapters 1 and 2 (or be willing to read the docs to find out).
457 @item Join the Frogs:
458 "Frogs" are a team of bug-fixers (because frogs eat bugs, and you
459 often find them in Ponds of Lilies) and new feature implementors.
461 Technical requirements: development environment (such as
462 @ref{LilyDev}), ability to read+write scheme and/or C++ code.
467 @node Policy decisions
468 @subsection Policy decisions
470 There are a number of policy decisions -- some of them fairly
471 important -- which we have been postponing for a few years. We
472 are now discussing them slowly and thoroughly; agenda and exact
473 proposals are online:
476 @uref{http://lilypond.org/~graham/gop/index.html}
479 Below is a list of policies which are not @qq{on the agenda} yet.
481 Note that the presence of an item on this list does @emph{not}
482 mean that everybody thinks that something needs to be done.
483 Inclusion in this simply means that one developer thinks that we
484 should discuss it. We are not going to filter this list; if any
485 developer thinks we should discuss something, just add it to the
486 bottom of the list. (the list is unsorted)
488 As GOP progresses, items from this list will be put on the agenda
489 and removed from this list. I generally try to have one month's
490 discussion planned in advance, but I may shuffle things around to
491 respond to any immediate problems in the developer community.
493 There are some item(s) not displayed here; these are questions
494 that were posed to me privately, and I do not feel justified in
495 discussing them publicly without the consent of the person(s) that
496 brought them up. They will initially be discussed privately on the
497 lilypond-hackers mailing list -- but the first question will be
498 "do we absolutely need to do this privately", and if not, the
499 discussion will take place on lilypond-devel like the other items.
501 In most policy discussions in lilypond over the past few years,
502 the first half (or more) is wasted arguing on the basis of
503 incorrect or incomplete data; once all the relevant facts are
504 brought to light, the argument is generally resolved fairly
505 quickly. In order to keep the GOP discussions focused, each topic
506 will be introduced with a collection of relevant facts and/or
507 proposals. It is, of course, impossible to predict exactly which
508 facts will be relevant to the discussion -- but spending an hour
509 or two collecting information could still save hours of
512 @warning{The estimated time required for "prep work", and the
513 following discussion, has been added to each item. At the moment,
514 there is an estimated 30 hours of prep work and 140 hours of
518 @item @strong{Patch reviewing}:
519 At the time of this writing, we have 23 (known) patches waiting
520 for review. Some from main developers; some from new developers.
521 We desperately need more people helping with lilypond, but
522 ignoring patches is the best way to drive potential contributors
523 away. This is not good.
525 (prep: 2 hours. discuss: 10 hours)
527 @item @strong{Official links to other organizations?}:
528 There's something called the "software freedom conservancy", and
529 in general, there's a bunch of "umbrella organizations". Joining
530 some of these might give us more visibility, possibly leading to
531 more users, more developers, maybe even financial grants or use in
534 (prep: 2 hours. discuss: 5 hours)
536 @item @strong{Issue tracking with google code}:
537 We use the google issue tracker, but this means that we are
538 relying on a commercial entity for a large part of our
539 development. Would it be better (safer in the long run) to use the
540 savannah bug tracker?
542 (prep: 1 hour. discuss: 5 hours)
544 @item @strong{Patch review tool}:
545 Reitveld is inconvenient in some respects: it requires a google
546 account, and there's no way to see all patches relating to
547 lilypond. Should we switch to something like gerritt?
548 @uref{http://code.google.com/p/lilypond/issues/detail?id=1184}
550 (prep: 5 hours. discuss: 15 hours)
552 @item @strong{Clarity for sponsorships}:
553 We currently do not advertize bounties and sponsorships on the
554 webpage. How much advertising do we want, and what type?
555 Should we change the "structure" / "framework" for bounties?
557 (prep: 2 hours. discuss: 10 hours)
559 @item @strong{code readability}:
560 "Our aim when producing source code for Lilypond in whatever
561 language is that it should be totally comprehensible to a
562 relatively inexperienced developer at the second reading."
565 - aids maintainability of code base
566 - "second reading" so newer developers can look up unfamiliar
568 - will help to keep things simple, even if the code is doing
569 complex stuff discourages "secret squirrel" coding, e.g. "how
570 much functionality can I squeeze into as few characters as
571 possible" "comments are for wimps"
572 - will aid not *discouraging* new developers to join the project
574 (prep: 2 hours. discuss: 10 hours)
576 @item @strong{C++ vs. scheme}:
577 what should be in scheme, what should be in C++, what can/should
578 be ported from one to the other, etc. Questions of
579 maintainability, speed (especially considering guile 2.0), and the
580 amount of current material in either form, are important.
582 (prep: 5 hours. discuss: 15 hours)
584 @item @strong{always make an issue number for patches}:
585 there is a proposal that we should always have a google code issue
586 number for every patch. This proposal is closely tied to our
587 choice of patch review tool; if we switch to a different tool (as
588 suggested in a different proposal), this proposal may become moot.
590 (prep: 1 hour. discuss: 5 hours)
592 @item @strong{initalizer lists}:
593 shoudl we use initalizer lists for C++? AFAIK they make no
594 difference for built-in types, but there's some weird case where
595 it's more efficient for objects, or something.
597 Probably not worth making this a weekly thing on its own, but we
598 can probably wrap it up with some other code-related questions.
600 (prep: 15 minutes. discuss: 3 hours)
604 @node Policy decisions (finished)
605 @subsection Policy decisions (finished)
607 Here is a record the final decisions, along with links to the
611 * GOP-PROP 1 - python formatting::
612 * GOP-PROP 2 - mentors and frogs::
613 * GOP-PROP 3 - C++ formatting::
614 * GOP-PROP 4 - lessons from 2.14::
615 * GOP-PROP 5 - build system output (not accepted)::
616 * GOP-PROP 6 - private mailing lists::
617 * GOP-PROP 7 - developers as resources::
618 * GOP-PROP 8 - issue priorities::
619 * GOP-PROP 9 - behavior of make doc::
622 @node GOP-PROP 1 - python formatting
623 @subsubsection GOP-PROP 1 - python formatting
625 We will follow the indentation described in PEP-8.
626 @uref{http://www.python.org/dev/peps/pep-0008/}
630 use 4 spaces per indentation level
633 never mix tabs and spaces (for indentation)
636 Code indented with a mixture of tabs and spaces should be
637 converted to using spaces exclusively
639 Once this is done, we should add @code{python -tt} to the build
640 system to avoid such errors in the future.
644 There should be absolutely no tab characters for indentation in
645 any @code{.py} file in lilypond git. All such files should be
646 converted to use spaces only.
648 @subsubheading Discussions
651 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00060.html}
652 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00084.html}
653 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00310.html}
654 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00574.html}
658 @node GOP-PROP 2 - mentors and frogs
659 @subsubsection GOP-PROP 2 - mentors and frogs
661 Nothing much was decided. The list of responsibilities was
662 slightly altered; see the new one in @ref{Mentors}. We should
663 encourage more use of the Frogs mailing list. There's a list of
664 contributor-mentor pairs in:
667 @uref{https://github.com/gperciva/lilypond-extra/blob/master/people/mentors.txt}
670 That's pretty much it.
672 @subsubheading Discussions
675 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00311.html}
681 @node GOP-PROP 3 - C++ formatting
682 @subsubsection GOP-PROP 3 - C++ formatting
684 Speaking academically, C++ code style is a "solved problem". Let's
685 pick one of the existing solutions, and let a computer deal with
686 this. Humans should not waste their time, energy, and creativity
687 manually adding tabs or spaces to source code.
689 We have modified @code{fixcc.py} to use astyle, along with extra
694 the final script will be run @strong{blindly} on the lilypond
695 source code. We will accept whatever formatting the final version
696 of this script produces, with no manual tweaking.
699 patches which have been run through this tool will not be rejected
700 for style reasons. Any code formatting @qq{desires} which are not
701 enforced by @code{fixcc.py} will not be considered grounds for
705 for now, this style will not be enforced. It is not cause for
706 concern if patches which do not follow the formatting done by
707 @code{fixcc.py} are pushed. From time to time, Graham will run
708 the formatter on the entire code base, and commit the resulting
711 In a few months, we will tighten up this policy item (with some
712 sort of automatic processing), but that is outside the scope of
713 this policy item and is a matter for later discussion.
716 after the proposal is accepted, we will leave some time for
717 existing patches to be accepted and pushed. The script was
718 run on the source code on @strong{2011 August 01}.
724 LilyPond is a GNU project, so it makes sense to follow the GNU
725 coding standards. These standards state:
728 We don’t think of these recommendations as requirements, because
729 it causes no problems for users if two different programs have
730 different formatting styles.
732 But whatever style you use, please use it consistently, since a
733 mixture of styles within one program tends to look ugly. If you
734 are contributing changes to an existing program, please follow the
735 style of that program.
738 (@uref{http://www.gnu.org/prep/standards/html_node/Formatting.html})
740 With that in mind, we do not think that we must blindly follow the
741 formatting given by the currrent version of Emacs.
743 @subheading Implementation notes
745 We can avoid some of the style change pollution in git history by
746 ignoring whitespaces changes:
752 @subsubheading Discussions
755 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00526.html}
756 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00796.html}
757 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00200.html}
758 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00525.html}
759 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00751.html}
760 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00751.html}
764 @node GOP-PROP 4 - lessons from 2.14
765 @subsubsection GOP-PROP 4 - lessons from 2.14
769 A brief history of releases:
771 @multitable @columnfractions .2 .2 .3
772 @headitem date (YYYY-MM-DD) @tab version @tab comment
773 @item 2008-10-28 @tab 2.11.63 @tab nobody checking regtests
774 @item 2008-11-17 @tab 2.11.64
775 @item 2008-11-29 @tab 2.11.65
776 @item 2008-12-23 @tab 2.12.0
777 @item 2009-01-01 @tab @tab somewhere around here, Graham becomes
778 officially release manager, but Han-Wen still builds the actual
780 @item 2009-01-01 @tab 2.12.1
781 @item 2009-01-25 @tab 2.12.2
782 @item 2009-02-28 @tab 2.13.0
783 @item 2009-06-01 @tab 2.13.1 @tab note jump in time!
784 @item 2009-06-27 @tab 2.13.2 @tab first Graham release?
785 @item 2009-07-03 @tab 2.13.3
786 @item 2009-09-09 @tab @tab Graham arrives in Glasgow, gets a
787 powerful desktop computer, and begins serious work on GUB (sending
788 bug reports to Jan). It takes approximately 100 hours until GUB
789 is stable enough to make regular releases.
790 @item 2009-09-24 @tab 2.13.4
791 @item 2009-10-02 @tab 2.13.5
792 @item 2009-10-22 @tab 2.13.6
793 @item 2009-11-05 @tab 2.13.7
795 @item 2010-01-13 @tab 2.12.3
797 @item 2010-03-19 @tab 2.13.16 @tab Bug squad starts doing a few
798 regtest comparisons, but IIRC the effort dies out after a few
801 @item 2010-08-04 @tab 2.13.29 @tab Phil starts checking regtests (BLUE)
803 @item 2011-01-12 @tab 2.13.46 @tab release candidate 1 (GREEN)
805 @item 2011-05-30 @tab 2.13.63 @tab release candidate 7 (GREEN)
806 @item 2011-06-06 @tab 2.14.0
809 @c A graphical display of bugs:
811 @c @image{bugs-2.13-visualization,png}
812 @c @image{zoom-2.13-visualization,png}
814 @subheading Carl's analysis of the bugs
816 A @file{csv} spreadsheet is available.
819 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00852.html}
823 @uref{lilypond-issues-analysis.csv}
824 @uref{lilypond-issues-analysis-trim-duplicates.csv}
827 There 148 issues marked with Priority=Critical in the tracker.
829 I've done an analysis, and it looks to me like there was initially
830 a backlog of critical issues that weren't fixed, and little work
831 was being done to eliminate critical issues.
833 Somewhere about 2010-08-01, critical issues started to disappear,
834 but occasional new ones appeared.
836 There were a couple of major changes that introduced unanticipated
837 regressions (new spacing code, beam collision avoidance). These
838 produced more than the expected number of regressions.
840 It appears to me that we didn't really get serious about
841 eliminating critical bugs until about 2010-06-15 or so. After
842 that point, the number of critical bugs more-or-less steadily
843 decreased until we got to a release candidate.
845 Of particular interest, the first release candidate of 2.14 was
846 released on 2011-01-12. Over the next 10 days, about a dozen bugs
847 were reported and fixed. Release candidate 2 came out on
848 2011-02-09. No surge of bugs occurred with this release.
849 Candidate 3 came out on 2011-03-13; we got 2 bugs per week.
850 Candidate 4 came out on 2011-03-29; 2 new bugs. Candidate 6 came
851 out on 2011-04-07. We got a couple of bugs per week.
853 @subheading Notes, commentary, and opinions
856 Han-Wen: Overall, I think this cycle took too long
861 @subsubheading Discussions
864 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00797.html}
865 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00364.html}
870 @node GOP-PROP 5 - build system output (not accepted)
871 @subsubsection GOP-PROP 5 - build system output (not accepted)
873 This proposal was too broad; after a month of discussion, Graham
874 withdrew the proposal. Portions of it will be introduced in later
877 @subsubheading Discussions
880 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00320.html}
881 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00527.html}
882 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00753.html}
883 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg01042.html}
884 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00116.html}
885 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00310.html}
889 @node GOP-PROP 6 - private mailing lists
890 @subsubsection GOP-PROP 6 - private mailing list
892 Potentially sensitive or private matters will be referred to
893 Graham. He will then decide who should discuss the matter on an
894 ad-hoc basis, and forward or CC them on future emails.
896 For emphasis, the project administrators are Han-Wen, Jan, and
897 Graham; those three will always be CC'd on any important
900 The lilypond-hackers mailing list will be removed.
904 There is some unhappy history about this idea in our development
908 @uref{http://lists.gnu.org/archive/html/lilypond-devel/2010-09/msg00178.html}
909 @uref{http://news.lilynet.net/spip.php?article121}
910 @uref{http://lists.gnu.org/archive/html/lilypond-devel/2010-11/msg00076.html}
913 @subheading Other projects
915 The idea of private mailing lists is hardly uncommon in
916 open-source software. For example,
919 @uref{http://lwn.net/Articles/394660/} about debian-private
920 @uref{http://subversion.apache.org/mailing-lists.html} private@@
921 @uref{http://www.freebsd.org/administration.html#t-core}
922 @uref{http://foundation.gnome.org/legal/} board members pledge
923 to keep certain matters confidential
925 every security team of every GNU/Linux distribution and OS
928 In fact, Karl Fogel's @qq{Producing Open Source Software}
929 explicitly suggests a private mailing list for some circumstances:
932 [on granting commit/push access to a contributor]
934 But here is one of the rare instances where secrecy is
935 appropriate. You can't have votes about potential committers
936 posted to a public mailing list, because the candidate's feelings
937 (and reputation) could be hurt.
939 @uref{http://producingoss.com/en/consensus-democracy.html#electorate}
942 @subheading Board of governers, voting, etc?
944 Many projects have an official board of directors, or a list of
945 @qq{core developers}, with set term limits and elections and
948 I don't think that we're that big. I think we're still small
949 enough, and there's enough trust and consensus decisions, that we
950 can avoid that. I would rather that we kept on going with
951 trust+consensus for at least the next 2-3 years, and spent more
952 time+energy on bug fixes and new features instead of
953 administrative stuff.
955 Project administrators are Han-Wen, Jan, and Graham.
957 @subsubheading Discussions
960 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00783.html}
961 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg01004.html}
962 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00117.html}
966 @node GOP-PROP 7 - developers as resources
967 @subsubsection GOP-PROP 7 - developers as resources
969 We shall treat developers (and contributors) as
970 @strong{Independent volunteers}: each person does whatever they
971 want, whenever they want. We have busy careers and lives; we make
972 no expectations of action from anybody (with the exception of the
973 6 people in @qq{Meister} positions).
975 @subsubheading Discussions
978 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg01092.html}
979 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00087.html}
980 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00497.html}
984 @node GOP-PROP 8 - issue priorities
985 @subsubsection GOP-PROP 8 - issue priorities
987 We will delete the @qq{priority} field of the issue tracker
988 altogether. The @qq{type} system will be tweaked.
995 a reproducible failure to build either @code{make} or @code{make
996 doc}, from an empty build tree, in a first run, if
997 @code{configure} does not report any errors.
1000 any program behaviour which is @strong{unintentionally} worse than
1001 the previous stable version or the current development version.
1002 Developers may always use the @qq{this is intentional}, or even
1003 the @qq{this is an unavoidable effect of an improvement in another
1004 area}, reason to move this to a different type.
1007 anything which stops contributors from helping out (e.g.
1008 lily-git.tcl not working, source tree(s) not being available,
1009 LilyDev being unable to compile git master, inaccurate
1010 instructions in the Contributor's Guide 2 Quick start).
1012 To limit this scope of this point, we will assume that the
1013 contributor is using the latest LilyDev and has read the relevant
1014 part(s) of the Contributor's Guide. Problems in other chapters of
1015 the CG are not sufficient to qualify as Type-Critical.
1019 @subsubheading More new/changed types and labels
1021 Unless otherwise specified, the current types and labels will
1022 continue to be used. The new types introduced by this proposal
1028 Type-crash: any segfault, regardless of what the input file looks
1029 like or which options are given. Disclaimer: this might not be
1030 possible in some cases, for example certain guile programs (we
1031 certainly can't predict if a piece of scheme will ever stop
1032 running, i.e. the halting problem), or if we rely on other
1033 programs (i.e. ghostscript). If there are any such cases that
1034 make segfault-prevention impossible, we will document those
1035 exceptions (and the issue will remain as a "crash" instead of
1036 "documentation" until the warning has been pushed).
1039 Type-maintainability: anything which makes it difficult for
1040 serious contributors to help out (e.g. difficult to find the
1041 relevant source tree(s), confusing policies, problems with
1042 automatic indentation tools, etc).
1045 Type-ugly: replaces Type-collision, and it will include things
1046 like bad slurs in addition to actual collision.
1050 A new label will be added:
1054 (label) Needs_evidence: it is not clear what the correct output
1055 should look like. We need scans, references, examples, etc.
1059 @subheading Reminding users about stars
1061 We can remind users that they can @qq{star} an issue to indicate
1062 that they care about it. Since we resolved to treat developers as
1063 independent volunteers, there is no expectation that anybody will
1064 look at those stars, but if any developer want to organize their
1065 work schedule according to the stars, they are welcome to do so.
1067 @subsubheading Discussions
1070 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00019.html}
1071 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00277.html}
1072 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00413.html}
1073 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00624.html}
1078 @node GOP-PROP 9 - behavior of make doc
1079 @subsubsection GOP-PROP 9 - behavior of make doc
1081 If there are build problems, then it should be easier to find out
1082 why it's failing. This will be achieved with log files, as well
1083 as possibly including scripts which automatically display portions
1084 of those log files for a failing build.
1086 We will also add targets for building a specific manual (for
1087 quick+easy checking of doc work), as well as for building all
1088 documentation in a specific language (either English or a
1089 translated language).
1091 When you run @code{make doc},
1096 All output will be saved to various log files, with the exception
1097 of output directly from @code{make(1)}.
1099 Note that @code{make(1)} refers to a specific executable file on
1100 unix computers, and is not a general term for the build system.
1103 By default, no other output will be displayed on the console, with
1104 one exception: if a build fails, we might display some portion(s)
1105 of log file(s) which give useful clues about the reason for the
1108 The user may optionally request additional output to be printed;
1109 this is controlled with the @code{VERBOSE=x} flag. In such cases,
1110 all output will still be written to log files; the console output
1111 is strictly additional to the log files.
1114 Logfiles from calling lilypond (as part of lilypond-book) will go in
1116 @file{$LILYPOND_BUILD_DIR/out/lybook-db/12/lily-123456.log} file. All
1117 other logfiles will go in the @file{$LILYPOND_BUILD_DIR/logfiles/}
1120 A single @code{make doc} will therefore result in hundreds of log
1121 files. Log files produced from individual lilypond runs are not
1122 under our control; apart from that, I anticipate having one or two
1123 dozen log files. As long as it is clear which log file is
1124 associated with which operation(s), I think this is entirely
1125 appropriate. The precise implementation will be discussed for
1126 specific patches as they appear.
1129 Both stderr and stdout will be saved in @code{*.log}. The order
1130 of lines from these streams should be preserved.
1133 There will be no additional @qq{progress messages} during the
1134 build process. If you run @code{make --silent}, a non-failing
1135 build should print absolutely nothing to the screen.
1138 Assuming that the loglevels patch is accepted, lilypond (inside
1139 lilypond-book) will be run with --loglevel=WARN.
1140 @uref{http://codereview.appspot.com/4822055/}
1143 Ideally, a failing build should provide hints about the reason why
1144 it failed, or at least hints about which log file(s) to examine.
1148 If this proposal is accepted, none of these policies will be
1149 assumed to apply to any other aspect of the build system.
1150 Policies for any other aspect of the build system will be
1151 discussed in separate proposals.
1153 @subheading Don't cause more build problems
1155 However, there is a danger in this approach, that vital error
1156 messages can also be lost, thus preventing the cause of the
1157 failure of a make being found. We therefore need to be
1158 exceptionally careful to move cautiously, include plenty of tests,
1159 and give time for people to experiment/find problems in each stage
1160 before proceeding to the next stage.
1162 This will be done by starting from individual lilypond calls
1163 within lilypond-book, and slowly moving to @qq{larger} targets of
1164 the build system -- after the individual lilypond calls are are
1165 producing the appropriate amount of output and this is saved in
1166 the right place and we can automatically isolate parts of a
1167 failing build, we will work on lilypond-book in general, and only
1168 then will we look at the build system itself.
1170 @subheading Implementation notes
1172 There is an existing make variable QUIET_BUILD, which
1173 alter the amount of output being displayed
1175 http://lilypond.org/doc/v2.15/Documentation/contributor/useful-make-variables}
1176 ). We are not planning on keeping this make variable.
1178 The standard way for GNU packages to give more output is with a
1179 @code{V=x} option. Presumably this is done by increasing
1180 @code{x}? If we support this option, we should still write log
1181 files; we would simply print more of the info in those log files
1184 The command @code{tee} may be useful to write to a file and
1185 display to stdout (in the case of VERBOSE).
1188 @subsubheading Discussions
1191 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00378.html}
1192 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00703.html}
1197 @n ode GOP-PROP 10 - scheme indentation
1198 @s ubsubsection GOP-PROP 10 - scheme indentation
1200 still under discussion
1202 @subsubheading Discussions
1205 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00625.html}
1206 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg01026.html}
1213 @node Grand LilyPond Input Syntax Standardization (GLISS)
1214 @section Grand LilyPond Input Syntax Standardization (GLISS)
1220 Start: sortly after 2.14 comes out, which is currently estimated
1221 to happen in January 2011.
1224 Length: 6-12 months. We're not going to rush this.
1227 Goal: define an input which we commit to being
1228 machine-updateable for the forseeable future. Any future patches
1229 which change the syntax in a non-convert-ly-able format will be
1230 rejected. (subject to the limitations, below)
1231 Once this is finished, we will release lilypond 3.0.
1236 @subheading The Problem
1238 One of the biggest complaints people have with lilypond -- other
1239 than silly thing like "there's no gui" -- is the changing syntax.
1240 Now, inventing a language or standards is difficult. If you set
1241 it in stone too soon, you risk being stuck with decisions which
1242 may limit matters. If you keep on updating the syntax,
1243 interaction with older data (and other programs!) becomes complex.
1245 @subheading Scope and Limitations
1249 tweaks will not be included. Anything with \override, \set,
1250 \overrideProperty, \tweak, \revert, \unset... including even those
1251 command names themselves... is still fair game for NOT_SMART
1255 other than that, everything is on the table. Is it a problem to
1256 have the tagline inside \header? What should the default behavior
1260 we need to get standards for command names. This will help users
1261 remember them, and reduce the options for future names (and
1262 potential renamings later on). \commandOn and \commandOff seem to
1263 work well (should we *always* have an Off command?), but what
1264 about the "command" part? Should it be \nounVerbOn, or
1265 \verbNounOn ? Or \verbNotesWithExtraInformationOn ?
1268 we need standards for the location of commands. Ligature
1269 brackets, I'm looking at you. (non-postfix notation must die!)
1272 this Grand Project doesn't affect whether we have a 2.16 or not.
1273 The main problem will be deciding what to do (with a bit of
1274 messiness anticipated for \tuplet); we should definitely release a
1275 2.16 before merging _any_ of these changes.
1278 we obviously can't /guarantee/ that we'll /never/ make any
1279 non-convert-ly changes in the basic format. But we *can*
1280 guarantee that such changes would force lilypond 4.0, and that we
1281 would only do so for overwhelmingly good reasons.
1285 @subheading Workflow
1289 We're going to have lots and lots of emails flying around. The
1290 vast majority won't really fit into either -devel or -user, so
1291 we'll use a list devoted to syntax issues.
1294 Once we have a serious proposal that gained general acceptance
1295 from the separate syntax mailing list, I'll bring it to -devel.
1296 We're not going to make any changes without discussing it on
1297 -devel, but if we're going to have huge threads about English
1298 grammar and silly ideas, and I don't want to clutter up -devel.
1299 Once whatever chaotic silliness on the syntax list is settled
1300 down, I'll bring the ideas to -devel.
1303 as with GDP, I'll moderate the discussion. Not as with mailist
1304 moderation, but rather by introducing issues at specific times.
1305 We don't want a free-for-all discussion of all parts of the syntax
1306 at once; nothing will get resolved.
1309 Whenever possible, we'll decide on policies at the highest level
1310 of abstraction. For example, consider \numericTimeSignature,
1311 \slurUp, \xNotesOn, \startTextSpan, and \verylongfermata. One of
1312 them starts with the name of the notation first (slur). One has
1313 an abbreviation (x instead of cross). One has the verb at the end
1314 (On), another has it at the beginning (start). The adjective can
1315 come at the beginning (numeric, x) or end (Up). Most are in
1316 camelCase, but one isn't (verylongfermata).
1319 Instead of arguing about each individual command, we'll decide on
1320 abstract questions. Should each command begin the notation-noun,
1321 or the verb? Should all commands be in camelCase, or should we
1322 make everything other than articulations in camelCase but make
1323 articulations all lower-case? Are abbreviations allowed?
1326 Once we've answered such fundamental questions, most of the syntax
1327 should fall into place pretty easily. There might be a few odd
1328 questions left ("is it a span, or a spanner?"), but those can be
1329 settled fairly quickly.
1333 @subheading Implementation
1335 Nothing until the project is finished, then we declare the next
1336 stable release (2.16.0 or 2.18.0 ?) to be the final 2.x version,
1337 release it, then apply all the GLISS syntax changes and start
1338 testing a beta for 3.0 a week or two later.
1340 @subheading Discussion
1342 Don't respond to any of the specifics yet. Yes, we all have our
1343 pet irritations (like "what's up with \paper and \layout?!").
1344 There will be plenty of time to discuss them once GLISS starts.
1346 That said, we have a list of specific items that people really
1347 wanted to have written down. See @ref{Specific GLISS issues}.
1350 * Specific GLISS issues::
1354 @node Specific GLISS issues
1355 @subsection Specific GLISS issues
1359 add regtests for every piece of syntax (not one-command-per-file,
1360 but making a few files which, between them, use every single piece
1361 of syntax.) This is a great test for convert-ly.
1364 should GLISS cover suggested conventions? (indentation,
1365 one-bar-per-line, etc -- the kind of stuff we list for the
1366 lilypond formatting in the docs ?)
1369 how much (if any) syntactic sugar should we add? i.e.
1371 \instrumentName #'foo
1373 \set Staff.instrumentName
1375 ? Carl: maybe yes, Neil: no. (for example, it fails for
1379 the values that are used as arguments to common used overrides.
1380 Sometimes they are a symbol (e.g. #'around), sometimes a
1381 predefined variable referring to a Scheme value or object (e.g.
1382 #LEFT, #all-visible ). The main trouble is that for novice users
1383 it is not clear when there should be an apostrophe and when not.
1386 When do we need -\command and when is it just \command ?
1390 Command-line options to the lilypond binary. -dfoo counts as a
1391 tweak; we won't be trying to pin those down.
1404 If would be pedagogically simpler to realize this difference if
1405 the syntax was separate if you define a context from scratch (as
1406 is the case with \RemoveEmptyStaffContext) or if it's defined by
1407 adding onto an existing context. For example, a syntax like
1411 % Copy the current settings of the Staff context:
1413 % do whatever additional settings
1415 %%% could be used to distinguish from
1417 % Take settings from a variable:
1419 % do whatever additional settings
1425 % Start from scratch:
1434 Capitalization of identifiers: \VoiceOne ?
1439 { music expression } * 4
1441 \repeat unfold 4 { music expression }
1446 @uref{http://lists.gnu.org/archive/html/lilypond-devel/2010-04/msg00467.html}
1450 Personally, I find it easier to understand when there's a repeated
1451 8 in the half-bar position; it's much easier to see that you have
1456 %%% instead of one group of eight:
1461 trivially simple bar-lines:
1465 encourage, allow, or discourage, or disallow?
1468 indentation of \\ inside a @{@} construct.
1472 barline checks at the end of line should be preceded by at least 2
1473 spaces? barline checks should line up if possible (i.e. if you
1474 can use less than 4, 8, X empty spaces before a barline check to
1478 Why doesn't \transpose respect \relative mode?
1482 on \score vs. \new Score
1484 But in the light of a consistent syntax and semantic, I see no
1485 reason (from the users POV) to disallow it. After all, the real
1486 top-level context is a \book @{@}, isn't it, and I don't see a point
1487 in disallowing a \new Score construct just like \new Staff.
1489 From a syntactical POV, I see the following pros for \new Score:
1490 - You can write \with @{ ... @} for every other context but \Score,
1491 which (for consistency) should also work with \new Score.
1492 - When there's a \new Foo Bar, there's also a \context Foo Bar,
1493 which makes the same as a parallel instantiation of all Bar's.
1494 - [Quoting Rune from
1495 @uref{http://www.mail-archive.com/lilypond-devel@@gnu.org/msg14713.html}
1496 "I know that the \score-statement is a syntactical construct,
1497 but I think it would be nice to hide this fact from the users. I
1498 think we could make the use of score-block much more intuitive if
1499 changing the syntax to \new \Score and adding an implicit
1500 sequential-statement to the score."
1505 @uref{http://code.google.com/p/lilypond/issues/detail?id=1322}
1506 about \new vs. \context.
1510 Let users add their own items to the parser? comment 11 on:
1511 @uref{http://code.google.com/p/lilypond/issues/detail?id=1322}
1514 should engravers be pluralized (note_heads_engraver) or not
1515 (note_head_engraver) ?
1518 should we allow numbers in identifier names? Issue:
1519 @uref{http://code.google.com/p/lilypond/issues/detail?id=1670}
1522 should we officially allow accented characters? in general, how
1523 do we feel about utf-8 stuff?
1526 for the sake of completeness/simplicity, what about *disallowing*
1527 the "one-note" form of a music expression? i.e. only allowing
1530 \transpose c d { e1 }
1531 \transpose c d << e1 >>
1540 What should be the officially encouraged way of writing music for
1541 transposing instruments? Maybe it should be simplified?
1542 See http://lists.gnu.org/archive/html/lilypond-user/2011-07/msg00130.html
1547 @node Unsorted policies
1548 @section Unsorted policies
1550 @subsubheading Language-specific mailing lists
1552 A translator can ask for an official lilypond-xy mailing list once
1553 they've finished all @qq{priority 1} translation items.
1555 @subsubheading Performing yearly copyright update (@qq{grand-replace})
1557 At the start of each year, copyright notices for all source files
1558 should be refreshed by running the following command from the top of
1565 Internally, this invokes the script @file{scripts/build/grand-replace.py},
1566 which performs a regular expression substitution for old-year -> new-year
1567 wherever it finds a valid copyright notice.
1569 Note that snapshots of third party files such as @file{texinfo.tex} should
1570 not be included in the automatic update; @file{grand-replace.py} ignores these
1571 files if they are listed in the variable @code{copied_files}.
1574 @subsubheading Push git access
1576 Git access is given out when a contributor has a significant
1577 record of patches being accepted without problems. If existing
1578 developers are tired of pushing patches for a contributor, we'll
1579 discuss giving them push access. Unsolicited requests from
1580 contributors for access will almost always be turned down.