1 @c -*- coding: utf-8; mode: texinfo; -*-
2 @node Administrative policies
3 @chapter Administrative policies
5 This chapter discusses miscellaneous administrative issues which
6 don't fit anywhere else.
9 * Meta-policy for this document::
12 * Administrative mailing list::
13 * Grand Organization Project (GOP)::
14 * Grand LilyPond Input Syntax Standardization (GLISS)::
18 @node Meta-policy for this document
19 @section Meta-policy for this document
21 The Contributor's Guide as a whole is still a work in progress,
22 but some chapters are much more complete than others. Chapters
23 which are @qq{almost finished} should not have major changes
24 without a discussion on @w{@code{-devel}}; in other chapters, a
25 disorganized @qq{wiki-style dump} of information is encouraged.
27 Do not change (other than spelling mistakes) without discussion:
32 @ref{Introduction to contributing}
35 @ref{Working with source code}
39 Please dump info in an appropriate @@section within these manuals,
40 but discuss any large-scale reorganization:
48 @ref{Documentation work}
54 @ref{Regression tests}
57 @ref{Programming work}
62 Totally disorganized; do whatever the mao you want:
76 @ref{Administrative policies}
85 We have four jobs for organizing a team of contributors:
90 Bug Meister: trains new Bug Squad volunteers, organizes who works
91 on which part of their job, checks to make sure that everything is
92 running smoothly, and has final say on our policy for Issues.
97 Doc Meister: trains new doc editors/writers, organizes who works
98 on which part of the job, checks to make sure that everything is
99 running smoothly, and has final say on our policy for
100 Documentation. Also includes LSR work.
105 Translation Meister: trains new translators, updates the
106 translation priority list, and handles merging branches (in both
112 Frog Meister: is responsible for code patches from (relatively)
113 inexperienced contributors. Keeps track of patches, does initial
114 reviewing of those patches, sends them to @w{@code{-devel}} when
115 they've had some initial review on the Frog list, pesters the
116 @w{@code{-devel}} community into actually reviewing said patches, and
117 finally pushes the patches once they're accepted. This person is
118 @emph{not} responsible for training new programmers, because that
119 would be far too much work -- he job is @qq{only} to guide
120 completed patches through our process.
129 @subheading Introduction
131 Patchy is a set of Python scripts written by Graham to automate
132 two administrative tasks:
136 checking that patches compile
139 trying to ensure that git master is not corrupted by non-compiling
144 Checking that patches compile is the job of @code{test-patches.py},
145 whilst @code{lilypond-patchy-staging.py} tests patches in staging
146 before pushing them to master.
148 @subheading Installing patchy
150 To install patchy, you should run do the following:
154 Create a new user on your box - use this solely to run patchy. It
155 is recommended that this should not be an administrator. New
156 users are created from System; Administration; Users and Groups.
159 Get the patchy scripts from
161 https://github.com/gperciva/lilypond-extra/
163 Patchy is in the @code{patches} directory.
166 Put the scripts in a sensible place on your system
169 Create a new git repository with
171 git clone git://git.sv.gnu.org/lilypond.git
173 This will create a directory called lilypond with the repo in it.
174 Make sure it's where you want it and name it lilypond-git
175 (assuming you want to follow the standard naming conventions).
178 Create an environment variable called LILYPOND_GIT and make it
179 equal to the location of your new git repo. You can do this by
180 editing .profile and adding the line:
182 export LILYPOND_GIT=~/lilypond-git
184 and logging out and in.
187 Edit lilypond-patchy-config-DEFAULT to provide working directories
188 for your build directory, your results directory, compiler options
189 and notification method. If you don't want to use email
190 notification, then delete everything after @code{smtp_command:}.
191 Save this as lilypond-patchy-config.
194 Ensure that your new user has git push access. Follow the
195 instructions in the CG at @ref{Commit access}. Do not set
196 password protection for the key - if you do you will not be able
197 to run patchy unattended.
201 @subheading lilypond-patchy-staging.py
203 lilypond-patchy-staging.py is run with
205 python lilypond-patchy-staging.py
207 Not much appears to happen except you can see a lot of CPU gets
208 used if you open System Monitor. There's not much point running
209 patch-merge unless there something in staging to be merged to
212 The script fetches the current patches in staging and runs
213 @code{make}, @code{make test} and @code{make doc} to ensure that all of
214 these complete error-free. If you have set patchy up to use email,
215 it emails its results to you. If you haven't, then you can view
216 them in a logfile. It also merges staging into master.
218 @subheading test-patches.py
219 test-patches prepares a regtest comparison for a human to quickly
220 glance at, to determine if the patch is ready for a review. If
221 the patch passes test-patches, it should be reviewed by relevant
222 developers, and if it passes this, it can be considered for
223 countdown (see @ref{Commits and patches}) and pushing to staging
224 (see @ref{Pushing to staging}).
227 @node Administrative mailing list
228 @section Administrative mailing list
230 An mailing list for administrative issues is maintained at
231 @code{lilypond-hackers@@gnu.org}.
233 This list is intended to be used for discussions that should be kept
234 private. Therefore, the archives are closed to the public.
236 Subscription to this list is limited to certain senior developers.
238 At the present time, the list is dormant.
240 Details about the criteria for membership, the types of discussion
241 to take place on the list, and other policies for the hackers list
242 will be finalized during the
243 @ref{Grand Organization Project (GOP)}.
247 @node Grand Organization Project (GOP)
248 @section Grand Organization Project (GOP)
254 Clarify the various development tasks by writing down the polices
255 and techniques and/or simplifying the tasks directly.
258 Get more people involved in development: specifically, find people
259 to do easy tasks to allow advanced developers to concentrate on
268 * Policy decisions (finished)::
272 @subsection Motivation
274 Most readers are probably familiar with the LilyPond Grand
275 Documentation Project, which ran from Aug 2007 to Aug 2008. This
276 project involved over 20 people and resulted in an almost complete
277 rewrite of the documentation. Most of those contributors were
278 normal users who decided to volunteer their time and effort to
279 improve lilypond for everybody. By any measure, it was a great
282 The Grand Organization Project aims to do the same thing with a
283 larger scope -- instead of focusing purely on documentation, the
284 project aims to improve all parts of LilyPond and its community.
285 Just as with GDP, the main goal is to encourage and train users to
286 become more involved.
288 If you have never contributed to an open-source project before --
289 especially if you use Windows or OSX and do not know how to
290 program or compile programs -- you may be wondering if there's
291 anything you can do. Rest assured that you @emph{can} help.
293 @subheading "Trickle-up" development
295 One of the reasons I'm organizing GOP is "trickle-up"
296 development. The idea is this: doing easy tasks frees up advanced
297 developers to do harder tasks. Don't ask "am I the @emph{best}
298 person for this job"; instead, ask "am I @emph{capable} of doing
299 this job, so that the current person can do stuff I @emph{can't}
302 For example, consider lilypond's poor handling of grace notes in
303 conjunction with clef and tempo changes. Fixing this will require
304 a fair amount of code rewriting, and would take an advanced
305 developer a few weeks to do. It's clearly beyond the scope of a
306 normal user, so we might as well sit back and do nothing, right?
308 No; we @emph{can} help, indirectly. Suppose that our normal user
309 starts answering more emails on lilypond-user. This in turn means
310 that documentation writers don't need to answer those emails, so
311 they can spend more time improving the docs. I've noticed that all
312 doc writers tackle harder and harder subjects, and when they start
313 writing docs on scheme programming and advanced tweaks, they start
314 contributing bug fixes to lilypond. Having people performing these
315 easy-to-moderate bug fixes frees up the advanced developers to
316 work on the really hard stuff... like rewriting the grace note
319 Having 1 more normal user answering emails on lilypond-user won't
320 have a dramatic trick-up affect all by himself, of course. But if
321 we had 8 users volunteering to answer emails, 6 users starting to
322 write documentation, and 2 users editing LSR... well, that would
323 free up a lot of current bug-fixing-capable contributors to focus
324 on that, and we could start to make a real dent in the number of
325 bugs in lilypond. Quite apart from the eased workload, having that
326 many new helpers will provide a great moral boost!
329 @subsection Ongoing jobs
331 Although GOP is a short-term project, the main goal is to train
332 more people to handle ongoing jobs. The more people doing these
333 jobs, the ligher the work will be, and the more we can get done
336 Also, it would be nice if we had at least one "replacement" /
337 "understudy" for each role -- too many tasks are only being done
338 by one person, so if that person goes on vacation or gets very
339 busy with other matters, work in that area grinds to a halt.
341 @subheading Jobs for normal users
345 LilyPond is sometimes critized for not listening to users, but
346 whenever we ask for opinions about specific issues, we never get
347 enough feedback. This is somewhat aggravating.
348 We need a group of users to make a dedicated effort to test and
349 give feedback. If there's new documentation, read it. If there's
350 an experimental binary, download it and try compiling a score with
351 it. If we're trying to name a new command, think about it and give
354 @item lilypond-user support:
355 I think it would be nice if we had an official team of users
358 @item LilyPond Report:
359 Keeping a monthly newsletter running is a non-trivial task. A lot
360 of work is needed to organize it; it would be great if we could
361 split up the work. One person could write the Snippet of the
362 Month, another person could do Quotes of the Month, another person
363 could do interviews, etc.
366 Although GDP (the Grand Documentation Project) did great work,
367 there's still many tasks remaining.
370 Keeping the documentation translations is a monumental task; we
371 need all the help we can get!
375 @subheading Jobs for advanced users for developers
378 @item Git help for writers:
379 We often receive reports of typos and minor text updates to the
380 documentation. It would be great if somebody could create
381 properly-formatted patches for these corrections.
383 Technical requirements: ability to run @ref{Lilydev}.
386 LSR contains many useful examples of lilypond, but some snippets
387 are out of date and need updating. Other snippets need to be
388 advertized, and new snippets need to be sorted. We could use
389 another person to handle LSR.
391 Technical requirements: use of a web browser. LilyPond
392 requirements: you should be familiar with most of Notation
393 chapters 1 and 2 (or be willing to read the docs to find out).
395 @item Join the Frogs:
396 "Frogs" are a team of bug-fixers (because frogs eat bugs, and you
397 often find them in Ponds of Lilies) and new feature implementors.
399 Technical requirements: development environment (such as
400 @ref{Lilydev}), ability to read+write scheme and/or C++ code.
405 @node Policy decisions
406 @subsection Policy decisions
408 There are a number of policy decisions -- some of them fairly
409 important -- which we have been postponing for a few years. We
410 are now discussing them slowly and thoroughly; agenda and exact
411 proposals are online:
414 @uref{http://lilypond.org/~graham/gop/index.html}
417 Below is a list of policies which are not @qq{on the agenda} yet.
419 Note that the presence of an item on this list does @emph{not}
420 mean that everybody thinks that something needs to be done.
421 Inclusion in this simply means that one developer thinks that we
422 should discuss it. We are not going to filter this list; if any
423 developer thinks we should discuss something, just add it to the
424 bottom of the list. (the list is unsorted)
426 As GOP progresses, items from this list will be put on the agenda
427 and removed from this list. I generally try to have one month's
428 discussion planned in advance, but I may shuffle things around to
429 respond to any immediate problems in the developer community.
431 There are some item(s) not displayed here; these are questions
432 that were posed to me privately, and I do not feel justified in
433 discussing them publicly without the consent of the person(s) that
434 brought them up. They will initially be discussed privately on the
435 lilypond-hackers mailing list -- but the first question will be
436 "do we absolutely need to do this privately", and if not, the
437 discussion will take place on lilypond-devel like the other items.
439 In most policy discussions in lilypond over the past few years,
440 the first half (or more) is wasted arguing on the basis of
441 incorrect or incomplete data; once all the relevant facts are
442 brought to light, the argument is generally resolved fairly
443 quickly. In order to keep the GOP discussions focused, each topic
444 will be introduced with a collection of relevant facts and/or
445 proposals. It is, of course, impossible to predict exactly which
446 facts will be relevant to the discussion -- but spending an hour
447 or two collecting information could still save hours of
450 @warning{The estimated time required for "prep work", and the
451 following discussion, has been added to each item. At the moment,
452 there is an estimated 30 hours of prep work and 140 hours of
456 @item @strong{Patch reviewing}:
457 At the time of this writing, we have 23 (known) patches waiting
458 for review. Some from main developers; some from new developers.
459 We desperately need more people helping with lilypond, but
460 ignoring patches is the best way to drive potential contributors
461 away. This is not good.
463 (prep: 2 hours. discuss: 10 hours)
465 @item @strong{Official links to other organizations?}:
466 There's something called the "software freedom conservancy", and
467 in general, there's a bunch of "umbrella organizations". Joining
468 some of these might give us more visibility, possibly leading to
469 more users, more developers, maybe even financial grants or use in
472 (prep: 2 hours. discuss: 5 hours)
474 @item @strong{Issue tracking with google code}:
475 We use the google issue tracker, but this means that we are
476 relying on a commercial entity for a large part of our
477 development. Would it be better (safer in the long run) to use the
478 savannah bug tracker?
480 (prep: 1 hour. discuss: 5 hours)
482 @item @strong{Patch review tool}:
483 Reitveld is inconvenient in some respects: it requires a google
484 account, and there's no way to see all patches relating to
485 lilypond. Should we switch to something like gerritt?
486 @uref{http://code.google.com/p/lilypond/issues/detail?id=1184}
488 (prep: 5 hours. discuss: 15 hours)
490 @item @strong{Clarity for sponsorships}:
491 We currently do not advertize bounties and sponsorships on the
492 webpage. How much advertising do we want, and what type?
493 Should we change the "structure" / "framework" for bounties?
495 (prep: 2 hours. discuss: 10 hours)
497 @item @strong{code readability}:
498 "Our aim when producing source code for Lilypond in whatever
499 language is that it should be totally comprehensible to a
500 relatively inexperienced developer at the second reading."
503 - aids maintainability of code base
504 - "second reading" so newer developers can look up unfamiliar
506 - will help to keep things simple, even if the code is doing
507 complex stuff discourages "secret squirrel" coding, e.g. "how
508 much functionality can I squeeze into as few characters as
509 possible" "comments are for wimps"
510 - will aid not *discouraging* new developers to join the project
512 (prep: 2 hours. discuss: 10 hours)
514 @item @strong{C++ vs. scheme}:
515 what should be in scheme, what should be in C++, what can/should
516 be ported from one to the other, etc. Questions of
517 maintainability, speed (especially considering guile 2.0), and the
518 amount of current material in either form, are important.
520 (prep: 5 hours. discuss: 15 hours)
522 @item @strong{always make an issue number for patches}:
523 there is a proposal that we should always have a google code issue
524 number for every patch. This proposal is closely tied to our
525 choice of patch review tool; if we switch to a different tool (as
526 suggested in a different proposal), this proposal may become moot.
528 (prep: 1 hour. discuss: 5 hours)
530 @item @strong{initalizer lists}:
531 shoudl we use initalizer lists for C++? AFAIK they make no
532 difference for built-in types, but there's some weird case where
533 it's more efficient for objects, or something.
535 Probably not worth making this a weekly thing on its own, but we
536 can probably wrap it up with some other code-related questions.
538 (prep: 15 minutes. discuss: 3 hours)
542 @node Policy decisions (finished)
543 @subsection Policy decisions (finished)
545 Here is a record the final decisions, along with links to the
549 * GOP-PROP 1 - python formatting::
550 * GOP-PROP 2 - mentors and frogs::
551 * GOP-PROP 3 - C++ formatting::
552 * GOP-PROP 4 - lessons from 2.14::
553 * GOP-PROP 5 - build system output (not accepted)::
554 * GOP-PROP 6 - private mailing lists::
555 * GOP-PROP 7 - developers as resources::
556 * GOP-PROP 8 - issue priorities::
557 * GOP-PROP 9 - behavior of make doc::
560 @node GOP-PROP 1 - python formatting
561 @subsubsection GOP-PROP 1 - python formatting
563 We will follow the indentation described in PEP-8.
564 @uref{http://www.python.org/dev/peps/pep-0008/}
568 use 4 spaces per indentation level
571 never mix tabs and spaces (for indentation)
574 Code indented with a mixture of tabs and spaces should be
575 converted to using spaces exclusively
577 Once this is done, we should add @code{python -tt} to the build
578 system to avoid such errors in the future.
582 There should be absolutely no tab characters for indentation in
583 any @code{.py} file in lilypond git. All such files should be
584 converted to use spaces only.
586 @subsubheading Discussions
589 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00060.html}
590 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00084.html}
591 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00310.html}
592 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00574.html}
596 @node GOP-PROP 2 - mentors and frogs
597 @subsubsection GOP-PROP 2 - mentors and frogs
599 Nothing much was decided. The list of responsibilities was
600 slightly altered; see the new one in @ref{Mentors}. We should
601 encourage more use of the Frogs mailing list. There's a list of
602 contributor-mentor pairs in:
605 @uref{https://github.com/gperciva/lilypond-extra/blob/master/people/mentors.txt}
608 That's pretty much it.
610 @subsubheading Discussions
613 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00311.html}
619 @node GOP-PROP 3 - C++ formatting
620 @subsubsection GOP-PROP 3 - C++ formatting
622 Speaking academically, C++ code style is a "solved problem". Let's
623 pick one of the existing solutions, and let a computer deal with
624 this. Humans should not waste their time, energy, and creativity
625 manually adding tabs or spaces to source code.
627 We have modified @code{fixcc.py} to use astyle, along with extra
632 the final script will be run @strong{blindly} on the lilypond
633 source code. We will accept whatever formatting the final version
634 of this script produces, with no manual tweaking.
637 patches which have been run through this tool will not be rejected
638 for style reasons. Any code formatting @qq{desires} which are not
639 enforced by @code{fixcc.py} will not be considered grounds for
643 for now, this style will not be enforced. It is not cause for
644 concern if patches which do not follow the formatting done by
645 @code{fixcc.py} are pushed. From time to time, Graham will run
646 the formatter on the entire code base, and commit the resulting
649 In a few months, we will tighten up this policy item (with some
650 sort of automatic processing), but that is outside the scope of
651 this policy item and is a matter for later discussion.
654 after the proposal is accepted, we will leave some time for
655 existing patches to be accepted and pushed. The script was
656 run on the source code on @strong{2011 August 01}.
662 LilyPond is a GNU project, so it makes sense to follow the GNU
663 coding standards. These standards state:
666 We don’t think of these recommendations as requirements, because
667 it causes no problems for users if two different programs have
668 different formatting styles.
670 But whatever style you use, please use it consistently, since a
671 mixture of styles within one program tends to look ugly. If you
672 are contributing changes to an existing program, please follow the
673 style of that program.
676 (@uref{http://www.gnu.org/prep/standards/html_node/Formatting.html})
678 With that in mind, we do not think that we must blindly follow the
679 formatting given by the currrent version of Emacs.
681 @subheading Implementation notes
683 We can avoid some of the style change pollution in git history by
684 ignoring whitespaces changes:
690 @subsubheading Discussions
693 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00526.html}
694 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00796.html}
695 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00200.html}
696 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00525.html}
697 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00751.html}
698 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00751.html}
702 @node GOP-PROP 4 - lessons from 2.14
703 @subsubsection GOP-PROP 4 - lessons from 2.14
707 A brief history of releases:
709 @multitable @columnfractions .2 .2 .3
710 @headitem date (YYYY-MM-DD) @tab version @tab comment
711 @item 2008-10-28 @tab 2.11.63 @tab nobody checking regtests
712 @item 2008-11-17 @tab 2.11.64
713 @item 2008-11-29 @tab 2.11.65
714 @item 2008-12-23 @tab 2.12.0
715 @item 2009-01-01 @tab @tab somewhere around here, Graham becomes
716 officially release manager, but Han-Wen still builds the actual
718 @item 2009-01-01 @tab 2.12.1
719 @item 2009-01-25 @tab 2.12.2
720 @item 2009-02-28 @tab 2.13.0
721 @item 2009-06-01 @tab 2.13.1 @tab note jump in time!
722 @item 2009-06-27 @tab 2.13.2 @tab first Graham release?
723 @item 2009-07-03 @tab 2.13.3
724 @item 2009-09-09 @tab @tab Graham arrives in Glasgow, gets a
725 powerful desktop computer, and begins serious work on GUB (sending
726 bug reports to Jan). It takes approximately 100 hours until GUB
727 is stable enough to make regular releases.
728 @item 2009-09-24 @tab 2.13.4
729 @item 2009-10-02 @tab 2.13.5
730 @item 2009-10-22 @tab 2.13.6
731 @item 2009-11-05 @tab 2.13.7
733 @item 2010-01-13 @tab 2.12.3
735 @item 2010-03-19 @tab 2.13.16 @tab Bug squad starts doing a few
736 regtest comparisons, but IIRC the effort dies out after a few
739 @item 2010-08-04 @tab 2.13.29 @tab Phil starts checking regtests (BLUE)
741 @item 2011-01-12 @tab 2.13.46 @tab release candidate 1 (GREEN)
743 @item 2011-05-30 @tab 2.13.63 @tab release candidate 7 (GREEN)
744 @item 2011-06-06 @tab 2.14.0
747 @c A graphical display of bugs:
749 @c @image{bugs-2.13-visualization,png}
750 @c @image{zoom-2.13-visualization,png}
752 @subheading Carl's analysis of the bugs
754 A @file{csv} spreadsheet is available.
757 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00852.html}
761 @uref{lilypond-issues-analysis.csv}
762 @uref{lilypond-issues-analysis-trim-duplicates.csv}
765 There 148 issues marked with Priority=Critical in the tracker.
767 I've done an analysis, and it looks to me like there was initially
768 a backlog of critical issues that weren't fixed, and little work
769 was being done to eliminate critical issues.
771 Somewhere about 2010-08-01, critical issues started to disappear,
772 but occasional new ones appeared.
774 There were a couple of major changes that introduced unanticipated
775 regressions (new spacing code, beam collision avoidance). These
776 produced more than the expected number of regressions.
778 It appears to me that we didn't really get serious about
779 eliminating critical bugs until about 2010-06-15 or so. After
780 that point, the number of critical bugs more-or-less steadily
781 decreased until we got to a release candidate.
783 Of particular interest, the first release candidate of 2.14 was
784 released on 2011-01-12. Over the next 10 days, about a dozen bugs
785 were reported and fixed. Release candidate 2 came out on
786 2011-02-09. No surge of bugs occurred with this release.
787 Candidate 3 came out on 2011-03-13; we got 2 bugs per week.
788 Candidate 4 came out on 2011-03-29; 2 new bugs. Candidate 6 came
789 out on 2011-04-07. We got a couple of bugs per week.
791 @subheading Notes, commentary, and opinions
794 Han-Wen: Overall, I think this cycle took too long
799 @subsubheading Discussions
802 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00797.html}
803 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00364.html}
808 @node GOP-PROP 5 - build system output (not accepted)
809 @subsubsection GOP-PROP 5 - build system output (not accepted)
811 This proposal was too broad; after a month of discussion, Graham
812 withdrew the proposal. Portions of it will be introduced in later
815 @subsubheading Discussions
818 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00320.html}
819 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00527.html}
820 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00753.html}
821 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg01042.html}
822 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00116.html}
823 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00310.html}
827 @node GOP-PROP 6 - private mailing lists
828 @subsubsection GOP-PROP 6 - private mailing list
830 Potentially sensitive or private matters will be referred to
831 Graham. He will then decide who should discuss the matter on an
832 ad-hoc basis, and forward or CC them on future emails.
834 For emphasis, the project administrators are Han-Wen, Jan, and
835 Graham; those three will always be CC'd on any important
838 The lilypond-hackers mailing list will be removed.
842 There is some unhappy history about this idea in our development
846 @uref{http://lists.gnu.org/archive/html/lilypond-devel/2010-09/msg00178.html}
847 @uref{http://news.lilynet.net/spip.php?article121}
848 @uref{http://lists.gnu.org/archive/html/lilypond-devel/2010-11/msg00076.html}
851 @subheading Other projects
853 The idea of private mailing lists is hardly uncommon in
854 open-source software. For example,
857 @uref{http://lwn.net/Articles/394660/} about debian-private
858 @uref{http://subversion.apache.org/mailing-lists.html} private@@
859 @uref{http://www.freebsd.org/administration.html#t-core}
860 @uref{http://foundation.gnome.org/legal/} board members pledge
861 to keep certain matters confidential
863 every security team of every linux distribution and OS
866 In fact, Karl Fogel's @qq{Producing Open Source Software}
867 explicitly suggests a private mailing list for some circumstances:
870 [on granting commit/push access to a contributor]
872 But here is one of the rare instances where secrecy is
873 appropriate. You can't have votes about potential committers
874 posted to a public mailing list, because the candidate's feelings
875 (and reputation) could be hurt.
877 @uref{http://producingoss.com/en/consensus-democracy.html#electorate}
880 @subheading Board of governers, voting, etc?
882 Many projects have an official board of directors, or a list of
883 @qq{core developers}, with set term limits and elections and
886 I don't think that we're that big. I think we're still small
887 enough, and there's enough trust and consensus decisions, that we
888 can avoid that. I would rather that we kept on going with
889 trust+consensus for at least the next 2-3 years, and spent more
890 time+energy on bug fixes and new features instead of
891 administrative stuff.
893 Project administrators are Han-Wen, Jan, and Graham.
895 @subsubheading Discussions
898 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00783.html}
899 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg01004.html}
900 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00117.html}
904 @node GOP-PROP 7 - developers as resources
905 @subsubsection GOP-PROP 7 - developers as resources
907 We shall treat developers (and contributors) as
908 @strong{Independent volunteers}: each person does whatever they
909 want, whenever they want. We have busy careers and lives; we make
910 no expectations of action from anybody (with the exception of the
911 6 people in @qq{Meister} positions).
913 @subsubheading Discussions
916 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg01092.html}
917 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00087.html}
918 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00497.html}
922 @node GOP-PROP 8 - issue priorities
923 @subsubsection GOP-PROP 8 - issue priorities
925 We will delete the @qq{priority} field of the issue tracker
926 altogether. The @qq{type} system will be tweaked.
933 a reproducible failure to build either @code{make} or @code{make
934 doc}, from an empty build tree, in a first run, if
935 @code{configure} does not report any errors.
938 any program behaviour which is @strong{unintentionally} worse than
939 the previous stable version or the current development version.
940 Developers may always use the @qq{this is intentional}, or even
941 the @qq{this is an unavoidable effect of an improvement in another
942 area}, reason to move this to a different type.
945 anything which stops contributors from helping out (e.g.
946 lily-git.tcl not working, source tree(s) not being available,
947 lilydev being unable to compile git master, inaccurate
948 instructions in the Contributor's Guide 2 Quick start).
950 To limit this scope of this point, we will assume that the
951 contributor is using the latest lilydev and has read the relevant
952 part(s) of the Contributor's Guide. Problems in other chapters of
953 the CG are not sufficient to qualify as Type-Critical.
957 @subsubheading More new/changed types and labels
959 Unless otherwise specified, the current types and labels will
960 continue to be used. The new types introduced by this proposal
966 Type-crash: any segfault, regardless of what the input file looks
967 like or which options are given. Disclaimer: this might not be
968 possible in some cases, for example certain guile programs (we
969 certainly can't predict if a piece of scheme will ever stop
970 running, i.e. the halting problem), or if we rely on other
971 programs (i.e. ghostscript). If there are any such cases that
972 make segfault-prevention impossible, we will document those
973 exceptions (and the issue will remain as a "crash" instead of
974 "documentation" until the warning has been pushed).
977 Type-maintainability: anything which makes it difficult for
978 serious contributors to help out (e.g. difficult to find the
979 relevant source tree(s), confusing policies, problems with
980 automatic indentation tools, etc).
983 Type-ugly: replaces Type-collision, and it will include things
984 like bad slurs in addition to actual collision.
988 A new label will be added:
992 (label) Needs_evidence: it is not clear what the correct output
993 should look like. We need scans, references, examples, etc.
997 @subheading Reminding users about stars
999 We can remind users that they can @qq{star} an issue to indicate
1000 that they care about it. Since we resolved to treat developers as
1001 independent volunteers, there is no expectation that anybody will
1002 look at those stars, but if any developer want to organize their
1003 work schedule according to the stars, they are welcome to do so.
1005 @subsubheading Discussions
1008 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00019.html}
1009 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00277.html}
1010 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00413.html}
1011 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00624.html}
1016 @node GOP-PROP 9 - behavior of make doc
1017 @subsubsection GOP-PROP 9 - behavior of make doc
1019 If there are build problems, then it should be easier to find out
1020 why it's failing. This will be achieved with log files, as well
1021 as possibly including scripts which automatically display portions
1022 of those log files for a failing build.
1024 We will also add targets for building a specific manual (for
1025 quick+easy checking of doc work), as well as for building all
1026 documentation in a specific language (either English or a
1027 translated language).
1029 When you run @code{make doc},
1034 All output will be saved to various log files, with the exception
1035 of output directly from @code{make(1)}.
1037 Note that @code{make(1)} refers to a specific executable file on
1038 unix computers, and is not a general term for the build system.
1041 By default, no other output will be displayed on the console, with
1042 one exception: if a build fails, we might display some portion(s)
1043 of log file(s) which give useful clues about the reason for the
1046 The user may optionally request additional output to be printed;
1047 this is controlled with the @code{VERBOSE=x} flag. In such cases,
1048 all output will still be written to log files; the console output
1049 is strictly additional to the log files.
1052 Logfiles from calling lilypond (as part of lilypond-book) will go
1053 in the relevant @file{build/out/lybook-db/12/lily-123456.log}
1054 file. All other logfiles will go in the @file{build/logfiles/}
1057 A single @code{make doc} will therefore result in hundreds of log
1058 files. Log files produced from individual lilypond runs are not
1059 under our control; apart from that, I anticipate having one or two
1060 dozen log files. As long as it is clear which log file is
1061 associated with which operation(s), I think this is entirely
1062 appropriate. The precise implementation will be discussed for
1063 specific patches as they appear.
1066 Both stderr and stdout will be saved in @code{*.log}. The order
1067 of lines from these streams should be preserved.
1070 There will be no additional @qq{progress messages} during the
1071 build process. If you run @code{make --silent}, a non-failing
1072 build should print absolutely nothing to the screen.
1075 Assuming that the loglevels patch is accepted, lilypond (inside
1076 lilypond-book) will be run with --loglevel=WARN.
1077 @uref{http://codereview.appspot.com/4822055/}
1080 Ideally, a failing build should provide hints about the reason why
1081 it failed, or at least hints about which log file(s) to examine.
1085 If this proposal is accepted, none of these policies will be
1086 assumed to apply to any other aspect of the build system.
1087 Policies for any other aspect of the build system will be
1088 discussed in separate proposals.
1090 @subheading Don't cause more build problems
1092 However, there is a danger in this approach, that vital error
1093 messages can also be lost, thus preventing the cause of the
1094 failure of a make being found. We therefore need to be
1095 exceptionally careful to move cautiously, include plenty of tests,
1096 and give time for people to experiment/find problems in each stage
1097 before proceeding to the next stage.
1099 This will be done by starting from individual lilypond calls
1100 within lilypond-book, and slowly moving to @qq{larger} targets of
1101 the build system -- after the individual lilypond calls are are
1102 producing the appropriate amount of output and this is saved in
1103 the right place and we can automatically isolate parts of a
1104 failing build, we will work on lilypond-book in general, and only
1105 then will we look at the build system itself.
1107 @subheading Implementation notes
1109 There is an existing make variable QUIET_BUILD, which
1110 alter the amount of output being displayed
1112 http://lilypond.org/doc/v2.15/Documentation/contributor/useful-make-variables}
1113 ). We are not planning on keeping this make variable.
1115 The standard way for GNU packages to give more output is with a
1116 @code{V=x} option. Presumably this is done by increasing
1117 @code{x}? If we support this option, we should still write log
1118 files; we would simply print more of the info in those log files
1121 The command @code{tee} may be useful to write to a file and
1122 display to stdout (in the case of VERBOSE).
1125 @subsubheading Discussions
1128 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00378.html}
1129 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00703.html}
1134 @n ode GOP-PROP 10 - scheme indentation
1135 @s ubsubsection GOP-PROP 10 - scheme indentation
1137 still under discussion
1139 @subsubheading Discussions
1142 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00625.html}
1143 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg01026.html}
1150 @node Grand LilyPond Input Syntax Standardization (GLISS)
1151 @section Grand LilyPond Input Syntax Standardization (GLISS)
1157 Start: sortly after 2.14 comes out, which is currently estimated
1158 to happen in January 2011.
1161 Length: 6-12 months. We're not going to rush this.
1164 Goal: define an input which we commit to being
1165 machine-updateable for the forseeable future. Any future patches
1166 which change the syntax in a non-convert-ly-able format will be
1167 rejected. (subject to the limitations, below)
1168 Once this is finished, we will release lilypond 3.0.
1173 @subheading The Problem
1175 One of the biggest complaints people have with lilypond -- other
1176 than silly thing like "there's no gui" -- is the changing syntax.
1177 Now, inventing a language or standards is difficult. If you set
1178 it in stone too soon, you risk being stuck with decisions which
1179 may limit matters. If you keep on updating the syntax,
1180 interaction with older data (and other programs!) becomes complex.
1182 @subheading Scope and Limitations
1186 tweaks will not be included. Anything with \override, \set,
1187 \overrideProperty, \tweak, \revert, \unset... including even those
1188 command names themselves... is still fair game for NOT_SMART
1192 other than that, everything is on the table. Is it a problem to
1193 have the tagline inside \header? What should the default behavior
1194 of \include be? When we abolish \times, do we move to \tuplet 3:2
1195 or \tuplet 2/3 or what (for typical triplets in 4/4 time)?
1198 we need to get standards for command names. This will help users
1199 remember them, and reduce the options for future names (and
1200 potential renamings later on). \commandOn and \commandOff seem to
1201 work well (should we *always* have an Off command?), but what
1202 about the "command" part? Should it be \nounVerbOn, or
1203 \verbNounOn ? Or \verbNotesWithExtraInformationOn ?
1206 we need standards for the location of commands. Ligature
1207 brackets, I'm looking at you. (non-postfix notation must die!)
1210 this Grand Project doesn't affect whether we have a 2.16 or not.
1211 The main problem will be deciding what to do (with a bit of
1212 messiness anticipated for \tuplet); we should definitely release a
1213 2.16 before merging _any_ of these changes.
1216 we obviously can't /guarantee/ that we'll /never/ make any
1217 non-convert-ly changes in the basic format. But we *can*
1218 guarantee that such changes would force lilypond 4.0, and that we
1219 would only do so for overwhelmingly good reasons.
1223 @subheading Workflow
1227 We're going to have lots and lots of emails flying around. The
1228 vast majority won't really fit into either -devel or -user, so
1229 we'll use a list devoted to syntax issues.
1232 Once we have a serious proposal that gained general acceptance
1233 from the separate syntax mailing list, I'll bring it to -devel.
1234 We're not going to make any changes without discussing it on
1235 -devel, but if we're going to have huge threads about English
1236 grammar and silly ideas, and I don't want to clutter up -devel.
1237 Once whatever chaotic silliness on the syntax list is settled
1238 down, I'll bring the ideas to -devel.
1241 as with GDP, I'll moderate the discussion. Not as with mailist
1242 moderation, but rather by introducing issues at specific times.
1243 We don't want a free-for-all discussion of all parts of the syntax
1244 at once; nothing will get resolved.
1247 Whenever possible, we'll decide on policies at the highest level
1248 of abstraction. For example, consider \numericTimeSignature,
1249 \slurUp, \xNotesOn, \startTextSpan, and \verylongfermata. One of
1250 them starts with the name of the notation first (slur). One has
1251 an abbreviation (x instead of cross). One has the verb at the end
1252 (On), another has it at the beginning (start). The adjective can
1253 come at the beginning (numeric, x) or end (Up). Most are in
1254 camelCase, but one isn't (verylongfermata).
1257 Instead of arguing about each individual command, we'll decide on
1258 abstract questions. Should each command begin the notation-noun,
1259 or the verb? Should all commands be in camelCase, or should we
1260 make everything other than articulations in camelCase but make
1261 articulations all lower-case? Are abbreviations allowed?
1264 Once we've answered such fundamental questions, most of the syntax
1265 should fall into place pretty easily. There might be a few odd
1266 questions left ("is it a span, or a spanner?"), but those can be
1267 settled fairly quickly.
1271 @subheading Implementation
1273 Nothing until the project is finished, then we declare the next
1274 stable release (2.16.0 or 2.18.0 ?) to be the final 2.x version,
1275 release it, then apply all the GLISS syntax changes and start
1276 testing a beta for 3.0 a week or two later.
1278 @subheading Discussion
1280 Don't respond to any of the specifics yet. Yes, we all have our
1281 pet irritations (like "what's up with \paper and \layout?!").
1282 There will be plenty of time to discuss them once GLISS starts.
1284 That said, we have a list of specific items that people really
1285 wanted to have written down. See @ref{Specific GLISS issues}.
1288 * Specific GLISS issues::
1292 @node Specific GLISS issues
1293 @subsection Specific GLISS issues
1297 add regtests for every piece of syntax (not one-command-per-file,
1298 but making a few files which, between them, use every single piece
1299 of syntax.) This is a great test for convert-ly.
1302 should GLISS cover suggested conventions? (indentation,
1303 one-bar-per-line, etc -- the kind of stuff we list for the
1304 lilypond formatting in the docs ?)
1307 how much (if any) syntactic sugar should we add? i.e.
1309 \instrumentName #'foo
1311 \set Staff.instrumentName
1313 ? Carl: maybe yes, Neil: no. (for example, it fails for
1317 the values that are used as arguments to common used overrides.
1318 Sometimes they are a symbol (e.g. #'around), sometimes a
1319 predefined variable referring to a Scheme value or object (e.g.
1320 #LEFT, #all-visible ). The main trouble is that for novice users
1321 it is not clear when there should be an apostrophe and when not.
1324 When do we need -\command and when is it just \command ?
1328 Command-line options to the lilypond binary. -dfoo counts as a
1329 tweak; we won't be trying to pin those down.
1342 If would be pedagogically simpler to realize this difference if
1343 the syntax was separate if you define a context from scratch (as
1344 is the case with \RemoveEmptyStaffContext) or if it's defined by
1345 adding onto an existing context. For example, a syntax like
1349 % Copy the current settings of the Staff context:
1351 % do whatever additional settings
1353 %%% could be used to distinguish from
1355 % Take settings from a variable:
1357 % do whatever additional settings
1363 % Start from scratch:
1372 Capitalization of identifiers: \VoiceOne ?
1377 { music expression } * 4
1379 \repeat unfold 4 { music expression }
1384 @uref{http://lists.gnu.org/archive/html/lilypond-devel/2010-04/msg00467.html}
1388 Personally, I find it easier to understand when there's a repeated
1389 8 in the half-bar position; it's much easier to see that you have
1394 %%% instead of one group of eight:
1399 trivially simple bar-lines:
1403 encourage, allow, or discourage, or disallow?
1406 indentation of \\ inside a @{@} construct.
1410 barline checks at the end of line should be preceded by at least 2
1411 spaces? barline checks should line up if possible (i.e. if you
1412 can use less than 4, 8, X empty spaces before a barline check to
1416 Why doesn't \transpose respect \relative mode?
1420 on \score vs. \new Score
1422 But in the light of a consistent syntax and semantic, I see no
1423 reason (from the users POV) to disallow it. After all, the real
1424 top-level context is a \book @{@}, isn't it, and I don't see a point
1425 in disallowing a \new Score construct just like \new Staff.
1427 From a syntactical POV, I see the following pros for \new Score:
1428 - You can write \with @{ ... @} for every other context but \Score,
1429 which (for consistency) should also work with \new Score.
1430 - When there's a \new Foo Bar, there's also a \context Foo Bar,
1431 which makes the same as a parallel instantiation of all Bar's.
1432 - [Quoting Rune from
1433 @uref{http://www.mail-archive.com/lilypond-devel@@gnu.org/msg14713.html}
1434 "I know that the \score-statement is a syntactical construct,
1435 but I think it would be nice to hide this fact from the users. I
1436 think we could make the use of score-block much more intuitive if
1437 changing the syntax to \new \Score and adding an implicit
1438 sequential-statement to the score."
1443 @uref{http://code.google.com/p/lilypond/issues/detail?id=1322}
1444 about \new vs. \context.
1448 Let users add their own items to the parser? comment 11 on:
1449 @uref{http://code.google.com/p/lilypond/issues/detail?id=1322}
1452 should engravers be pluralized (note_heads_engraver) or not
1453 (note_head_engraver) ?
1456 should we allow numbers in identifier names? Issue:
1457 @uref{http://code.google.com/p/lilypond/issues/detail?id=1670}
1460 should we officially allow accented characters? in general, how
1461 do we feel about utf-8 stuff?
1464 for the sake of completeness/simplicity, what about *disallowing*
1465 the "one-note" form of a music expression? i.e. only allowing
1468 \transpose c d { e1 }
1469 \transpose c d << e1 >>
1478 What should be the officially encouraged way of writing music for
1479 transposing instruments? Maybe it should be simplified?
1480 See http://lists.gnu.org/archive/html/lilypond-user/2011-07/msg00130.html
1485 @node Unsorted policies
1486 @section Unsorted policies
1488 @subsubheading Language-specific mailing lists
1490 A translator can ask for an official lilypond-xy mailing list once
1491 they've finished all @qq{priority 1} translation items.
1493 @subsubheading Performing yearly copyright update (@qq{grand-replace})
1495 At the start of each year, copyright notices for all source files
1496 should be refreshed by running the following command from the top of
1503 Internally, this invokes the script @file{scripts/build/grand-replace.py},
1504 which performs a regular expression substitution for old-year -> new-year
1505 wherever it finds a valid copyright notice.
1507 Note that snapshots of third party files such as @file{texinfo.tex} should
1508 not be included in the automatic update; @file{grand-replace.py} ignores these
1509 files if they are listed in the variable @code{copied_files}.
1512 @subsubheading Push git access
1514 Git access is given out when a contributor has a significant
1515 record of patches being accepted without problems. If existing
1516 developers are tired of pushing patches for a contributor, we'll
1517 discuss giving them push access. Unsolicited requests from
1518 contributors for access will almost always be turned down.