1 @c -*- coding: utf-8; mode: texinfo; -*-
2 @node Administrative policies
3 @chapter Administrative policies
5 This chapter discusses miscellaneous administrative issues which
6 don't fit anywhere else.
9 * Meta-policy for this document::
11 * Administrative mailing list::
12 * Grand Organization Project (GOP)::
13 * Grand LilyPond Input Syntax Standardization (GLISS)::
17 @node Meta-policy for this document
18 @section Meta-policy for this document
20 The Contributor's Guide as a whole is still a work in progress,
21 but some chapters are much more complete than others. Chapters
22 which are @qq{almost finished} should not have major changes
23 without a discussion on @code{-devel}; in other chapters, a
24 disorganized @qq{wiki-style dump} of information is encouraged.
26 Do not change (other than spelling mistakes) without discussion:
31 @ref{Introduction to contributing}
34 @ref{Working with source code}
38 Please dump info in an appropriate @@section within these manuals,
39 but discuss any large-scale reorganization:
47 @ref{Documentation work}
53 @ref{Regression tests}
56 @ref{Programming work}
61 Totally disorganized; do whatever the mao you want:
75 @ref{Administrative policies}
84 We have four jobs for organizing a team of contributors:
89 Bug Meister: trains new Bug Squad volunteers, organizes who works
90 on which part of their job, checks to make sure that everything is
91 running smoothly, and has final say on our policy for Issues.
96 Doc Meister: trains new doc editors/writers, organizes who works
97 on which part of the job, checks to make sure that everything is
98 running smoothly, and has final say on our policy for
99 Documentation. Also includes LSR work.
104 Translation Meister: trains new translators, updates the
105 translation priority list, and handles merging branches (in both
111 Frog Meister: is responsible for code patches from (relatively)
112 inexperienced contributors. Keeps track of patches, does initial
113 reviewing of those patches, sends them to @code{-devel} when
114 they've had some initial review on the Frog list, pesters the
115 @code{-devel} community into actually reviewing said patches, and
116 finally pushes the patches once they're accepted. This person is
117 @emph{not} responsible for training new programmers, because that
118 would be far too much work -- he job is @qq{only} to guide
119 completed patches through our process.
125 @node Administrative mailing list
126 @section Administrative mailing list
128 An mailing list for administrative issues is maintained at
129 @code{lilypond-hackers@@gnu.org}.
131 This list is intended to be used for discussions that should be kept
132 private. Therefore, the archives are closed to the public.
134 Subscription to this list is limited to certain senior developers.
136 At the present time, the list is dormant.
138 Details about the criteria for membership, the types of discussion
139 to take place on the list, and other policies for the hackers list
140 will be finalized during the
141 @ref{Grand Organization Project (GOP)}.
145 @node Grand Organization Project (GOP)
146 @section Grand Organization Project (GOP)
152 Clarify the various development tasks by writing down the polices
153 and techniques and/or simplifying the tasks directly.
156 Get more people involved in development: specifically, find people
157 to do easy tasks to allow advanced developers to concentrate on
169 @subsection Motivation
171 Most readers are probably familiar with the LilyPond Grand
172 Documentation Project, which ran from Aug 2007 to Aug 2008. This
173 project involved over 20 people and resulted in an almost complete
174 rewrite of the documentation. Most of those contributors were
175 normal users who decided to volunteer their time and effort to
176 improve lilypond for everybody. By any measure, it was a great
179 The Grand Organization Project aims to do the same thing with a
180 larger scope -- instead of focusing purely on documentation, the
181 project aims to improve all parts of LilyPond and its community.
182 Just as with GDP, the main goal is to encourage and train users to
183 become more involved.
185 If you have never contributed to an open-source project before --
186 especially if you use Windows or OSX and do not know how to
187 program or compile programs -- you may be wondering if there's
188 anything you can do. Rest assured that you @emph{can} help.
190 @subheading "Trickle-up" development
192 One of the reasons I'm organizing GOP is "trickle-up"
193 development. The idea is this: doing easy tasks frees up advanced
194 developers to do harder tasks. Don't ask "am I the @emph{best}
195 person for this job"; instead, ask "am I @emph{capable} of doing
196 this job, so that the current person can do stuff I @emph{can't}
199 For example, consider lilypond's poor handling of grace notes in
200 conjunction with clef and tempo changes. Fixing this will require
201 a fair amount of code rewriting, and would take an advanced
202 developer a few weeks to do. It's clearly beyond the scope of a
203 normal user, so we might as well sit back and do nothing, right?
205 No; we @emph{can} help, indirectly. Suppose that our normal user
206 starts answering more emails on lilypond-user. This in turn means
207 that documentation writers don't need to answer those emails, so
208 they can spend more time improving the docs. I've noticed that all
209 doc writers tackle harder and harder subjects, and when they start
210 writing docs on scheme programming and advanced tweaks, they start
211 contributing bug fixes to lilypond. Having people performing these
212 easy-to-moderate bug fixes frees up the advanced developers to
213 work on the really hard stuff... like rewriting the grace note
216 Having 1 more normal user answering emails on lilypond-user won't
217 have a dramatic trick-up affect all by himself, of course. But if
218 we had 8 users volunteering to answer emails, 6 users starting to
219 write documentation, and 2 users editing LSR... well, that would
220 free up a lot of current bug-fixing-capable contributors to focus
221 on that, and we could start to make a real dent in the number of
222 bugs in lilypond. Quite apart from the eased workload, having that
223 many new helpers will provide a great moral boost!
226 @subsection Ongoing jobs
228 Although GOP is a short-term project, the main goal is to train
229 more people to handle ongoing jobs. The more people doing these
230 jobs, the ligher the work will be, and the more we can get done
233 Also, it would be nice if we had at least one "replacement" /
234 "understudy" for each role -- too many tasks are only being done
235 by one person, so if that person goes on vacation or gets very
236 busy with other matters, work in that area grinds to a halt.
238 @subheading Jobs for normal users
242 LilyPond is sometimes critized for not listening to users, but
243 whenever we ask for opinions about specific issues, we never get
244 enough feedback. This is somewhat aggravating.
245 We need a group of users to make a dedicated effort to test and
246 give feedback. If there's new documentation, read it. If there's
247 an experimental binary, download it and try compiling a score with
248 it. If we're trying to name a new command, think about it and give
251 @item lilypond-user support:
252 I think it would be nice if we had an official team of users
255 @item LilyPond Report:
256 Keeping a monthly newsletter running is a non-trivial task. A lot
257 of work is needed to organize it; it would be great if we could
258 split up the work. One person could write the Snippet of the
259 Month, another person could do Quotes of the Month, another person
260 could do interviews, etc.
263 Although GDP (the Grand Documentation Project) did great work,
264 there's still many tasks remaining.
267 Keeping the documentation translations is a monumental task; we
268 need all the help we can get!
272 @subheading Jobs for advanced users for developers
275 @item Git help for writers:
276 We often receive reports of typos and minor text updates to the
277 documentation. It would be great if somebody could create
278 properly-formatted patches for these corrections.
280 Technical requirements: ability to run @ref{Lilydev}.
283 LSR contains many useful examples of lilypond, but some snippets
284 are out of date and need updating. Other snippets need to be
285 advertized, and new snippets need to be sorted. We could use
286 another person to handle LSR.
288 Technical requirements: use of a web browser. LilyPond
289 requirements: you should be familiar with most of Notation
290 chapters 1 and 2 (or be willing to read the docs to find out).
292 @item Join the Frogs:
293 "Frogs" are a team of bug-fixers (because frogs eat bugs, and you
294 often find them in Ponds of Lilies) and new feature implementors.
296 Technical requirements: development environment (such as
297 @ref{Lilydev}), ability to read+write scheme and/or C++ code.
302 @node Policy decisions
303 @subsection Policy decisions
305 There are a number of policy decisions -- some of them fairly
306 important -- which we have been postponing for a few years. We
307 are now discussing them slowly and thoroughly; agenda and exact
308 proposals are online:
311 @uref{http://lilypond.org/~graham/gop/index.html}
314 Below is a list of policies which are not @qq{on the agenda} yet.
316 Note that the presence of an item on this list does @emph{not}
317 mean that everybody thinks that something needs to be done.
318 Inclusion in this simply means that one developer thinks that we
319 should discuss it. We are not going to filter this list; if any
320 developer thinks we should discuss something, just add it to the
321 bottom of the list. (the list is unsorted)
323 As GOP progresses, items from this list will be put on the agenda
324 and removed from this list. I generally try to have one month's
325 discussion planned in advance, but I may shuffle things around to
326 respond to any immediate problems in the developer community.
328 There are some item(s) not displayed here; these are questions
329 that were posed to me privately, and I do not feel justified in
330 discussing them publicly without the consent of the person(s) that
331 brought them up. They will initially be discussed privately on the
332 lilypond-hackers mailing list -- but the first question will be
333 "do we absolutely need to do this privately", and if not, the
334 discussion will take place on lilypond-devel like the other items.
336 In most policy discussions in lilypond over the past few years,
337 the first half (or more) is wasted arguing on the basis of
338 incorrect or incomplete data; once all the relevant facts are
339 brought to light, the argument is generally resolved fairly
340 quickly. In order to keep the GOP discussions focused, each topic
341 will be introduced with a collection of relevant facts and/or
342 proposals. It is, of course, impossible to predict exactly which
343 facts will be relevant to the discussion -- but spending an hour
344 or two collecting information could still save hours of
347 @warning{The estimated time required for "prep work", and the
348 following discussion, has been added to each item. At the moment,
349 there is an estimated 30 hours of prep work and 140 hours of
353 @item @strong{Patch reviewing}:
354 At the time of this writing, we have 23 (known) patches waiting
355 for review. Some from main developers; some from new developers.
356 We desperately need more people helping with lilypond, but
357 ignoring patches is the best way to drive potential contributors
358 away. This is not good.
360 (prep: 2 hours. discuss: 10 hours)
362 @item @strong{Future release policy}:
363 (how) should we change any policies pertaining to releases? Should
364 an undocumented new feature count as release-blocking?
366 (prep: 1 hour. discuss: 15 hours)
368 @item @strong{lilypond-hackers mailing list}:
369 Should we have a private mailing list for senior developers? If
370 so, who should be on it?
372 (prep: 2 hours+3 weeks. discuss: 10 hours)
374 @item @strong{Hackers B}:
377 @item @strong{Git repository(s)}:
378 We currently have a web/ branch in our main repo; this seems
379 misleading to new developers. More generally, should we have
380 branches that aren't related to the master? i.e. should we
381 restrict a git branch to code which is an actual "branch" of
382 development? Also, some of our code (notably the windows and osx
383 lilypad) isn't in a git repository at all.
384 We can add new repositories very easily; should make repositories
387 git://git.sv.gnu.org/lilypond/gub.git
388 git://git.sv.gnu.org/lilypond/lilypad.git
389 git://git.sv.gnu.org/lilypond/misc.git
391 ? More information here:
392 @uref{http://code.google.com/p/lilypond/issues/detail?id=980}
394 (prep: 2 hours. discuss: 10 hours)
396 @item @strong{Roadmap of future development}:
397 Many projects have a roadmap of planned (or desired) future work.
398 Should we use one? If so, what should go on it, bearing in mind
399 our volunteer status? Is there any way of having a roadmap that
402 (prep: 1 hour. discuss: 5 hours)
404 @item @strong{Official links to other organizations?}:
405 There's something called the "software freedom conservancy", and
406 in general, there's a bunch of "umbrella organizations". Joining
407 some of these might give us more visibility, possibly leading to
408 more users, more developers, maybe even financial grants or use in
411 (prep: 2 hours. discuss: 5 hours)
413 @item @strong{Mailing lists}:
414 We currently have a mix of official GNU mailing lists and lilynet
415 lists. Is there a strong rationale for having separate mailing
416 list servers? Why not pick one place, and put all our lists there?
417 (or at least, all "permanent" lists?)
419 (prep: 1 hour. discuss: 5 hours)
421 @item @strong{Issue tracking with google code}:
422 We use the google issue tracker, but this means that we are
423 relying on a commercial entity for a large part of our
424 development. Would it be better (safer in the long run) to use the
425 savannah bug tracker?
427 (prep: 1 hour. discuss: 5 hours)
429 @item @strong{Patch review tool}:
430 Reitveld is inconvenient in some respects: it requires a google
431 account, and there's no way to see all patches relating to
432 lilypond. Should we switch to something like gerritt?
433 @uref{http://code.google.com/p/lilypond/issues/detail?id=1184}
435 (prep: 5 hours. discuss: 15 hours)
437 @item @strong{Subdomains of *.lilypond.org}:
438 Unless Jan has a really weird DNS hosting setup, there are no
439 technical barriers to having names like lsr.lilypond.org,
440 frog.lilypond.org, or news.lilypond.org. Is this something that we
443 (prep: 1 hours+2 weeks. discuss: 5 hours)
445 @item @strong{Authorship in source files}:
446 Our documentation currently does not attempt to track individual
447 authors of each file, while our source code makes a confused and
448 jumbled attempt to track this. A number of guidelines for F/OSS
449 projects explicitly recommends _not_ tracking this in individual
450 files, since the code repository will track that for you.
452 (prep: 2 hours. discuss: 15 hours)
454 @item @strong{Clarity for sponsorships}:
455 We currently do not advertize bounties and sponsorships on the
456 webpage. How much advertising do we want, and what type?
457 Should we change the "structure" / "framework" for bounties?
459 (prep: 2 hours. discuss: 10 hours)
461 @item @strong{Separate branches for active development}:
462 it might be good to have @emph{everybody} working on separate
463 branches. This complicates the git setup, but with sufficient
464 logic in lily-git.tcl, we can probably make it transparent to
465 newbies. However, we'd need a reliable person to handle all the
466 required merging and stuff.
468 (prep: 2 hours. discuss: 10 hours)
470 @item @strong{When do we add regtests?}:
471 There is a discrepancy between our stated policy on adding
472 regtests, and our actual practice in handling bugs and patches.
475 There is also a wider question how to organize the regtests, such
476 as where to put interesting-console-output regtests, including
477 stuff like lilypond-book and midi2ly in a sensible manner, and
478 possibly including regtests for currently-broken functionality.
480 (prep: 2 hours. discuss: 5 hours)
482 @item @strong{code readability}:
483 "Our aim when producing source code for Lilypond in whatever
484 language is that it should be totally comprehensible to a
485 relatively inexperienced developer at the second reading."
488 - aids maintainability of code base
489 - "second reading" so newer developers can look up unfamiliar
491 - will help to keep things simple, even if the code is doing
492 complex stuff discourages "secret squirrel" coding, e.g. "how
493 much functionality can I squeeze into as few characters as
494 possible" "comments are for wimps"
495 - will aid not *discouraging* new developers to join the project
497 (prep: 2 hours. discuss: 10 hours)
499 @item @strong{C++ vs. scheme}:
500 what should be in scheme, what should be in C++, what can/should
501 be ported from one to the other, etc. Questions of
502 maintainability, speed (especially considering guile 2.0), and the
503 amount of current material in either form, are important.
505 (prep: 5 hours. discuss: 15 hours)
511 @node Grand LilyPond Input Syntax Standardization (GLISS)
512 @section Grand LilyPond Input Syntax Standardization (GLISS)
518 Start: sortly after 2.14 comes out, which is currently estimated
519 to happen in January 2011.
522 Length: 6-12 months. We're not going to rush this.
525 Goal: define an input which we commit to being
526 machine-updateable for the forseeable future. Any future patches
527 which change the syntax in a non-convert-ly-able format will be
528 rejected. (subject to the limitations, below)
529 Once this is finished, we will release lilypond 3.0.
534 @subheading The Problem
536 One of the biggest complaints people have with lilypond -- other
537 than silly thing like "there's no gui" -- is the changing syntax.
538 Now, inventing a language or standards is difficult. If you set
539 it in stone too soon, you risk being stuck with decisions which
540 may limit matters. If you keep on updating the syntax,
541 interaction with older data (and other programs!) becomes complex.
543 @subheading Scope and Limitations
547 tweaks will not be included. Anything with \override, \set,
548 \overrideProperty, \tweak, \revert, \unset... including even those
549 command names themselves... is still fair game for NOT_SMART
553 other than that, everything is on the table. Is it a problem to
554 have the tagline inside \header? What should the default behavior
555 of \include be? When we abolish \times, do we move to \tuplet 3:2
556 or \tuplet 2/3 or what (for typical triplets in 4/4 time)?
559 we need to get standards for command names. This will help users
560 remember them, and reduce the options for future names (and
561 potential renamings later on). \commandOn and \commandOff seem to
562 work well (should we *always* have an Off command?), but what
563 about the "command" part? Should it be \nounVerbOn, or
564 \verbNounOn ? Or \verbNotesWithExtraInformationOn ?
567 we need standards for the location of commands. Ligature
568 brackets, I'm looking at you. (non-postfix notation must die!)
571 this Grand Project doesn't affect whether we have a 2.16 or not.
572 The main problem will be deciding what to do (with a bit of
573 messiness anticipated for \tuplet); we should definitely release a
574 2.16 before merging _any_ of these changes.
577 we obviously can't /guarantee/ that we'll /never/ make any
578 non-convert-ly changes in the basic format. But we *can*
579 guarantee that such changes would force lilypond 4.0, and that we
580 would only do so for overwhelmingly good reasons.
588 We're going to have lots and lots of emails flying around. The
589 vast majority won't really fit into either -devel or -user, so
590 we'll use a list devoted to syntax issues.
593 Once we have a serious proposal that gained general acceptance
594 from the separate syntax mailing list, I'll bring it to -devel.
595 We're not going to make any changes without discussing it on
596 -devel, but if we're going to have huge threads about English
597 grammar and silly ideas, and I don't want to clutter up -devel.
598 Once whatever chaotic silliness on the syntax list is settled
599 down, I'll bring the ideas to -devel.
602 as with GDP, I'll moderate the discussion. Not as with mailist
603 moderation, but rather by introducing issues at specific times.
604 We don't want a free-for-all discussion of all parts of the syntax
605 at once; nothing will get resolved.
608 Whenever possible, we'll decide on policies at the highest level
609 of abstraction. For example, consider \numericTimeSignature,
610 \slurUp, \xNotesOn, \startTextSpan, and \verylongfermata. One of
611 them starts with the name of the notation first (slur). One has
612 an abbreviation (x instead of cross). One has the verb at the end
613 (On), another has it at the beginning (start). The adjective can
614 come at the beginning (numeric, x) or end (Up). Most are in
615 camelCase, but one isn't (verylongfermata).
618 Instead of arguing about each individual command, we'll decide on
619 abstract questions. Should each command begin the notation-noun,
620 or the verb? Should all commands be in camelCase, or should we
621 make everything other than articulations in camelCase but make
622 articulations all lower-case? Are abbreviations allowed?
625 Once we've answered such fundamental questions, most of the syntax
626 should fall into place pretty easily. There might be a few odd
627 questions left ("is it a span, or a spanner?"), but those can be
628 settled fairly quickly.
632 @subheading Implementation
634 Nothing until the project is finished, then we declare the next
635 stable release (2.16.0 or 2.18.0 ?) to be the final 2.x version,
636 release it, then apply all the GLISS syntax changes and start
637 testing a beta for 3.0 a week or two later.
639 @subheading Discussion
641 Don't respond to any of the specifics yet. Yes, we all have our
642 pet irritations (like "what's up with \paper and \layout?!").
643 There will be plenty of time to discuss them once GLISS starts.
645 That said, we have a list of specific items that people really
646 wanted to have written down. See @ref{Specific GLISS issues}.
649 * Specific GLISS issues::
653 @node Specific GLISS issues
654 @subsection Specific GLISS issues
658 add regtests for every piece of syntax (not one-command-per-file,
659 but making a few files which, between them, use every single piece
660 of syntax.) This is a great test for convert-ly.
663 should GLISS cover suggested conventions? (indentation,
664 one-bar-per-line, etc -- the kind of stuff we list for the
665 lilypond formatting in the docs ?)
668 how much (if any) syntactic sugar should we add? i.e.
670 \instrumentName #'foo
672 \set Staff.instrumentName
674 ? Carl: maybe yes, Neil: no. (for example, it fails for
678 the values that are used as arguments to common used overrides.
679 Sometimes they are a symbol (e.g. #'around), sometimes a
680 predefined variable referring to a Scheme value or object (e.g.
681 #LEFT, #all-visible ). The main trouble is that for novice users
682 it is not clear when there should be an apostrophe and when not.
685 When do we need -\command and when is it just \command ?
689 Command-line options to the lilypond binary. -dfoo counts as a
690 tweak; we won't be trying to pin those down.
703 If would be pedagogically simpler to realize this difference if
704 the syntax was separate if you define a context from scratch (as
705 is the case with \RemoveEmptyStaffContext) or if it's defined by
706 adding onto an existing context. For example, a syntax like
710 % Copy the current settings of the Staff context:
712 % do whatever additional settings
714 %%% could be used to distinguish from
716 % Take settings from a variable:
718 % do whatever additional settings
724 % Start from scratch:
733 Capitalization of identifiers: \VoiceOne ?
738 { music expression } * 4
740 \repeat unfold 4 { music expression }
744 @uref{http://lists.gnu.org/archive/html/lilypond-devel/2010-04/msg00467.html}
747 Personally, I find it easier to understand when there's a repeated
748 8 in the half-bar position; it's much easier to see that you have
753 %%% instead of one group of eight:
758 trivially simple bar-lines:
762 encourage, allow, or discourage, or disallow?
765 indentation of \\ inside a @{@} construct.
769 barline checks at the end of line should be preceded by at least 2
770 spaces? barline checks should line up if possible (i.e. if you
771 can use less than 4, 8, X empty spaces before a barline check to
775 Why doesn't \transpose respect \relative mode?
779 on \score vs. \new Score
781 But in the light of a consistent syntax and semantic, I see no
782 reason (from the users POV) to disallow it. After all, the real
783 top-level context is a \book @{@}, isn't it, and I don't see a point
784 in disallowing a \new Score construct just like \new Staff.
786 From a syntactical POV, I see the following pros for \new Score:
787 - You can write \with @{ ... @} for every other context but \Score,
788 which (for consistency) should also work with \new Score.
789 - When there's a \new Foo Bar, there's also a \context Foo Bar,
790 which makes the same as a parallel instantiation of all Bar's.
792 @uref{http://www.mail-archive.com/lilypond-devel@@gnu.org/msg14713.html}
793 "I know that the \score-statement is a syntactical construct,
794 but I think it would be nice to hide this fact from the users. I
795 think we could make the use of score-block much more intuitive if
796 changing the syntax to \new \Score and adding an implicit
797 sequential-statement to the score."
802 http://code.google.com/p/lilypond/issues/detail?id=1322
803 about \new vs. \context.
807 Let users add their own items to the parser? comment 11 on:
808 http://code.google.com/p/lilypond/issues/detail?id=1322
811 should engravers be pluralized (note_heads_engraver) or not
812 (note_head_engraver) ?
815 should we allow numbers in identifier names? Issue:
816 http://code.google.com/p/lilypond/issues/detail?id=1670
819 should we officially allow accented characters? in general, how
820 do we feel about utf-8 stuff?
823 for the sake of completeness/simplicity, what about *disallowing*
824 the "one-note" form of a music expression? i.e. only allowing
827 \transpose c d { e1 }
828 \transpose c d << e1 >>
839 @node Unsorted policies
840 @section Unsorted policies
842 @subsubheading Language-specific mailing lists
844 A translator can ask for an official lilypond-xy mailing list once
845 they've finished all @qq{priority 1} translation items.
847 @subsubheading Performing yearly copyright update (@qq{grand-replace})
849 At the start of each year, copyright notices for all source files
850 should be refreshed by running the following command from the top of
857 Internally, this invokes the script @file{scripts/build/grand-replace.py},
858 which performs a regular expression substitution for old-year -> new-year
859 wherever it finds a valid copyright notice.
861 Note that snapshots of third party files such as @file{texinfo.tex} should
862 not be included in the automatic update; @file{grand-replace.py} ignores these
863 files if they are listed in the variable @code{copied_files}.
866 @subsubheading Push git access
868 Git access is given out when a contributor has a significant
869 record of patches being accepted without problems. If existing
870 developers are tired of pushing patches for a contributor, we'll
871 discuss giving them push access. Unsolicited requests from
872 contributors for access will almost always be turned down.