1 @node Administrative policies
2 @chapter Administrative policies
4 This chapter discusses miscellaneous administrative issues which
5 don't fit anywhere else.
8 * Meta-policy for this document::
9 * Environment variables::
11 * Automated testing with Patchy::
12 * Administrative mailing list::
13 * Grand Organization Project (GOP)::
14 * Grand LilyPond Input Syntax Standardization (GLISS)::
18 @node Meta-policy for this document
19 @section Meta-policy for this document
21 The Contributor's Guide as a whole is still a work in progress,
22 but some chapters are much more complete than others. Chapters
23 which are @qq{almost finished} should not have major changes
24 without a discussion on @w{@code{-devel}}; in other chapters, a
25 disorganized @qq{wiki-style dump} of information is encouraged.
27 Do not change (other than spelling mistakes) without discussion:
32 @ref{Introduction to contributing}
35 @ref{Working with source code}
39 Please dump info in an appropriate @@section within these manuals,
40 but discuss any large-scale reorganization:
48 @ref{Documentation work}
54 @ref{Regression tests}
57 @ref{Programming work}
62 Totally disorganized; do whatever the mao you want:
76 @ref{Administrative policies}
81 @node Environment variables
82 @section Environment variables
84 Some maintenance scripts and instructions in this guide rely on
85 the following environment variables. They should be predefined in
86 LilyDev distribution (see @ref{LilyDev}); if you set up your own
87 development environment, you can set them by appending these settings to
88 your @file{~/.bashrc} (or whatever defines your default environment
89 variables for the user account for LilyPond development), then logging
90 out and in (adapt directories to your setup):
93 LILYPOND_GIT=~/lilypond-git
95 LILYPOND_BUILD_DIR=~/lilypond-git/build
96 export LILYPOND_BUILD_DIR
99 The standard build and install procedure (with @code{autogen.sh},
100 @code{configure}, @code{make}, @code{make install}, @code{make doc}
101 @dots{}) does not rely on them.
103 In addition, for working on the website, @code{LILYPOND_WEB_MEDIA_GIT}
104 should be set to the repository lilypond-extra, see
105 @ref{lilypond-extra}.
111 We have four jobs for organizing a team of contributors:
116 Bug Meister: trains new Bug Squad volunteers, organizes who works
117 on which part of their job, checks to make sure that everything is
118 running smoothly, and has final say on our policy for Issues.
123 Doc Meister: trains new doc editors/writers, organizes who works
124 on which part of the job, checks to make sure that everything is
125 running smoothly, and has final say on our policy for
126 Documentation. Also includes LSR work.
131 Translation Meister: trains new translators, updates the
132 translation priority list, and handles merging branches (in both
140 @node Automated testing with Patchy
141 @section Automated testing with Patchy
144 * Overview of Patchy::
145 * Patchy requirements::
146 * Installing Patchy::
147 * Configuring Patchy::
148 * Running the scripts::
149 * Automating Patchy::
150 * Troubleshooting Patchy::
153 @node Overview of Patchy
154 @subsection Overview of Patchy
156 Patchy is a set of Python scripts used for testing patches or testing &
157 pushing new commits added to @code{remote/origin/staging} to
158 @code{remote/origin/master}.
160 No programmatic skill is required to run either of the scripts; although
161 knowledge of compiling LilyPond and its documentation along with
162 understanding how to configure the @var{PATH} environment of your
163 computer is required. See @ref{Working with source code}.
165 The two scripts that are used for each function are:
169 @code{test-patches.py}. This script tests issues labelled as
170 @qq{Patch-New} on the LilyPond issue tracker. Part of the testing
171 process involves running the regression tests, so this script always
172 requires some human intervention in order to visually check for any
173 differences that may be present after a successful test has occurred.
176 @code{lilypond-patchy-staging.py}. This script checks for any new
177 commits in @code{remote/origin/staging}, makes sure that the new HEAD
178 compiles along with all the LilyPond documentation. Then finally
179 pushing to @code{remote/origin/master}. This script can be run and left
180 unattended, requiring no human intervention.
184 Both of the scripts can be run independently of each other and it is not
185 necessary to be able to run both. So if you wanted to contribute to
186 LilyPond development, for example by @emph{just} testing patches then
187 this would still be a helpful contribution to LilyPond's development.
189 Patchy can also be configured to send emails after each successful (or
190 unsuccessful) operation. This is not a requirement and is turned off
193 @c Need to explain in more detail how to set up Patchy for email but
194 @c as I don't use myself it I have no experience - JL
197 @node Patchy requirements
198 @subsection Patchy requirements
200 @subsubsubheading Testing new patches
205 A full local copy of the source code. See
206 @ref{Working with source code}.
209 All the software needed for compiling LilyPond @emph{and} the
210 documentation. Although being able to build the full set of LilyPond's
211 manuals is not mandatory for testing (most) patches, part of the patch
212 testing process requires that the regression tests are run and it is
213 this that requires the software normally used for compiling
214 documentation. See @ref{Compiling}.
217 Commit access is @emph{not} required to test patches, but a valid login
218 to @uref{http://code.google.com/} @emph{is}.
221 Note that a google account does not need to be a Gmail account; you can
222 use @emph{any} email address for your google account when you sign up.
224 @warning{In order for @code{patchy} to work as expected, your Google
225 Account Settings must have the @q{Access for less secure apps} set to
226 @q{Allowed}. This is normally the default setting.}
231 @subsubsubheading Testing & pushing new commits
236 A full local copy of the source code. See
237 @ref{Working with source code}.
240 All the software needed for compiling LilyPond @emph{and} the
241 documentation. Unlike testing patches, being able to build the full set
242 of LilyPond's documentation is required to be able to test & push new
243 commits. See @ref{Compiling}.
246 Commit access @emph{is} required to test and push new commits, but a
247 valid login to @uref{http://code.google.com/} is @emph{not}. See
253 @node Installing Patchy
254 @subsection Installing Patchy
256 The Patchy scripts are not part of the LilyPond code base, but can be
257 downloaded from @uref{https://github.com/gperciva/lilypond-extra/}. The
258 scripts and related Python libraries are all located in the
259 @file{patches/} directory.
261 Alternatively, use @code{git clone};
264 git clone https://github.com/gperciva/lilypond-extra/
267 This makes it simpler to update the scripts if any changes are ever made
268 to them. Finally, add the location of the @file{patches/} directory to
272 @node Configuring Patchy
273 @subsection Configuring Patchy
275 @warning{It is recommended to create a new user on your computer
276 specifically to run the Patchy scripts as a security precaution and that
277 this user should not have any administrative privileges. Also do not
278 set password protection for your ssh key else you will not be able to
279 run the scripts unattended.}
284 Make sure the environment variables @var{LILYPOND_GIT} and
285 @var{LILYPOND_BUILD_DIR} are configured appropriately. See
286 @ref{Environment variables}.
289 To save being prompted for your login and password to
290 @uref{http://code.google.com/} when testing patches. create a
291 @emph{plain-text} file in your Patchy user's @code{$HOME} directory
292 called @code{.lilypond-project-hosting-login} containing your login and
293 password, each on a separate line.
296 joe_smith123@@gmail.com
301 Manually run either the @code{test-patches.py} or
302 @code{lilypond-patchy-staging.py} scripts and when prompted:
305 Warning: using default config; please edit /home/joe/.lilypond-patchy-config
306 Are you sure that you want to continue with the default config? (y/[n])
309 Answer @qq{@code{n}} and press enter.
311 The next time either of the scripts are run they will use the
312 @code{.lilypond-patchy-config} settings copied to your @code{$HOME}
316 Manually edit the @file{.lilypond-patchy-config} file, located in your
317 @code{$HOME} directory to change any of the default settings.
326 All @code{make} operations are run with;
328 extra_make_options = -j3 CPU_COUNT=3
331 See @ref{Saving time with the -j option}
334 A complete build of all the LilyPond documentation is @emph{not}
337 patch_test_build_docs = no
341 Each instance of either a patch test or commit test & push is logged in;
343 auto_compile_results_dir = ~/lilypond-auto-compile-results/
347 Both scripts will perform their build operations in;
349 build_dir = /tmp/lilypond-autobuild/
354 Each completed patch test will also generate its own directory in
355 @file{/tmp/...} labelled with the tracker issue number prefixed by
361 Both the scripts create clones of @code{staging} and @code{master}
362 branches (prefixed with @code{test-}) with a third branch, called
363 @code{test-master-lock} used as a check to protect against two or more
364 instances of Patchy being run locally at the same time.
367 @node Running the scripts
368 @subsection Running the scripts
370 @subsubsubheading Testing & pushing new commits
372 @code{lilypond-patchy-staging.py} is run @emph{without} any arguments.
373 It then checks to see if @code{remote/origin/staging} is
374 @qq{further ahead} than @code{remote/origin/master}.
377 If there are no new differences between the two branches since the last
378 run check, the script will report something like this:
381 (UTC) Begin LilyPond compile, previous commit at 4726764cb591f622e7893407db0e7d42bcde90d9
382 Success: No new commits in staging
386 If there are any differences between the two branches since the last
387 run check, (or if the script cannot for any reason, locate the last
388 instance of a commit that it checked) it will report something like
392 (UTC) Begin LilyPond compile, previous commit at 4726764cb591f622e7893407db0e7d42bcde90d9
393 Merged staging, now at: 79e98a773b6570cfa28a15775a9dea3d3e54d6b5
394 Success: ./autogen.sh --noconfigure
395 Success: /tmp/lilypond-autobuild/configure --disable-optimising
399 and proceed with running @code{make}, @code{make test} and a
400 @code{make doc}. Unlike @code{test-patches.py} if all the tests pass,
401 the script then pushes the changes to @code{remote/origin/master}.
405 Success: nice make clean
406 Success: nice make -j7 CPU_COUNT=7
407 Success: nice make test -j7 CPU_COUNT=7
408 Success: nice make doc -j7 CPU_COUNT=7
409 To ssh://joe@@git.sv.gnu.org/srv/git/lilypond.git
410 79e98a7..4726764 test-staging -> master
411 Success: pushed to master
414 @warning{In the case where any of the @code{lilypond-patchy-staging.py}
415 tests fail, do not try to push your own fixes but report the failures to
416 the Developers List <lilypond-devel@@gnu.org> for advice.}
419 @subsubsubheading Testing new patches
421 When run without any argument, @code{test-patches.py} will check
422 @uref{http://code.google.com/p/lilypond/issues/list} for all tracker
423 issues that are marked with the label @code{Patch-new}. It then scrapes
424 the issue, looking for the last Rietveld URL entered. It then downloads
425 the patch file and applies it to @code{test-master}.
427 Here is an example where two tracker issues labeled as @code{Patch-new}
434 Found url: http://codereview.appspot.com/112210043
435 Found patch: 4007,/home/joe/lilypond-git/issue112210043_1.diff,
437 Found url: http://codereview.appspot.com/115770043
438 Found patch: 4008,/home/joe/lilypond-git/issue115770043_1.diff,
439 Fetching, cloning, compiling master.
443 If run no tracker items with the @var{Patch-New} label are found it will
450 The script can also be run using the tracker issue number(s) as an
451 argument regardless if the @var{Patch-New} label has been assigned;
460 test-patches.py 4006 3992 4020
463 The script then checks to see if any previously
464 @code{make test-baseline}s have been generated and if the commit ID of
465 @code{remote/origin/master} is different from that previously completed
469 If no previous @code{make test-baseline} test is discovered or if the
470 commit ID of @code{remote/origin/master} has changed, then a new
471 @code{make test-baseline} will run first automatically before the patch
474 This shows when the commit ID has changed:
478 (UTC) Begin LilyPond compile, previous commit at 3f92dcb2c81dcd2755542b57a0a5f2039f29a211
479 Merged master, now at: 4726764cb591f622e7893407db0e7d42bcde90d9
480 Success: ./autogen.sh --noconfigure
481 Success: /tmp/lilypond-autobuild/configure --disable-optimising
482 Success: nice make clean
483 Success: nice make -j3 CPU_COUNT=3
484 Success: nice make test-baseline -j3 CPU_COUNT=3
485 Success: nice make doc -j3 CPU_COUNT=3
486 Success: nice make doc-clean
491 If a previous regression test @emph{is} discovered @emph{and} if the
492 commit ID of @code{remote/origin/master} has not changed, then the patch
493 will be tested against the previous @code{make test-baseline} without
494 the need to re-generate a new one:
500 Found url: http://codereview.appspot.com/110540043
501 Found patch: 4009,/home/joe/lilypond-extra/patches/issue110540043_1.diff,
502 Fetching, cloning, compiling master.
503 (UTC) Begin LilyPond compile, previous commit at 4726764cb591f622e7893407db0e7d42bcde90d9
504 Success: No new commits in master
505 Using test baseline from previous build.
509 The patch is then applied and a @code{make} and @code{make check} are
510 run. A full @code{make doc} is also run if the
511 @file{.lilypond-patchy-config} file has been edited accordingly;
516 Issue 4009: Testing patch issue110540043_1.diff
517 Success: git apply --index /home/joe/lilypond-git/issue112210043_1.diff
518 Success: ./autogen.sh --noconfigure
519 Success: /tmp/lilypond-autobuild/configure --disable-optimising
520 Success: nice make clean
521 Success: nice make -j3 CPU_COUNT=3
522 Success: nice make check -j3 CPU_COUNT=3
523 Success: nice make doc -j3 CPU_COUNT=3
527 Once all the tests have run (successfully or not), the script will clean
528 up from the previous patch and, if required, start testing the next
533 Issue 4007: Cleaning up
534 Success: nice make test-clean
535 Success: nice make doc-clean
536 Success: nice make clean
537 Success: git reset --hard
540 Issue 4008: Testing patch issue115770043_1_diff
541 Success: git apply --index /home/joe/lilypond-git/issue115770043_1.diff
542 Success: ./autogen.sh --noconfigure
548 @subsubsubheading Checking the regression test results
550 Assuming the patch passed all the @code{make} tests, the regression
551 differences will be located in the @file{/test-results/} directory
552 within the build location for the patch issue number;
555 /tmp/show-4007/test-results/
558 Open @file{index.html} in a browser of your choice to view any
562 Alternatively if the Firefox browser is installed, then the regression
563 test results can be opened by calling the appropriate
564 @file{show-regtests-} file located in the auto-compile log location;
567 sh ~/lilypond-auto-compile-results/show-regtests-4007
570 See @ref{Regression tests}.
572 @subsubsubheading Reporting test results
574 Once a patch has been tested and the regression tests have been
575 manually checked, the tracker can be updated manually by editing the
576 tracker issue directly in the web browser or by using two additional
577 python scripts that are included as part of the Patchy suite.
579 @unnumberedsubsubsec For patches that have passed
581 Use the @code{accept-patch.py} script and run it with the Google issue
582 tracker number (not the Rietveld issue number) as an argument;
588 This will automatically update the tracker issue with the phrase
589 @qq{Patchy the autobot says: passes tests.}.
592 It is also possible to add additional information to the default
593 message by adding a second argument within double-quote marks.
596 accept-patch.py 4007 "This also includes a full documentation build."
599 The tracker issue's label is then changed automatically to
602 @unnumberedsubsubsec Patches that have failed
604 Use the @code{reject-patch.py} script and run it with the Google issue
605 tracker number (not the Rietveld issue number) as an argument but you
606 @emph{must} also include a second argument, in double-quotes, stating
607 the reason the patch has been rejected;
610 reject-patch.py 4007 "Fails the 'make check' test."
613 Once the @code{reject-patch.py} script has been run, the tracker issue's
614 label is changed automatically to @qq{Patch-Needs_work}.
617 @node Automating Patchy
618 @subsection Automating Patchy
620 To run as a cron job make sure you have;
624 notify_non_action = no
627 in @file{$HOME/.lilypond-patchy-config} to avoid any unintentional email
630 Assuming that Patchy run a user @qq{patchy}, create a file called
631 @file{$HOME/lilypond-patchy.cron}, adapting it as necessary (the
632 @code{/2} means @qq{run this every 2 hours}):
635 02 0-23/2 * * * /home/patchy/lilypond-extra/patches/lilypond-patchy-staging.py
638 @warning{@code{cron} will not inherit environment variables so you must
639 re-define any variables inside @file{$HOME/lilypond-patchy.cron}. For
640 instance, @var{LILYPOND_GIT} may need to be defined if
641 @var{git_repository_dir} is not correctly set in
642 @file{$HOME/.lilypond-patchy-config}.}
644 Finally, apply the cron job (you may need superuser privileges for
648 crontab -u patchy /home/patchy/lilypond-patchy.cron
652 @node Troubleshooting Patchy
653 @subsection Troubleshooting Patchy
655 The following is a list of the most common messages that the scripts
656 may report with explanations.
668 There are currently no tracker issues with the @code{Patch-New} status.
671 If specific tracker issue number has been used as an argument when
672 running @code{test-patches.py}, then the issue contains no URL to
679 this Git revision has already been pushed by an operator other than this Patchy.
685 Another, remote, machine has already tested and pushed the new commits
689 You may also see this on your local machine if the auto-build files
690 have been deleted and this computer has previously already pushed the
691 listed commit ID to @code{master}.
699 Git revision has not changed but checksum of test baseline has, must rebuild.
702 This occurs when Patchy detects that the commit ID has not changed
703 since the last test but it cannot locate the last
704 @code{make test-baseline} (usually because it has been deleted or moved)
705 and so a new @code{test-baseline} is rebuilt.
709 Last patch for issue xxxx already tested or under testing
710 by another Patchy instance, skipping.
716 There is another instance of Patchy running on your computer that is
717 testing the same tracker issue.
720 A previous test attempt was unsuccessful for some reason and the scripts
721 were not able to tidy up after themselves (for example if you manually
722 halt the testing process by killing it or closing the terminal you may
723 have been running the script in).
726 There is a hidden file located in the @code{$HOME} directory of the user
727 running Patchy called @code{.lilypond-patchy-cache} that records the
728 current patches that are being tested, have been tested and the commit
729 ID of @code{remote/origin/master} since the last test. It will contain
734 issue105560044_120001_diff = testing
737 for any issue that it thinks is still in the process of being tested.
740 Manually remove this entry and re-run the script.
746 test-master-lock and PID entry exist but previous Patchy
747 run (PID xxxxx) died, resetting test-master-lock anyway.
751 A previous test attempt was unsuccessful for some reason and the scripts
752 were not able to tidy up after themselves (for example if you manually
753 halt the testing process by killing it or closing the terminal you may
754 have been running the script in). The @code{test-master-lock} branch
755 was therefore not able to be deleted cleanly however, nothing needs to
756 be done the scripts will rebuild any tests it needs to.
760 fatal: A branch named 'test-master-lock' already exists.
766 There is another instance of Patchy running on your computer that is
767 testing the same tracker issue.
770 A previous test attempt was unsuccessful for some reason and the scripts
771 were not able to tidy up after themselves (for example if you manually
772 halt the testing process by killing it or closing the terminal you may
773 have been running the script in). The @code{test-master-lock} branch
774 was therefore not able to be deleted cleanly, in this case you must
775 manually delete the @code{test-master-lock} branch in your
776 @code{$LILYPOND_GIT} directory.
779 git branch -d test-master-lock
783 It may be wise to also manually delete @code{test-master} and
784 @code{test-staging} too, just to be safe.
792 Another instance (PID xxxxx) is already running.
796 This occurs when trying to run @code{lilypond-patchy-staging.py} when
797 another instance of either script is already running locally.
801 Warning: something went wrong; omitting patch for issue 3976
807 The Rietveld URL as listed in the tracker is incorrect (e.g. missing or
808 incorrect issue number
811 The patch on Rietveld is too large to download
818 @node Administrative mailing list
819 @section Administrative mailing list
821 A mailing list for administrative issues is maintained at
822 @code{lilypond-hackers@@gnu.org}.
824 This list is intended to be used for discussions that should be kept
825 private. Therefore, the archives are closed to the public.
827 Subscription to this list is limited to certain senior developers.
829 At the present time, the list is dormant.
831 Details about the criteria for membership, the types of discussion
832 to take place on the list, and other policies for the hackers list
833 will be finalized during the
834 @ref{Grand Organization Project (GOP)}.
838 @node Grand Organization Project (GOP)
839 @section Grand Organization Project (GOP)
845 Clarify the various development tasks by writing down the policies
846 and techniques and/or simplifying the tasks directly.
849 Get more people involved in development: specifically, find people
850 to do easy tasks to allow advanced developers to concentrate on
859 * Policy decisions (finished)::
863 @subsection Motivation
865 Most readers are probably familiar with the LilyPond Grand
866 Documentation Project, which ran from Aug 2007 to Aug 2008. This
867 project involved over 20 people and resulted in an almost complete
868 rewrite of the documentation. Most of those contributors were
869 normal users who decided to volunteer their time and effort to
870 improve lilypond for everybody. By any measure, it was a great
873 The Grand Organization Project aims to do the same thing with a
874 larger scope -- instead of focusing purely on documentation, the
875 project aims to improve all parts of LilyPond and its community.
876 Just as with GDP, the main goal is to encourage and train users to
877 become more involved.
879 If you have never contributed to an open-source project before --
880 especially if you use Windows or OSX and do not know how to
881 program or compile programs -- you may be wondering if there's
882 anything you can do. Rest assured that you @emph{can} help.
884 @subheading "Trickle-up" development
886 One of the reasons I'm organizing GOP is "trickle-up"
887 development. The idea is this: doing easy tasks frees up advanced
888 developers to do harder tasks. Don't ask "am I the @emph{best}
889 person for this job"; instead, ask "am I @emph{capable} of doing
890 this job, so that the current person can do stuff I @emph{can't}
893 For example, consider lilypond's poor handling of grace notes in
894 conjunction with clef and tempo changes. Fixing this will require
895 a fair amount of code rewriting, and would take an advanced
896 developer a few weeks to do. It's clearly beyond the scope of a
897 normal user, so we might as well sit back and do nothing, right?
899 No; we @emph{can} help, indirectly. Suppose that our normal user
900 starts answering more emails on lilypond-user. This in turn means
901 that documentation writers don't need to answer those emails, so
902 they can spend more time improving the docs. I've noticed that all
903 doc writers tackle harder and harder subjects, and when they start
904 writing docs on scheme programming and advanced tweaks, they start
905 contributing bug fixes to lilypond. Having people performing these
906 easy-to-moderate bug fixes frees up the advanced developers to
907 work on the really hard stuff... like rewriting the grace note
910 Having 1 more normal user answering emails on lilypond-user won't
911 have a dramatic @q{trickle-up} effect all by itself, of course. But if
912 we had 8 users volunteering to answer emails, 6 users starting to
913 write documentation, and 2 users editing LSR... well, that would
914 free up a lot of current bug-fixing-capable contributors to focus
915 on that, and we could start to make a real dent in the number of
916 bugs in lilypond. Quite apart from the eased workload, having that
917 many new helpers will provide a great moral boost!
920 @subsection Ongoing jobs
922 Although GOP is a short-term project, the main goal is to train
923 more people to handle ongoing jobs. The more people doing these
924 jobs, the lighter the work will be, and the more we can get done
927 Also, it would be nice if we had at least one "replacement" /
928 "understudy" for each role -- too many tasks are only being done
929 by one person, so if that person goes on vacation or gets very
930 busy with other matters, work in that area grinds to a halt.
932 @subheading Jobs for normal users
936 LilyPond is sometimes critized for not listening to users, but
937 whenever we ask for opinions about specific issues, we never get
938 enough feedback. This is somewhat aggravating.
939 We need a group of users to make a dedicated effort to test and
940 give feedback. If there's new documentation, read it. If there's
941 an experimental binary, download it and try compiling a score with
942 it. If we're trying to name a new command, think about it and give
945 @item lilypond-user support:
946 I think it would be nice if we had an official team of users
949 @item LilyPond Report:
950 Keeping a monthly newsletter running is a non-trivial task. A lot
951 of work is needed to organize it; it would be great if we could
952 split up the work. One person could write the Snippet of the
953 Month, another person could do Quotes of the Month, another person
954 could do interviews, etc.
957 Although GDP (the Grand Documentation Project) did great work,
958 there's still many tasks remaining.
961 Keeping the documentation translations is a monumental task; we
962 need all the help we can get!
966 @subheading Jobs for advanced users for developers
969 @item Git help for writers:
970 We often receive reports of typos and minor text updates to the
971 documentation. It would be great if somebody could create
972 properly-formatted patches for these corrections.
974 Technical requirements: ability to run @ref{LilyDev}.
977 LSR contains many useful examples of lilypond, but some snippets
978 are out of date and need updating. Other snippets need to be
979 advertized, and new snippets need to be sorted. We could use
980 another person to handle LSR.
982 Technical requirements: use of a web browser. LilyPond
983 requirements: you should be familiar with most of Notation
984 chapters 1 and 2 (or be willing to read the docs to find out).
986 @item Join the Frogs:
987 "Frogs" are a team of bug-fixers (because frogs eat bugs, and you
988 often find them in Ponds of Lilies) and new feature implementors.
990 Technical requirements: development environment (such as
991 @ref{LilyDev}), ability to read+write scheme and/or C++ code.
996 @node Policy decisions
997 @subsection Policy decisions
999 There are a number of policy decisions -- some of them fairly
1000 important -- which we have been postponing for a few years. We
1001 are now discussing them slowly and thoroughly; agenda and exact
1002 proposals are online:
1005 @uref{http://lilypond.org/~graham/gop/index.html}
1008 Below is a list of policies which are not @qq{on the agenda} yet.
1010 Note that the presence of an item on this list does @emph{not}
1011 mean that everybody thinks that something needs to be done.
1012 Inclusion in this simply means that one developer thinks that we
1013 should discuss it. We are not going to filter this list; if any
1014 developer thinks we should discuss something, just add it to the
1015 bottom of the list. (the list is unsorted)
1017 As GOP progresses, items from this list will be put on the agenda
1018 and removed from this list. I generally try to have one month's
1019 discussion planned in advance, but I may shuffle things around to
1020 respond to any immediate problems in the developer community.
1022 There are some item(s) not displayed here; these are questions
1023 that were posed to me privately, and I do not feel justified in
1024 discussing them publicly without the consent of the person(s) that
1025 brought them up. They will initially be discussed privately on the
1026 lilypond-hackers mailing list -- but the first question will be
1027 "do we absolutely need to do this privately", and if not, the
1028 discussion will take place on lilypond-devel like the other items.
1030 In most policy discussions in lilypond over the past few years,
1031 the first half (or more) is wasted arguing on the basis of
1032 incorrect or incomplete data; once all the relevant facts are
1033 brought to light, the argument is generally resolved fairly
1034 quickly. In order to keep the GOP discussions focused, each topic
1035 will be introduced with a collection of relevant facts and/or
1036 proposals. It is, of course, impossible to predict exactly which
1037 facts will be relevant to the discussion -- but spending an hour
1038 or two collecting information could still save hours of
1041 @warning{The estimated time required for "prep work", and the
1042 following discussion, has been added to each item. At the moment,
1043 there is an estimated 30 hours of prep work and 140 hours of
1047 @item @strong{Patch reviewing}:
1048 At the time of this writing, we have 23 (known) patches waiting
1049 for review. Some from main developers; some from new developers.
1050 We desperately need more people helping with lilypond, but
1051 ignoring patches is the best way to drive potential contributors
1052 away. This is not good.
1054 (prep: 2 hours. discuss: 10 hours)
1056 @item @strong{Official links to other organizations?}:
1057 There's something called the "software freedom conservancy", and
1058 in general, there's a bunch of "umbrella organizations". Joining
1059 some of these might give us more visibility, possibly leading to
1060 more users, more developers, maybe even financial grants or use in
1063 (prep: 2 hours. discuss: 5 hours)
1065 @item @strong{Issue tracking with google code}:
1066 We use the google issue tracker, but this means that we are
1067 relying on a commercial entity for a large part of our
1068 development. Would it be better (safer in the long run) to use the
1069 savannah bug tracker?
1071 (prep: 1 hour. discuss: 5 hours)
1073 @item @strong{Patch review tool}:
1074 Reitveld is inconvenient in some respects: it requires a google
1075 account, and there's no way to see all patches relating to
1076 lilypond. Should we switch to something like gerritt?
1077 @uref{http://code.google.com/p/lilypond/issues/detail?id=1184}
1079 (prep: 5 hours. discuss: 15 hours)
1081 @item @strong{Clarity for sponsorships}:
1082 We currently do not advertize bounties and sponsorships on the
1083 webpage. How much advertising do we want, and what type?
1084 Should we change the "structure" / "framework" for bounties?
1086 (prep: 2 hours. discuss: 10 hours)
1088 @item @strong{code readability}:
1089 "Our aim when producing source code for Lilypond in whatever
1090 language is that it should be totally comprehensible to a
1091 relatively inexperienced developer at the second reading."
1094 - aids maintainability of code base
1095 - "second reading" so newer developers can look up unfamiliar
1097 - will help to keep things simple, even if the code is doing
1098 complex stuff discourages "secret squirrel" coding, e.g. "how
1099 much functionality can I squeeze into as few characters as
1100 possible" "comments are for wimps"
1101 - will aid not *discouraging* new developers to join the project
1103 (prep: 2 hours. discuss: 10 hours)
1105 @item @strong{C++ vs. scheme}:
1106 what should be in scheme, what should be in C++, what can/should
1107 be ported from one to the other, etc. Questions of
1108 maintainability, speed (especially considering guile 2.0), and the
1109 amount of current material in either form, are important.
1111 (prep: 5 hours. discuss: 15 hours)
1113 @item @strong{always make an issue number for patches}:
1114 there is a proposal that we should always have a google code issue
1115 number for every patch. This proposal is closely tied to our
1116 choice of patch review tool; if we switch to a different tool (as
1117 suggested in a different proposal), this proposal may become moot.
1119 (prep: 1 hour. discuss: 5 hours)
1121 @item @strong{initalizer lists}:
1122 shoudl we use initalizer lists for C++? AFAIK they make no
1123 difference for built-in types, but there's some weird case where
1124 it's more efficient for objects, or something.
1126 Probably not worth making this a weekly thing on its own, but we
1127 can probably wrap it up with some other code-related questions.
1129 (prep: 15 minutes. discuss: 3 hours)
1133 @node Policy decisions (finished)
1134 @subsection Policy decisions (finished)
1136 Here is a record the final decisions, along with links to the
1140 * GOP-PROP 1 - python formatting::
1141 * GOP-PROP 2 - mentors and frogs::
1142 * GOP-PROP 3 - C++ formatting::
1143 * GOP-PROP 4 - lessons from 2.14::
1144 * GOP-PROP 5 - build system output (not accepted)::
1145 * GOP-PROP 6 - private mailing lists::
1146 * GOP-PROP 7 - developers as resources::
1147 * GOP-PROP 8 - issue priorities::
1148 * GOP-PROP 9 - behavior of make doc::
1151 @node GOP-PROP 1 - python formatting
1152 @subsubsection GOP-PROP 1 - python formatting
1154 We will follow the indentation described in PEP-8.
1155 @uref{http://www.python.org/dev/peps/pep-0008/}
1159 use 4 spaces per indentation level
1162 never mix tabs and spaces (for indentation)
1165 Code indented with a mixture of tabs and spaces should be
1166 converted to using spaces exclusively
1168 Once this is done, we should add @code{python -tt} to the build
1169 system to avoid such errors in the future.
1173 There should be absolutely no tab characters for indentation in
1174 any @code{.py} file in lilypond git. All such files should be
1175 converted to use spaces only.
1177 @subsubheading Discussions
1180 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00060.html}
1181 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00084.html}
1182 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00310.html}
1183 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00574.html}
1187 @node GOP-PROP 2 - mentors and frogs
1188 @subsubsection GOP-PROP 2 - mentors and frogs
1190 Nothing much was decided. The list of responsibilities was
1191 slightly altered; see the new one in @ref{Mentors}. We should
1192 encourage more use of the Frogs mailing list. There's a list of
1193 contributor-mentor pairs in:
1196 @uref{https://github.com/gperciva/lilypond-extra/blob/master/people/mentors.txt}
1199 That's pretty much it.
1201 @subsubheading Discussions
1204 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00311.html}
1210 @node GOP-PROP 3 - C++ formatting
1211 @subsubsection GOP-PROP 3 - C++ formatting
1213 Speaking academically, C++ code style is a "solved problem". Let's
1214 pick one of the existing solutions, and let a computer deal with
1215 this. Humans should not waste their time, energy, and creativity
1216 manually adding tabs or spaces to source code.
1218 We have modified @code{fixcc.py} to use astyle, along with extra
1223 the final script will be run @strong{blindly} on the lilypond
1224 source code. We will accept whatever formatting the final version
1225 of this script produces, with no manual tweaking.
1228 patches which have been run through this tool will not be rejected
1229 for style reasons. Any code formatting @qq{desires} which are not
1230 enforced by @code{fixcc.py} will not be considered grounds for
1234 for now, this style will not be enforced. It is not cause for
1235 concern if patches which do not follow the formatting done by
1236 @code{fixcc.py} are pushed. From time to time, Graham will run
1237 the formatter on the entire code base, and commit the resulting
1240 In a few months, we will tighten up this policy item (with some
1241 sort of automatic processing), but that is outside the scope of
1242 this policy item and is a matter for later discussion.
1245 after the proposal is accepted, we will leave some time for
1246 existing patches to be accepted and pushed. The script was
1247 run on the source code on @strong{2011 August 01}.
1251 @subheading GNU code
1253 LilyPond is a GNU project, so it makes sense to follow the GNU
1254 coding standards. These standards state:
1257 We don’t think of these recommendations as requirements, because
1258 it causes no problems for users if two different programs have
1259 different formatting styles.
1261 But whatever style you use, please use it consistently, since a
1262 mixture of styles within one program tends to look ugly. If you
1263 are contributing changes to an existing program, please follow the
1264 style of that program.
1267 (@uref{http://www.gnu.org/prep/standards/html_node/Formatting.html})
1269 With that in mind, we do not think that we must blindly follow the
1270 formatting given by the currrent version of Emacs.
1272 @subheading Implementation notes
1274 We can avoid some of the style change pollution in git history by
1275 ignoring whitespaces changes:
1281 @subsubheading Discussions
1284 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00526.html}
1285 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00796.html}
1286 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00200.html}
1287 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00525.html}
1288 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00751.html}
1289 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00751.html}
1293 @node GOP-PROP 4 - lessons from 2.14
1294 @subsubsection GOP-PROP 4 - lessons from 2.14
1298 A brief history of releases:
1300 @multitable @columnfractions .2 .2 .3
1301 @headitem date (YYYY-MM-DD) @tab version @tab comment
1302 @item 2008-10-28 @tab 2.11.63 @tab nobody checking regtests
1303 @item 2008-11-17 @tab 2.11.64
1304 @item 2008-11-29 @tab 2.11.65
1305 @item 2008-12-23 @tab 2.12.0
1306 @item 2009-01-01 @tab @tab somewhere around here, Graham becomes
1307 officially release manager, but Han-Wen still builds the actual
1309 @item 2009-01-01 @tab 2.12.1
1310 @item 2009-01-25 @tab 2.12.2
1311 @item 2009-02-28 @tab 2.13.0
1312 @item 2009-06-01 @tab 2.13.1 @tab note jump in time!
1313 @item 2009-06-27 @tab 2.13.2 @tab first Graham release?
1314 @item 2009-07-03 @tab 2.13.3
1315 @item 2009-09-09 @tab @tab Graham arrives in Glasgow, gets a
1316 powerful desktop computer, and begins serious work on GUB (sending
1317 bug reports to Jan). It takes approximately 100 hours until GUB
1318 is stable enough to make regular releases.
1319 @item 2009-09-24 @tab 2.13.4
1320 @item 2009-10-02 @tab 2.13.5
1321 @item 2009-10-22 @tab 2.13.6
1322 @item 2009-11-05 @tab 2.13.7
1324 @item 2010-01-13 @tab 2.12.3
1326 @item 2010-03-19 @tab 2.13.16 @tab Bug squad starts doing a few
1327 regtest comparisons, but IIRC the effort dies out after a few
1330 @item 2010-08-04 @tab 2.13.29 @tab Phil starts checking regtests (BLUE)
1332 @item 2011-01-12 @tab 2.13.46 @tab release candidate 1 (GREEN)
1334 @item 2011-05-30 @tab 2.13.63 @tab release candidate 7 (GREEN)
1335 @item 2011-06-06 @tab 2.14.0
1338 @c A graphical display of bugs:
1340 @c @image{bugs-2.13-visualization,png}
1341 @c @image{zoom-2.13-visualization,png}
1343 @subheading Carl's analysis of the bugs
1345 A @file{csv} spreadsheet is available.
1348 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00852.html}
1352 @uref{lilypond-issues-analysis.csv}
1353 @uref{lilypond-issues-analysis-trim-duplicates.csv}
1356 There 148 issues marked with Priority=Critical in the tracker.
1358 I've done an analysis, and it looks to me like there was initially
1359 a backlog of critical issues that weren't fixed, and little work
1360 was being done to eliminate critical issues.
1362 Somewhere about 2010-08-01, critical issues started to disappear,
1363 but occasional new ones appeared.
1365 There were a couple of major changes that introduced unanticipated
1366 regressions (new spacing code, beam collision avoidance). These
1367 produced more than the expected number of regressions.
1369 It appears to me that we didn't really get serious about
1370 eliminating critical bugs until about 2010-06-15 or so. After
1371 that point, the number of critical bugs more-or-less steadily
1372 decreased until we got to a release candidate.
1374 Of particular interest, the first release candidate of 2.14 was
1375 released on 2011-01-12. Over the next 10 days, about a dozen bugs
1376 were reported and fixed. Release candidate 2 came out on
1377 2011-02-09. No surge of bugs occurred with this release.
1378 Candidate 3 came out on 2011-03-13; we got 2 bugs per week.
1379 Candidate 4 came out on 2011-03-29; 2 new bugs. Candidate 6 came
1380 out on 2011-04-07. We got a couple of bugs per week.
1382 @subheading Notes, commentary, and opinions
1385 Han-Wen: Overall, I think this cycle took too long
1390 @subsubheading Discussions
1393 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00797.html}
1394 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00364.html}
1399 @node GOP-PROP 5 - build system output (not accepted)
1400 @subsubsection GOP-PROP 5 - build system output (not accepted)
1402 This proposal was too broad; after a month of discussion, Graham
1403 withdrew the proposal. Portions of it will be introduced in later
1406 @subsubheading Discussions
1409 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00320.html}
1410 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00527.html}
1411 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00753.html}
1412 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg01042.html}
1413 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00116.html}
1414 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00310.html}
1418 @node GOP-PROP 6 - private mailing lists
1419 @subsubsection GOP-PROP 6 - private mailing list
1421 Potentially sensitive or private matters will be referred to
1422 Graham. He will then decide who should discuss the matter on an
1423 ad-hoc basis, and forward or CC them on future emails.
1425 For emphasis, the project administrators are Han-Wen, Jan, and
1426 Graham; those three will always be CC'd on any important
1429 The lilypond-hackers mailing list will be removed.
1433 There is some unhappy history about this idea in our development
1437 @uref{http://lists.gnu.org/archive/html/lilypond-devel/2010-09/msg00178.html}
1438 @uref{http://news.lilynet.net/spip.php?article121}
1439 @uref{http://lists.gnu.org/archive/html/lilypond-devel/2010-11/msg00076.html}
1442 @subheading Other projects
1444 The idea of private mailing lists is hardly uncommon in
1445 open-source software. For example,
1448 @uref{http://lwn.net/Articles/394660/} about debian-private
1449 @uref{http://subversion.apache.org/mailing-lists.html} private@@
1450 @uref{http://www.freebsd.org/administration.html#t-core}
1451 @uref{http://foundation.gnome.org/legal/} board members pledge
1452 to keep certain matters confidential
1454 every security team of every GNU/Linux distribution and OS
1457 In fact, Karl Fogel's @qq{Producing Open Source Software}
1458 explicitly suggests a private mailing list for some circumstances:
1461 [on granting commit/push access to a contributor]
1463 But here is one of the rare instances where secrecy is
1464 appropriate. You can't have votes about potential committers
1465 posted to a public mailing list, because the candidate's feelings
1466 (and reputation) could be hurt.
1468 @uref{http://producingoss.com/en/consensus-democracy.html#electorate}
1471 @subheading Board of governers, voting, etc?
1473 Many projects have an official board of directors, or a list of
1474 @qq{core developers}, with set term limits and elections and
1477 I don't think that we're that big. I think we're still small
1478 enough, and there's enough trust and consensus decisions, that we
1479 can avoid that. I would rather that we kept on going with
1480 trust+consensus for at least the next 2-3 years, and spent more
1481 time+energy on bug fixes and new features instead of
1482 administrative stuff.
1484 Project administrators are Han-Wen, Jan, and Graham.
1486 @subsubheading Discussions
1489 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00783.html}
1490 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg01004.html}
1491 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00117.html}
1495 @node GOP-PROP 7 - developers as resources
1496 @subsubsection GOP-PROP 7 - developers as resources
1498 We shall treat developers (and contributors) as
1499 @strong{Independent volunteers}: each person does whatever they
1500 want, whenever they want. We have busy careers and lives; we make
1501 no expectations of action from anybody (with the exception of the
1502 6 people in @qq{Meister} positions).
1504 @subsubheading Discussions
1507 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg01092.html}
1508 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00087.html}
1509 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00497.html}
1513 @node GOP-PROP 8 - issue priorities
1514 @subsubsection GOP-PROP 8 - issue priorities
1516 We will delete the @qq{priority} field of the issue tracker
1517 altogether. The @qq{type} system will be tweaked.
1524 a reproducible failure to build either @code{make} or @code{make
1525 doc}, from an empty build tree, in a first run, if
1526 @code{configure} does not report any errors.
1529 any program behaviour which is @strong{unintentionally} worse than
1530 the previous stable version or the current development version.
1531 Developers may always use the @qq{this is intentional}, or even
1532 the @qq{this is an unavoidable effect of an improvement in another
1533 area}, reason to move this to a different type.
1536 anything which stops contributors from helping out (e.g.
1537 lily-git.tcl not working, source tree(s) not being available,
1538 LilyDev being unable to compile git master, inaccurate
1539 instructions in the Contributor's Guide 2 Quick start).
1541 To limit this scope of this point, we will assume that the
1542 contributor is using the latest LilyDev and has read the relevant
1543 part(s) of the Contributor's Guide. Problems in other chapters of
1544 the CG are not sufficient to qualify as Type-Critical.
1548 @subsubheading More new/changed types and labels
1550 Unless otherwise specified, the current types and labels will
1551 continue to be used. The new types introduced by this proposal
1557 Type-crash: any segfault, regardless of what the input file looks
1558 like or which options are given. Disclaimer: this might not be
1559 possible in some cases, for example certain guile programs (we
1560 certainly can't predict if a piece of scheme will ever stop
1561 running, i.e. the halting problem), or if we rely on other
1562 programs (i.e. ghostscript). If there are any such cases that
1563 make segfault-prevention impossible, we will document those
1564 exceptions (and the issue will remain as a "crash" instead of
1565 "documentation" until the warning has been pushed).
1568 Type-maintainability: anything which makes it difficult for
1569 serious contributors to help out (e.g. difficult to find the
1570 relevant source tree(s), confusing policies, problems with
1571 automatic indentation tools, etc).
1574 Type-ugly: replaces Type-collision, and it will include things
1575 like bad slurs in addition to actual collision.
1579 A new label will be added:
1583 (label) Needs_evidence: it is not clear what the correct output
1584 should look like. We need scans, references, examples, etc.
1588 @subheading Reminding users about stars
1590 We can remind users that they can @qq{star} an issue to indicate
1591 that they care about it. Since we resolved to treat developers as
1592 independent volunteers, there is no expectation that anybody will
1593 look at those stars, but if any developer want to organize their
1594 work schedule according to the stars, they are welcome to do so.
1596 @subsubheading Discussions
1599 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00019.html}
1600 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00277.html}
1601 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00413.html}
1602 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00624.html}
1607 @node GOP-PROP 9 - behavior of make doc
1608 @subsubsection GOP-PROP 9 - behavior of make doc
1610 If there are build problems, then it should be easier to find out
1611 why it's failing. This will be achieved with log files, as well
1612 as possibly including scripts which automatically display portions
1613 of those log files for a failing build.
1615 We will also add targets for building a specific manual (for
1616 quick+easy checking of doc work), as well as for building all
1617 documentation in a specific language (either English or a
1618 translated language).
1620 When you run @code{make doc},
1625 All output will be saved to various log files, with the exception
1626 of output directly from @code{make(1)}.
1628 Note that @code{make(1)} refers to a specific executable file on
1629 unix computers, and is not a general term for the build system.
1632 By default, no other output will be displayed on the console, with
1633 one exception: if a build fails, we might display some portion(s)
1634 of log file(s) which give useful clues about the reason for the
1637 The user may optionally request additional output to be printed;
1638 this is controlled with the @code{VERBOSE=x} flag. In such cases,
1639 all output will still be written to log files; the console output
1640 is strictly additional to the log files.
1643 Logfiles from calling lilypond (as part of lilypond-book) will go in
1645 @file{$LILYPOND_BUILD_DIR/out/lybook-db/12/lily-123456.log} file. All
1646 other logfiles will go in the @file{$LILYPOND_BUILD_DIR/logfiles/}
1649 A single @code{make doc} will therefore result in hundreds of log
1650 files. Log files produced from individual lilypond runs are not
1651 under our control; apart from that, I anticipate having one or two
1652 dozen log files. As long as it is clear which log file is
1653 associated with which operation(s), I think this is entirely
1654 appropriate. The precise implementation will be discussed for
1655 specific patches as they appear.
1658 Both stderr and stdout will be saved in @code{*.log}. The order
1659 of lines from these streams should be preserved.
1662 There will be no additional @qq{progress messages} during the
1663 build process. If you run @code{make --silent}, a non-failing
1664 build should print absolutely nothing to the screen.
1667 Assuming that the loglevels patch is accepted, lilypond (inside
1668 lilypond-book) will be run with --loglevel=WARN.
1669 @uref{http://codereview.appspot.com/4822055/}
1672 Ideally, a failing build should provide hints about the reason why
1673 it failed, or at least hints about which log file(s) to examine.
1677 If this proposal is accepted, none of these policies will be
1678 assumed to apply to any other aspect of the build system.
1679 Policies for any other aspect of the build system will be
1680 discussed in separate proposals.
1682 @subheading Don't cause more build problems
1684 However, there is a danger in this approach, that vital error
1685 messages can also be lost, thus preventing the cause of the
1686 failure of a make being found. We therefore need to be
1687 exceptionally careful to move cautiously, include plenty of tests,
1688 and give time for people to experiment/find problems in each stage
1689 before proceeding to the next stage.
1691 This will be done by starting from individual lilypond calls
1692 within lilypond-book, and slowly moving to @qq{larger} targets of
1693 the build system -- after the individual lilypond calls are are
1694 producing the appropriate amount of output and this is saved in
1695 the right place and we can automatically isolate parts of a
1696 failing build, we will work on lilypond-book in general, and only
1697 then will we look at the build system itself.
1699 @subheading Implementation notes
1701 There is an existing make variable QUIET_BUILD, which
1702 alter the amount of output being displayed
1704 http://lilypond.org/doc/v2.15/Documentation/contributor/useful-make-variables}
1705 ). We are not planning on keeping this make variable.
1707 The standard way for GNU packages to give more output is with a
1708 @code{V=x} option. Presumably this is done by increasing
1709 @code{x}? If we support this option, we should still write log
1710 files; we would simply print more of the info in those log files
1713 The command @code{tee} may be useful to write to a file and
1714 display to stdout (in the case of VERBOSE).
1717 @subsubheading Discussions
1720 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00378.html}
1721 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00703.html}
1726 @n ode GOP-PROP 10 - scheme indentation
1727 @s ubsubsection GOP-PROP 10 - scheme indentation
1729 still under discussion
1731 @subsubheading Discussions
1734 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00625.html}
1735 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg01026.html}
1742 @node Grand LilyPond Input Syntax Standardization (GLISS)
1743 @section Grand LilyPond Input Syntax Standardization (GLISS)
1749 Start: sortly after 2.14 comes out, which is currently estimated
1750 to happen in January 2011.
1753 Length: 6-12 months. We're not going to rush this.
1756 Goal: define an input which we commit to being
1757 machine-updateable for the forseeable future. Any future patches
1758 which change the syntax in a non-convert-ly-able format will be
1759 rejected. (subject to the limitations, below)
1760 Once this is finished, we will release lilypond 3.0.
1765 @subheading The Problem
1767 One of the biggest complaints people have with lilypond -- other
1768 than silly thing like "there's no gui" -- is the changing syntax.
1769 Now, inventing a language or standards is difficult. If you set
1770 it in stone too soon, you risk being stuck with decisions which
1771 may limit matters. If you keep on updating the syntax,
1772 interaction with older data (and other programs!) becomes complex.
1774 @subheading Scope and Limitations
1778 tweaks will not be included. Anything with \override, \set,
1779 \overrideProperty, \tweak, \revert, \unset... including even those
1780 command names themselves... is still fair game for NOT_SMART
1784 other than that, everything is on the table. Is it a problem to
1785 have the tagline inside \header? What should the default behavior
1789 we need to get standards for command names. This will help users
1790 remember them, and reduce the options for future names (and
1791 potential renamings later on). \commandOn and \commandOff seem to
1792 work well (should we *always* have an Off command?), but what
1793 about the "command" part? Should it be \nounVerbOn, or
1794 \verbNounOn ? Or \verbNotesWithExtraInformationOn ?
1797 we need standards for the location of commands. Ligature
1798 brackets, I'm looking at you. (non-postfix notation must die!)
1801 this Grand Project doesn't affect whether we have a 2.16 or not.
1802 The main problem will be deciding what to do (with a bit of
1803 messiness anticipated for \tuplet); we should definitely release a
1804 2.16 before merging _any_ of these changes.
1807 we obviously can't /guarantee/ that we'll /never/ make any
1808 non-convert-ly changes in the basic format. But we *can*
1809 guarantee that such changes would force lilypond 4.0, and that we
1810 would only do so for overwhelmingly good reasons.
1814 @subheading Workflow
1818 We're going to have lots and lots of emails flying around. The
1819 vast majority won't really fit into either -devel or -user, so
1820 we'll use a list devoted to syntax issues.
1823 Once we have a serious proposal that gained general acceptance
1824 from the separate syntax mailing list, I'll bring it to -devel.
1825 We're not going to make any changes without discussing it on
1826 -devel, but if we're going to have huge threads about English
1827 grammar and silly ideas, and I don't want to clutter up -devel.
1828 Once whatever chaotic silliness on the syntax list is settled
1829 down, I'll bring the ideas to -devel.
1832 as with GDP, I'll moderate the discussion. Not as with mailist
1833 moderation, but rather by introducing issues at specific times.
1834 We don't want a free-for-all discussion of all parts of the syntax
1835 at once; nothing will get resolved.
1838 Whenever possible, we'll decide on policies at the highest level
1839 of abstraction. For example, consider \numericTimeSignature,
1840 \slurUp, \xNotesOn, \startTextSpan, and \verylongfermata. One of
1841 them starts with the name of the notation first (slur). One has
1842 an abbreviation (x instead of cross). One has the verb at the end
1843 (On), another has it at the beginning (start). The adjective can
1844 come at the beginning (numeric, x) or end (Up). Most are in
1845 camelCase, but one isn't (verylongfermata).
1848 Instead of arguing about each individual command, we'll decide on
1849 abstract questions. Should each command begin the notation-noun,
1850 or the verb? Should all commands be in camelCase, or should we
1851 make everything other than articulations in camelCase but make
1852 articulations all lower-case? Are abbreviations allowed?
1855 Once we've answered such fundamental questions, most of the syntax
1856 should fall into place pretty easily. There might be a few odd
1857 questions left ("is it a span, or a spanner?"), but those can be
1858 settled fairly quickly.
1862 @subheading Implementation
1864 Nothing until the project is finished, then we declare the next
1865 stable release (2.16.0 or 2.18.0 ?) to be the final 2.x version,
1866 release it, then apply all the GLISS syntax changes and start
1867 testing a beta for 3.0 a week or two later.
1869 @subheading Discussion
1871 Don't respond to any of the specifics yet. Yes, we all have our
1872 pet irritations (like "what's up with \paper and \layout?!").
1873 There will be plenty of time to discuss them once GLISS starts.
1875 That said, we have a list of specific items that people really
1876 wanted to have written down. See @ref{Specific GLISS issues}.
1879 * Specific GLISS issues::
1883 @node Specific GLISS issues
1884 @subsection Specific GLISS issues
1888 add regtests for every piece of syntax (not one-command-per-file,
1889 but making a few files which, between them, use every single piece
1890 of syntax.) This is a great test for convert-ly.
1893 should GLISS cover suggested conventions? (indentation,
1894 one-bar-per-line, etc -- the kind of stuff we list for the
1895 lilypond formatting in the docs ?)
1898 how much (if any) syntactic sugar should we add? i.e.
1900 \instrumentName #'foo
1902 \set Staff.instrumentName
1904 ? Carl: maybe yes, Neil: no. (for example, it fails for
1908 the values that are used as arguments to common used overrides.
1909 Sometimes they are a symbol (e.g. #'around), sometimes a
1910 predefined variable referring to a Scheme value or object (e.g.
1911 #LEFT, #all-visible ). The main trouble is that for novice users
1912 it is not clear when there should be an apostrophe and when not.
1915 When do we need -\command and when is it just \command ?
1919 Command-line options to the lilypond binary. -dfoo counts as a
1920 tweak; we won't be trying to pin those down.
1933 If would be pedagogically simpler to realize this difference if
1934 the syntax was separate if you define a context from scratch (as
1935 is the case with \RemoveEmptyStaffContext) or if it's defined by
1936 adding onto an existing context. For example, a syntax like
1940 % Copy the current settings of the Staff context:
1942 % do whatever additional settings
1944 %%% could be used to distinguish from
1946 % Take settings from a variable:
1948 % do whatever additional settings
1954 % Start from scratch:
1963 Capitalization of identifiers: \VoiceOne ?
1968 { music expression } * 4
1970 \repeat unfold 4 { music expression }
1975 @uref{http://lists.gnu.org/archive/html/lilypond-devel/2010-04/msg00467.html}
1979 Personally, I find it easier to understand when there's a repeated
1980 8 in the half-bar position; it's much easier to see that you have
1985 %%% instead of one group of eight:
1990 trivially simple bar-lines:
1994 encourage, allow, or discourage, or disallow?
1997 indentation of \\ inside a @{@} construct.
2001 barline checks at the end of line should be preceded by at least 2
2002 spaces? barline checks should line up if possible (i.e. if you
2003 can use less than 4, 8, X empty spaces before a barline check to
2007 Why doesn't \transpose respect \relative mode?
2011 on \score vs. \new Score
2013 But in the light of a consistent syntax and semantic, I see no
2014 reason (from the users POV) to disallow it. After all, the real
2015 top-level context is a \book @{@}, isn't it, and I don't see a point
2016 in disallowing a \new Score construct just like \new Staff.
2018 From a syntactical POV, I see the following pros for \new Score:
2019 - You can write \with @{ ... @} for every other context but \Score,
2020 which (for consistency) should also work with \new Score.
2021 - When there's a \new Foo Bar, there's also a \context Foo Bar,
2022 which makes the same as a parallel instantiation of all Bar's.
2023 - [Quoting Rune from
2024 @uref{http://www.mail-archive.com/lilypond-devel@@gnu.org/msg14713.html}
2025 "I know that the \score-statement is a syntactical construct,
2026 but I think it would be nice to hide this fact from the users. I
2027 think we could make the use of score-block much more intuitive if
2028 changing the syntax to \new \Score and adding an implicit
2029 sequential-statement to the score."
2034 @uref{http://code.google.com/p/lilypond/issues/detail?id=1322}
2035 about \new vs. \context.
2039 Let users add their own items to the parser? comment 11 on:
2040 @uref{http://code.google.com/p/lilypond/issues/detail?id=1322}
2043 should engravers be pluralized (note_heads_engraver) or not
2044 (note_head_engraver) ?
2047 should we allow numbers in identifier names? Issue:
2048 @uref{http://code.google.com/p/lilypond/issues/detail?id=1670}
2051 should we officially allow accented characters? in general, how
2052 do we feel about utf-8 stuff?
2055 for the sake of completeness/simplicity, what about *disallowing*
2056 the "one-note" form of a music expression? i.e. only allowing
2059 \transpose c d { e1 }
2060 \transpose c d << e1 >>
2069 What should be the officially encouraged way of writing music for
2070 transposing instruments? Maybe it should be simplified?
2071 See http://lists.gnu.org/archive/html/lilypond-user/2011-07/msg00130.html
2076 @node Unsorted policies
2077 @section Unsorted policies
2079 @subsubheading Language-specific mailing lists
2081 A translator can ask for an official lilypond-xy mailing list once
2082 they've finished all @qq{priority 1} translation items.
2084 @subsubheading Performing yearly copyright update (@qq{grand-replace})
2086 At the start of each year, copyright notices for all source files
2087 should be refreshed by running the following command from the top of
2094 Internally, this invokes the script @file{scripts/build/grand-replace.py},
2095 which performs a regular expression substitution for old-year -> new-year
2096 wherever it finds a valid copyright notice.
2098 Note that snapshots of third party files such as @file{texinfo.tex} should
2099 not be included in the automatic update; @file{grand-replace.py} ignores these
2100 files if they are listed in the variable @code{copied_files}.
2103 @subsubheading Push git access
2105 Git access is given out when a contributor has a significant
2106 record of patches being accepted without problems. If existing
2107 developers are tired of pushing patches for a contributor, we'll
2108 discuss giving them push access. Unsolicited requests from
2109 contributors for access will almost always be turned down.