1 @node Administrative policies
2 @chapter Administrative policies
4 This chapter discusses miscellaneous administrative issues which
5 don't fit anywhere else.
8 * Meta-policy for this document::
9 * Environment variables::
11 * Automated testing with Patchy::
12 * Administrative mailing list::
13 * Grand Organization Project (GOP)::
14 * Grand LilyPond Input Syntax Standardization (GLISS)::
18 @node Meta-policy for this document
19 @section Meta-policy for this document
21 The Contributor's Guide as a whole is still a work in progress,
22 but some chapters are much more complete than others. Chapters
23 which are @qq{almost finished} should not have major changes
24 without a discussion on @w{@code{-devel}}; in other chapters, a
25 disorganized @qq{wiki-style dump} of information is encouraged.
27 Do not change (other than spelling mistakes) without discussion:
32 @ref{Introduction to contributing}
35 @ref{Working with source code}
39 Please dump info in an appropriate @@section within these manuals,
40 but discuss any large-scale reorganization:
48 @ref{Documentation work}
54 @ref{Regression tests}
57 @ref{Programming work}
62 Totally disorganized; do whatever the mao you want:
76 @ref{Administrative policies}
81 @node Environment variables
82 @section Environment variables
84 Some maintenance scripts and instructions in this guide rely on
85 the following environment variables. They should be predefined in
86 LilyDev distribution (see @ref{LilyDev}); if you set up your own
87 development environment, you can set them by appending these settings to
88 your @file{~/.bashrc} (or whatever defines your default environment
89 variables for the user account for LilyPond development), then logging
90 out and in (adapt directories to your setup):
93 LILYPOND_GIT=~/lilypond-git
95 LILYPOND_BUILD_DIR=~/lilypond-git/build
96 export LILYPOND_BUILD_DIR
99 The standard build and install procedure (with @code{autogen.sh},
100 @code{configure}, @code{make}, @code{make install}, @code{make doc}
101 @dots{}) does not rely on them.
103 In addition, for working on the website, @code{LILYPOND_WEB_MEDIA_GIT}
104 should be set to the repository lilypond-extra, see
105 @ref{lilypond-extra}.
111 We have four jobs for organizing a team of contributors:
116 Bug Meister: trains new Bug Squad volunteers, organizes who works
117 on which part of their job, checks to make sure that everything is
118 running smoothly, and has final say on our policy for Issues.
123 Doc Meister: trains new doc editors/writers, organizes who works
124 on which part of the job, checks to make sure that everything is
125 running smoothly, and has final say on our policy for
126 Documentation. Also includes LSR work.
131 Translation Meister: trains new translators, updates the
132 translation priority list, and handles merging branches (in both
140 @node Automated testing with Patchy
141 @section Automated testing with Patchy
144 * Overview of Patchy::
145 * Patchy requirements::
146 * Installing Patchy::
147 * Configuring Patchy::
148 * Running the scripts::
149 * Automating Patchy::
150 * Troubleshooting Patchy::
153 @node Overview of Patchy
154 @subsection Overview of Patchy
156 Patchy is a set of Python scripts used for testing patches or testing &
157 pushing new commits added to @code{remote/origin/staging} to
158 @code{remote/origin/master}.
160 No programmatic skill is required to run either of the scripts; although
161 knowledge of compiling LilyPond and its documentation along with
162 understanding how to configure the @var{PATH} environment of your
163 computer is required. See @ref{Working with source code}.
165 The two scripts that are used for each function are:
169 @code{test-patches.py}. This script tests issues labelled as
170 @qq{Patch-New} on the LilyPond issue tracker. Part of the testing
171 process involves running the regression tests, so this script always
172 requires some human intervention in order to visually check for any
173 differences that may be present after a successful test has occurred.
176 @code{lilypond-patchy-staging.py}. This script checks for any new
177 commits in @code{remote/origin/staging}, makes sure that the new HEAD
178 compiles along with all the LilyPond documentation. Then finally
179 pushing to @code{remote/origin/master}. This script can be run and left
180 unattended, requiring no human intervention.
184 Both of the scripts can be run independently of each other and it is not
185 necessary to be able to run both. So if you wanted to contribute to
186 LilyPond development, for example by @emph{just} testing patches then
187 this would still be a helpful contribution to LilyPond's development.
189 Patchy can also be configured to send emails after each successful (or
190 unsuccessful) operation. This is not a requirement and is turned off
193 @c Need to explain in more detail how to set up Patchy for email but
194 @c as I don't use myself it I have no experience - JL
197 @node Patchy requirements
198 @subsection Patchy requirements
200 @unnumberedsubsec Testing new patches
205 A full local copy of the source code. See
206 @ref{Working with source code}.
209 All the software needed for compiling LilyPond @emph{and} the
210 documentation. Although being able to build the full set of LilyPond's
211 manuals is not mandatory for testing (most) patches, part of the patch
212 testing process requires that the regression tests are run and it is
213 this that requires the software normally used for compiling
214 documentation. See @ref{Compiling}.
217 Commit access is @emph{not} required to test patches, but a valid login
218 to @uref{http://code.google.com/} @emph{is}.
223 @unnumberedsubsec Testing & pushing new commits
228 A full local copy of the source code. See
229 @ref{Working with source code}.
232 All the software needed for compiling LilyPond @emph{and} the
233 documentation. Unlike testing patches, being able to build the full set
234 of LilyPond's documentation is required to be able to test & push new
235 commits. See @ref{Compiling}.
238 Commit access @emph{is} required to test and push new commits, but a
239 valid login to @uref{http://code.google.com/} is @emph{not}. See
245 @node Installing Patchy
246 @subsection Installing Patchy
248 The Patchy scripts are not part of the LilyPond code base, but can be
249 downloaded from @uref{https://github.com/gperciva/lilypond-extra/}. The
250 scripts and related Python libraries are all located in the
251 @file{patches/} directory.
253 Alternatively, use @code{git clone};
256 git clone https://github.com/gperciva/lilypond-extra/
259 This makes it simpler to update the scripts if any changes are ever made
260 to them. Finally, add the location of the @file{patches/} directory to
264 @node Configuring Patchy
265 @subsection Configuring Patchy
267 @warning{It is recommended to create a new user on your computer
268 specifically to run the Patchy scripts as a security precaution and that
269 this user should not have any administrative privileges. Also do not
270 set password protection for your ssh key else you will not be able to
271 run the scripts unattended.}
276 Make sure the environment variables @var{LILYPOND_GIT} and
277 @var{LILYPOND_BUILD_DIR} are configured appropriately. See
278 @ref{Environment variables}.
281 To save being prompted for your login and password to
282 @uref{http://code.google.com/} when testing patches. create a
283 @emph{plain-text} file in your Patchy user's @code{$HOME} directory
284 called @code{.lilypond-project-hosting-login} containing your login and
285 password, each on a separate line.
288 joe_smith123@@gmail.com
293 Manually run either the @code{test-patches.py} or
294 @code{lilypond-patchy-staging.py} scripts and when prompted:
297 Warning: using default config; please edit /home/joe/.lilypond-patchy-config
298 Are you sure that you want to continue with the default config? (y/[n])
301 Answer @qq{@code{n}} and press enter.
303 The next time either of the scripts are run they will use the
304 @code{.lilypond-patchy-config} settings copied to your @code{$HOME}
308 Manually edit the @file{.lilypond-patchy-config} file, located in your
309 @code{$HOME} directory to change any of the default settings.
318 All @code{make} operations are run with;
320 extra_make_options = -j3 CPU_COUNT=3
323 See @ref{Saving time with the -j option}
326 A complete build of all the LilyPond documentation is @emph{not}
329 patch_test_build_docs = no
333 Each instance of either a patch test or commit test & push is logged in;
335 auto_compile_results_dir = ~/lilypond-auto-compile-results/
339 Both scripts will perform their build operations in;
341 build_dir = /tmp/lilypond-autobuild/
346 Each completed patch test will also generate its own directory in
347 @file{/tmp/...} labelled with the tracker issue number prefixed by
353 Both the scripts create clones of @code{staging} and @code{master}
354 branches (prefixed with @code{test-}) with a third branch, called
355 @code{test-master-lock} used as a check to protect against two or more
356 instances of Patchy being run locally at the same time.
359 @node Running the scripts
360 @subsection Running the scripts
362 @unnumberedsubsec Testing & pushing new commits
364 @code{lilypond-patchy-staging.py} is run @emph{without} any arguments.
365 It then checks to see if @code{remote/origin/staging} is
366 @qq{further ahead} than @code{remote/origin/master}.
369 If there are no new differences between the two branches since the last
370 run check, the script will report something like this:
373 (UTC) Begin LilyPond compile, previous commit at 4726764cb591f622e7893407db0e7d42bcde90d9
374 Success: No new commits in staging
378 If there are any differences between the two branches since the last
379 run check, (or if the script cannot for any reason, locate the last
380 instance of a commit that it checked) it will report something like
384 (UTC) Begin LilyPond compile, previous commit at 4726764cb591f622e7893407db0e7d42bcde90d9
385 Merged staging, now at: 79e98a773b6570cfa28a15775a9dea3d3e54d6b5
386 Success: ./autogen.sh --noconfigure
387 Success: /tmp/lilypond-autobuild/configure --disable-optimising
391 and proceed with running @code{make}, @code{make test} and a
392 @code{make doc}. Unlike @code{test-patches.py} if all the tests pass,
393 the script then pushes the changes to @code{remote/origin/master}.
397 Success: nice make clean
398 Success: nice make -j7 CPU_COUNT=7
399 Success: nice make test -j7 CPU_COUNT=7
400 Success: nice make doc -j7 CPU_COUNT=7
401 To ssh://joe@@git.sv.gnu.org/srv/git/lilypond.git
402 79e98a7..4726764 test-staging -> master
403 Success: pushed to master
406 @warning{In the case where any of the @code{lilypond-patchy-staging.py}
407 tests fail, do not try to push your own fixes but report the failures to
408 the Developers List <lilypond-devel@@gnu.org> for advice.}
411 @unnumberedsubsec Testing new patches
413 When run without any argument, @code{test-patches.py} will check
414 @uref{http://code.google.com/p/lilypond/issues/list} for all tracker
415 issues that are marked with the label @code{Patch-new}. It then scrapes
416 the issue, looking for the last Rietveld URL entered. It then downloads
417 the patch file and applies it to @code{test-master}.
419 Here is an example where two tracker issues labeled as @code{Patch-new}
426 Found url: http://codereview.appspot.com/112210043
427 Found patch: 4007,/home/joe/lilypond-git/issue112210043_1.diff,
429 Found url: http://codereview.appspot.com/115770043
430 Found patch: 4008,/home/joe/lilypond-git/issue115770043_1.diff,
431 Fetching, cloning, compiling master.
435 If run no tracker items with the @var{Patch-New} label are found it will
442 The script can also be run using the tracker issue number(s) as an
443 argument regardless if the @var{Patch-New} label has been assigned;
452 test-patches.py 4006 3992 4020
455 The script then checks to see if any previously
456 @code{make test-baseline}s have been generated and if the commit ID of
457 @code{remote/origin/master} is different from that previously completed
461 If no previous @code{make test-baseline} test is discovered or if the
462 commit ID of @code{remote/origin/master} has changed, then a new
463 @code{make test-baseline} will run first automatically before the patch
466 This shows when the commit ID has changed:
470 (UTC) Begin LilyPond compile, previous commit at 3f92dcb2c81dcd2755542b57a0a5f2039f29a211
471 Merged master, now at: 4726764cb591f622e7893407db0e7d42bcde90d9
472 Success: ./autogen.sh --noconfigure
473 Success: /tmp/lilypond-autobuild/configure --disable-optimising
474 Success: nice make clean
475 Success: nice make -j3 CPU_COUNT=3
476 Success: nice make test-baseline -j3 CPU_COUNT=3
477 Success: nice make doc -j3 CPU_COUNT=3
478 Success: nice make doc-clean
483 If a previous regression test @emph{is} discovered @emph{and} if the
484 commit ID of @code{remote/origin/master} has not changed, then the patch
485 will be tested against the previous @code{make test-baseline} without
486 the need to re-generate a new one:
492 Found url: http://codereview.appspot.com/110540043
493 Found patch: 4009,/home/joe/lilypond-extra/patches/issue110540043_1.diff,
494 Fetching, cloning, compiling master.
495 (UTC) Begin LilyPond compile, previous commit at 4726764cb591f622e7893407db0e7d42bcde90d9
496 Success: No new commits in master
497 Using test baseline from previous build.
501 The patch is then applied and a @code{make} and @code{make check} are
502 run. A full @code{make doc} is also run if the
503 @file{.lilypond-patchy-config} file has been edited accordingly;
508 Issue 4009: Testing patch issue110540043_1.diff
509 Success: git apply --index /home/joe/lilypond-git/issue112210043_1.diff
510 Success: ./autogen.sh --noconfigure
511 Success: /tmp/lilypond-autobuild/configure --disable-optimising
512 Success: nice make clean
513 Success: nice make -j3 CPU_COUNT=3
514 Success: nice make check -j3 CPU_COUNT=3
515 Success: nice make doc -j3 CPU_COUNT=3
519 Once all the tests have run (successfully or not), the script will clean
520 up from the previous patch and, if required, start testing the next
525 Issue 4007: Cleaning up
526 Success: nice make test-clean
527 Success: nice make doc-clean
528 Success: nice make clean
529 Success: git reset --hard
532 Issue 4008: Testing patch issue115770043_1_diff
533 Success: git apply --index /home/joe/lilypond-git/issue115770043_1.diff
534 Success: ./autogen.sh --noconfigure
540 @unnumberedsubsubsec Checking the regression test results
542 Assuming the patch passed all the @code{make} tests, the regression
543 differences will be located in the @file{/test-results/} directory
544 within the build location for the patch issue number;
547 /tmp/show-4007/test-results/
550 Open @file{index.html} in a browser of your choice to view any
554 Alternatively if the Firefox browser is installed, then the regression
555 test results can be opened by calling the appropriate
556 @file{show-regtests-} file located in the auto-compile log location;
559 sh ~/lilypond-auto-compile-results/show-regtests-4007
562 See @ref{Regression tests}.
564 @unnumberedsubsec Reporting test results
566 Once a patch has been tested and the regression tests have been
567 manually checked, the tracker can be updated manually by editing the
568 tracker issue directly in the web browser or by using two additional
569 python scripts that are included as part of the Patchy suite.
571 @unnumberedsubsubsec For patches that have passed
573 Use the @code{accept-patch.py} script and run it with the Google issue
574 tracker number (not the Rietveld issue number) as an argument;
580 This will automatically update the tracker issue with the phrase
581 @qq{Patchy the autobot says: passes tests.}.
584 It is also possible to add additional information to the default
585 message by adding a second argument within double-quote marks.
588 accept-patch.py 4007 "This also includes a full documentation build."
591 The tracker issue's label is then changed automatically to
594 @unnumberedsubsubsec Patches that have failed
596 Use the @code{reject-patch.py} script and run it with the Google issue
597 tracker number (not the Rietveld issue number) as an argument but you
598 @emph{must} also include a second argument, in double-quotes, stating
599 the reason the patch has been rejected;
602 reject-patch.py 4007 "Fails the 'make check' test."
605 Once the @code{reject-patch.py} script has been run, the tracker issue's
606 label is changed automatically to @qq{Patch-Needs_work}.
609 @node Automating Patchy
610 @subsection Automating Patchy
612 To run as a cron job make sure you have;
616 notify_non_action = no
619 in @file{$HOME/.lilypond-patchy-config} to avoid any unintentional email
622 Assuming that Patchy run a user @qq{patchy}, create a file called
623 @file{$HOME/lilypond-patchy.cron}, adapting it as necessary (the
624 @code{/2} means @qq{run this every 2 hours}):
627 02 0-23/2 * * * /home/patchy/lilypond-extra/patches/lilypond-patchy-staging.py
630 @warning{@code{cron} will not inherit environment variables so you must
631 re-define any variables inside @file{$HOME/lilypond-patchy.cron}. For
632 instance, @var{LILYPOND_GIT} may need to be defined if
633 @var{git_repository_dir} is not correctly set in
634 @file{$HOME/.lilypond-patchy-config}.}
636 Finally, apply the cron job (you may need superuser privileges for
640 crontab -u patchy /home/patchy/lilypond-patchy.cron
644 @node Troubleshooting Patchy
645 @subsection Troubleshooting Patchy
647 The following is a list of the most common messages that the scripts
648 may report with explanations.
660 There are currently no tracker issues with the @code{Patch-New} status.
663 If specific tracker issue number has been used as an argument when
664 running @code{test-patches.py}, then the issue contains no URL to
671 this Git revision has already been pushed by an operator other than this Patchy.
677 Another, remote, machine has already tested and pushed the new commits
681 You may also see this on your local machine if the auto-build files
682 have been deleted and this computer has previously already pushed the
683 listed commit ID to @code{master}.
691 Git revision has not changed but checksum of test baseline has, must rebuild.
694 This occurs when Patchy detects that the commit ID has not changed
695 since the last test but it cannot locate the last
696 @code{make test-baseline} (usually because it has been deleted or moved)
697 and so a new @code{test-baseline} is rebuilt.
701 Last patch for issue xxxx already tested or under testing
702 by another Patchy instance, skipping.
708 There is another instance of Patchy running on your computer that is
709 testing the same tracker issue.
712 A previous test attempt was unsuccessful for some reason and the scripts
713 were not able to tidy up after themselves (for example if you manually
714 halt the testing process by killing it or closing the terminal you may
715 have been running the script in).
718 There is a hidden file located in the @code{$HOME} directory of the user
719 running Patchy called @code{.lilypond-patchy-cache} that records the
720 current patches that are being tested, have been tested and the commit
721 ID of @code{remote/origin/master} since the last test. It will contain
726 issue105560044_120001_diff = testing
729 for any issue that it thinks is still in the process of being tested.
732 Manually remove this entry and re-run the script.
738 test-master-lock and PID entry exist but previous Patchy
739 run (PID xxxxx) died, resetting test-master-lock anyway.
743 A previous test attempt was unsuccessful for some reason and the scripts
744 were not able to tidy up after themselves (for example if you manually
745 halt the testing process by killing it or closing the terminal you may
746 have been running the script in). The @code{test-master-lock} branch
747 was therefore not able to be deleted cleanly however, nothing needs to
748 be done the scripts will rebuild any tests it needs to.
752 fatal: A branch named 'test-master-lock' already exists.
758 There is another instance of Patchy running on your computer that is
759 testing the same tracker issue.
762 A previous test attempt was unsuccessful for some reason and the scripts
763 were not able to tidy up after themselves (for example if you manually
764 halt the testing process by killing it or closing the terminal you may
765 have been running the script in). The @code{test-master-lock} branch
766 was therefore not able to be deleted cleanly, in this case you must
767 manually delete the @code{test-master-lock} branch in your
768 @code{$LILYPOND_GIT} directory.
771 git branch -d test-master-lock
775 It may be wise to also manually delete @code{test-master} and
776 @code{test-staging} too, just to be safe.
784 Another instance (PID xxxxx) is already running.
788 This occurs when trying to run @code{lilypond-patchy-staging.py} when
789 another instance of either script is already running locally.
793 Warning: something went wrong; omitting patch for issue 3976
799 The Rietveld URL as listed in the tracker is incorrect (e.g. missing or
800 incorrect issue number
803 The patch on Rietveld is too large to download
810 @node Administrative mailing list
811 @section Administrative mailing list
813 A mailing list for administrative issues is maintained at
814 @code{lilypond-hackers@@gnu.org}.
816 This list is intended to be used for discussions that should be kept
817 private. Therefore, the archives are closed to the public.
819 Subscription to this list is limited to certain senior developers.
821 At the present time, the list is dormant.
823 Details about the criteria for membership, the types of discussion
824 to take place on the list, and other policies for the hackers list
825 will be finalized during the
826 @ref{Grand Organization Project (GOP)}.
830 @node Grand Organization Project (GOP)
831 @section Grand Organization Project (GOP)
837 Clarify the various development tasks by writing down the policies
838 and techniques and/or simplifying the tasks directly.
841 Get more people involved in development: specifically, find people
842 to do easy tasks to allow advanced developers to concentrate on
851 * Policy decisions (finished)::
855 @subsection Motivation
857 Most readers are probably familiar with the LilyPond Grand
858 Documentation Project, which ran from Aug 2007 to Aug 2008. This
859 project involved over 20 people and resulted in an almost complete
860 rewrite of the documentation. Most of those contributors were
861 normal users who decided to volunteer their time and effort to
862 improve lilypond for everybody. By any measure, it was a great
865 The Grand Organization Project aims to do the same thing with a
866 larger scope -- instead of focusing purely on documentation, the
867 project aims to improve all parts of LilyPond and its community.
868 Just as with GDP, the main goal is to encourage and train users to
869 become more involved.
871 If you have never contributed to an open-source project before --
872 especially if you use Windows or OSX and do not know how to
873 program or compile programs -- you may be wondering if there's
874 anything you can do. Rest assured that you @emph{can} help.
876 @subheading "Trickle-up" development
878 One of the reasons I'm organizing GOP is "trickle-up"
879 development. The idea is this: doing easy tasks frees up advanced
880 developers to do harder tasks. Don't ask "am I the @emph{best}
881 person for this job"; instead, ask "am I @emph{capable} of doing
882 this job, so that the current person can do stuff I @emph{can't}
885 For example, consider lilypond's poor handling of grace notes in
886 conjunction with clef and tempo changes. Fixing this will require
887 a fair amount of code rewriting, and would take an advanced
888 developer a few weeks to do. It's clearly beyond the scope of a
889 normal user, so we might as well sit back and do nothing, right?
891 No; we @emph{can} help, indirectly. Suppose that our normal user
892 starts answering more emails on lilypond-user. This in turn means
893 that documentation writers don't need to answer those emails, so
894 they can spend more time improving the docs. I've noticed that all
895 doc writers tackle harder and harder subjects, and when they start
896 writing docs on scheme programming and advanced tweaks, they start
897 contributing bug fixes to lilypond. Having people performing these
898 easy-to-moderate bug fixes frees up the advanced developers to
899 work on the really hard stuff... like rewriting the grace note
902 Having 1 more normal user answering emails on lilypond-user won't
903 have a dramatic @q{trickle-up} effect all by itself, of course. But if
904 we had 8 users volunteering to answer emails, 6 users starting to
905 write documentation, and 2 users editing LSR... well, that would
906 free up a lot of current bug-fixing-capable contributors to focus
907 on that, and we could start to make a real dent in the number of
908 bugs in lilypond. Quite apart from the eased workload, having that
909 many new helpers will provide a great moral boost!
912 @subsection Ongoing jobs
914 Although GOP is a short-term project, the main goal is to train
915 more people to handle ongoing jobs. The more people doing these
916 jobs, the lighter the work will be, and the more we can get done
919 Also, it would be nice if we had at least one "replacement" /
920 "understudy" for each role -- too many tasks are only being done
921 by one person, so if that person goes on vacation or gets very
922 busy with other matters, work in that area grinds to a halt.
924 @subheading Jobs for normal users
928 LilyPond is sometimes critized for not listening to users, but
929 whenever we ask for opinions about specific issues, we never get
930 enough feedback. This is somewhat aggravating.
931 We need a group of users to make a dedicated effort to test and
932 give feedback. If there's new documentation, read it. If there's
933 an experimental binary, download it and try compiling a score with
934 it. If we're trying to name a new command, think about it and give
937 @item lilypond-user support:
938 I think it would be nice if we had an official team of users
941 @item LilyPond Report:
942 Keeping a monthly newsletter running is a non-trivial task. A lot
943 of work is needed to organize it; it would be great if we could
944 split up the work. One person could write the Snippet of the
945 Month, another person could do Quotes of the Month, another person
946 could do interviews, etc.
949 Although GDP (the Grand Documentation Project) did great work,
950 there's still many tasks remaining.
953 Keeping the documentation translations is a monumental task; we
954 need all the help we can get!
958 @subheading Jobs for advanced users for developers
961 @item Git help for writers:
962 We often receive reports of typos and minor text updates to the
963 documentation. It would be great if somebody could create
964 properly-formatted patches for these corrections.
966 Technical requirements: ability to run @ref{LilyDev}.
969 LSR contains many useful examples of lilypond, but some snippets
970 are out of date and need updating. Other snippets need to be
971 advertized, and new snippets need to be sorted. We could use
972 another person to handle LSR.
974 Technical requirements: use of a web browser. LilyPond
975 requirements: you should be familiar with most of Notation
976 chapters 1 and 2 (or be willing to read the docs to find out).
978 @item Join the Frogs:
979 "Frogs" are a team of bug-fixers (because frogs eat bugs, and you
980 often find them in Ponds of Lilies) and new feature implementors.
982 Technical requirements: development environment (such as
983 @ref{LilyDev}), ability to read+write scheme and/or C++ code.
988 @node Policy decisions
989 @subsection Policy decisions
991 There are a number of policy decisions -- some of them fairly
992 important -- which we have been postponing for a few years. We
993 are now discussing them slowly and thoroughly; agenda and exact
994 proposals are online:
997 @uref{http://lilypond.org/~graham/gop/index.html}
1000 Below is a list of policies which are not @qq{on the agenda} yet.
1002 Note that the presence of an item on this list does @emph{not}
1003 mean that everybody thinks that something needs to be done.
1004 Inclusion in this simply means that one developer thinks that we
1005 should discuss it. We are not going to filter this list; if any
1006 developer thinks we should discuss something, just add it to the
1007 bottom of the list. (the list is unsorted)
1009 As GOP progresses, items from this list will be put on the agenda
1010 and removed from this list. I generally try to have one month's
1011 discussion planned in advance, but I may shuffle things around to
1012 respond to any immediate problems in the developer community.
1014 There are some item(s) not displayed here; these are questions
1015 that were posed to me privately, and I do not feel justified in
1016 discussing them publicly without the consent of the person(s) that
1017 brought them up. They will initially be discussed privately on the
1018 lilypond-hackers mailing list -- but the first question will be
1019 "do we absolutely need to do this privately", and if not, the
1020 discussion will take place on lilypond-devel like the other items.
1022 In most policy discussions in lilypond over the past few years,
1023 the first half (or more) is wasted arguing on the basis of
1024 incorrect or incomplete data; once all the relevant facts are
1025 brought to light, the argument is generally resolved fairly
1026 quickly. In order to keep the GOP discussions focused, each topic
1027 will be introduced with a collection of relevant facts and/or
1028 proposals. It is, of course, impossible to predict exactly which
1029 facts will be relevant to the discussion -- but spending an hour
1030 or two collecting information could still save hours of
1033 @warning{The estimated time required for "prep work", and the
1034 following discussion, has been added to each item. At the moment,
1035 there is an estimated 30 hours of prep work and 140 hours of
1039 @item @strong{Patch reviewing}:
1040 At the time of this writing, we have 23 (known) patches waiting
1041 for review. Some from main developers; some from new developers.
1042 We desperately need more people helping with lilypond, but
1043 ignoring patches is the best way to drive potential contributors
1044 away. This is not good.
1046 (prep: 2 hours. discuss: 10 hours)
1048 @item @strong{Official links to other organizations?}:
1049 There's something called the "software freedom conservancy", and
1050 in general, there's a bunch of "umbrella organizations". Joining
1051 some of these might give us more visibility, possibly leading to
1052 more users, more developers, maybe even financial grants or use in
1055 (prep: 2 hours. discuss: 5 hours)
1057 @item @strong{Issue tracking with google code}:
1058 We use the google issue tracker, but this means that we are
1059 relying on a commercial entity for a large part of our
1060 development. Would it be better (safer in the long run) to use the
1061 savannah bug tracker?
1063 (prep: 1 hour. discuss: 5 hours)
1065 @item @strong{Patch review tool}:
1066 Reitveld is inconvenient in some respects: it requires a google
1067 account, and there's no way to see all patches relating to
1068 lilypond. Should we switch to something like gerritt?
1069 @uref{http://code.google.com/p/lilypond/issues/detail?id=1184}
1071 (prep: 5 hours. discuss: 15 hours)
1073 @item @strong{Clarity for sponsorships}:
1074 We currently do not advertize bounties and sponsorships on the
1075 webpage. How much advertising do we want, and what type?
1076 Should we change the "structure" / "framework" for bounties?
1078 (prep: 2 hours. discuss: 10 hours)
1080 @item @strong{code readability}:
1081 "Our aim when producing source code for Lilypond in whatever
1082 language is that it should be totally comprehensible to a
1083 relatively inexperienced developer at the second reading."
1086 - aids maintainability of code base
1087 - "second reading" so newer developers can look up unfamiliar
1089 - will help to keep things simple, even if the code is doing
1090 complex stuff discourages "secret squirrel" coding, e.g. "how
1091 much functionality can I squeeze into as few characters as
1092 possible" "comments are for wimps"
1093 - will aid not *discouraging* new developers to join the project
1095 (prep: 2 hours. discuss: 10 hours)
1097 @item @strong{C++ vs. scheme}:
1098 what should be in scheme, what should be in C++, what can/should
1099 be ported from one to the other, etc. Questions of
1100 maintainability, speed (especially considering guile 2.0), and the
1101 amount of current material in either form, are important.
1103 (prep: 5 hours. discuss: 15 hours)
1105 @item @strong{always make an issue number for patches}:
1106 there is a proposal that we should always have a google code issue
1107 number for every patch. This proposal is closely tied to our
1108 choice of patch review tool; if we switch to a different tool (as
1109 suggested in a different proposal), this proposal may become moot.
1111 (prep: 1 hour. discuss: 5 hours)
1113 @item @strong{initalizer lists}:
1114 shoudl we use initalizer lists for C++? AFAIK they make no
1115 difference for built-in types, but there's some weird case where
1116 it's more efficient for objects, or something.
1118 Probably not worth making this a weekly thing on its own, but we
1119 can probably wrap it up with some other code-related questions.
1121 (prep: 15 minutes. discuss: 3 hours)
1125 @node Policy decisions (finished)
1126 @subsection Policy decisions (finished)
1128 Here is a record the final decisions, along with links to the
1132 * GOP-PROP 1 - python formatting::
1133 * GOP-PROP 2 - mentors and frogs::
1134 * GOP-PROP 3 - C++ formatting::
1135 * GOP-PROP 4 - lessons from 2.14::
1136 * GOP-PROP 5 - build system output (not accepted)::
1137 * GOP-PROP 6 - private mailing lists::
1138 * GOP-PROP 7 - developers as resources::
1139 * GOP-PROP 8 - issue priorities::
1140 * GOP-PROP 9 - behavior of make doc::
1143 @node GOP-PROP 1 - python formatting
1144 @subsubsection GOP-PROP 1 - python formatting
1146 We will follow the indentation described in PEP-8.
1147 @uref{http://www.python.org/dev/peps/pep-0008/}
1151 use 4 spaces per indentation level
1154 never mix tabs and spaces (for indentation)
1157 Code indented with a mixture of tabs and spaces should be
1158 converted to using spaces exclusively
1160 Once this is done, we should add @code{python -tt} to the build
1161 system to avoid such errors in the future.
1165 There should be absolutely no tab characters for indentation in
1166 any @code{.py} file in lilypond git. All such files should be
1167 converted to use spaces only.
1169 @subsubheading Discussions
1172 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00060.html}
1173 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00084.html}
1174 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00310.html}
1175 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00574.html}
1179 @node GOP-PROP 2 - mentors and frogs
1180 @subsubsection GOP-PROP 2 - mentors and frogs
1182 Nothing much was decided. The list of responsibilities was
1183 slightly altered; see the new one in @ref{Mentors}. We should
1184 encourage more use of the Frogs mailing list. There's a list of
1185 contributor-mentor pairs in:
1188 @uref{https://github.com/gperciva/lilypond-extra/blob/master/people/mentors.txt}
1191 That's pretty much it.
1193 @subsubheading Discussions
1196 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00311.html}
1202 @node GOP-PROP 3 - C++ formatting
1203 @subsubsection GOP-PROP 3 - C++ formatting
1205 Speaking academically, C++ code style is a "solved problem". Let's
1206 pick one of the existing solutions, and let a computer deal with
1207 this. Humans should not waste their time, energy, and creativity
1208 manually adding tabs or spaces to source code.
1210 We have modified @code{fixcc.py} to use astyle, along with extra
1215 the final script will be run @strong{blindly} on the lilypond
1216 source code. We will accept whatever formatting the final version
1217 of this script produces, with no manual tweaking.
1220 patches which have been run through this tool will not be rejected
1221 for style reasons. Any code formatting @qq{desires} which are not
1222 enforced by @code{fixcc.py} will not be considered grounds for
1226 for now, this style will not be enforced. It is not cause for
1227 concern if patches which do not follow the formatting done by
1228 @code{fixcc.py} are pushed. From time to time, Graham will run
1229 the formatter on the entire code base, and commit the resulting
1232 In a few months, we will tighten up this policy item (with some
1233 sort of automatic processing), but that is outside the scope of
1234 this policy item and is a matter for later discussion.
1237 after the proposal is accepted, we will leave some time for
1238 existing patches to be accepted and pushed. The script was
1239 run on the source code on @strong{2011 August 01}.
1243 @subheading GNU code
1245 LilyPond is a GNU project, so it makes sense to follow the GNU
1246 coding standards. These standards state:
1249 We don’t think of these recommendations as requirements, because
1250 it causes no problems for users if two different programs have
1251 different formatting styles.
1253 But whatever style you use, please use it consistently, since a
1254 mixture of styles within one program tends to look ugly. If you
1255 are contributing changes to an existing program, please follow the
1256 style of that program.
1259 (@uref{http://www.gnu.org/prep/standards/html_node/Formatting.html})
1261 With that in mind, we do not think that we must blindly follow the
1262 formatting given by the currrent version of Emacs.
1264 @subheading Implementation notes
1266 We can avoid some of the style change pollution in git history by
1267 ignoring whitespaces changes:
1273 @subsubheading Discussions
1276 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00526.html}
1277 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00796.html}
1278 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00200.html}
1279 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00525.html}
1280 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00751.html}
1281 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00751.html}
1285 @node GOP-PROP 4 - lessons from 2.14
1286 @subsubsection GOP-PROP 4 - lessons from 2.14
1290 A brief history of releases:
1292 @multitable @columnfractions .2 .2 .3
1293 @headitem date (YYYY-MM-DD) @tab version @tab comment
1294 @item 2008-10-28 @tab 2.11.63 @tab nobody checking regtests
1295 @item 2008-11-17 @tab 2.11.64
1296 @item 2008-11-29 @tab 2.11.65
1297 @item 2008-12-23 @tab 2.12.0
1298 @item 2009-01-01 @tab @tab somewhere around here, Graham becomes
1299 officially release manager, but Han-Wen still builds the actual
1301 @item 2009-01-01 @tab 2.12.1
1302 @item 2009-01-25 @tab 2.12.2
1303 @item 2009-02-28 @tab 2.13.0
1304 @item 2009-06-01 @tab 2.13.1 @tab note jump in time!
1305 @item 2009-06-27 @tab 2.13.2 @tab first Graham release?
1306 @item 2009-07-03 @tab 2.13.3
1307 @item 2009-09-09 @tab @tab Graham arrives in Glasgow, gets a
1308 powerful desktop computer, and begins serious work on GUB (sending
1309 bug reports to Jan). It takes approximately 100 hours until GUB
1310 is stable enough to make regular releases.
1311 @item 2009-09-24 @tab 2.13.4
1312 @item 2009-10-02 @tab 2.13.5
1313 @item 2009-10-22 @tab 2.13.6
1314 @item 2009-11-05 @tab 2.13.7
1316 @item 2010-01-13 @tab 2.12.3
1318 @item 2010-03-19 @tab 2.13.16 @tab Bug squad starts doing a few
1319 regtest comparisons, but IIRC the effort dies out after a few
1322 @item 2010-08-04 @tab 2.13.29 @tab Phil starts checking regtests (BLUE)
1324 @item 2011-01-12 @tab 2.13.46 @tab release candidate 1 (GREEN)
1326 @item 2011-05-30 @tab 2.13.63 @tab release candidate 7 (GREEN)
1327 @item 2011-06-06 @tab 2.14.0
1330 @c A graphical display of bugs:
1332 @c @image{bugs-2.13-visualization,png}
1333 @c @image{zoom-2.13-visualization,png}
1335 @subheading Carl's analysis of the bugs
1337 A @file{csv} spreadsheet is available.
1340 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00852.html}
1344 @uref{lilypond-issues-analysis.csv}
1345 @uref{lilypond-issues-analysis-trim-duplicates.csv}
1348 There 148 issues marked with Priority=Critical in the tracker.
1350 I've done an analysis, and it looks to me like there was initially
1351 a backlog of critical issues that weren't fixed, and little work
1352 was being done to eliminate critical issues.
1354 Somewhere about 2010-08-01, critical issues started to disappear,
1355 but occasional new ones appeared.
1357 There were a couple of major changes that introduced unanticipated
1358 regressions (new spacing code, beam collision avoidance). These
1359 produced more than the expected number of regressions.
1361 It appears to me that we didn't really get serious about
1362 eliminating critical bugs until about 2010-06-15 or so. After
1363 that point, the number of critical bugs more-or-less steadily
1364 decreased until we got to a release candidate.
1366 Of particular interest, the first release candidate of 2.14 was
1367 released on 2011-01-12. Over the next 10 days, about a dozen bugs
1368 were reported and fixed. Release candidate 2 came out on
1369 2011-02-09. No surge of bugs occurred with this release.
1370 Candidate 3 came out on 2011-03-13; we got 2 bugs per week.
1371 Candidate 4 came out on 2011-03-29; 2 new bugs. Candidate 6 came
1372 out on 2011-04-07. We got a couple of bugs per week.
1374 @subheading Notes, commentary, and opinions
1377 Han-Wen: Overall, I think this cycle took too long
1382 @subsubheading Discussions
1385 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00797.html}
1386 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00364.html}
1391 @node GOP-PROP 5 - build system output (not accepted)
1392 @subsubsection GOP-PROP 5 - build system output (not accepted)
1394 This proposal was too broad; after a month of discussion, Graham
1395 withdrew the proposal. Portions of it will be introduced in later
1398 @subsubheading Discussions
1401 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00320.html}
1402 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00527.html}
1403 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00753.html}
1404 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg01042.html}
1405 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00116.html}
1406 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00310.html}
1410 @node GOP-PROP 6 - private mailing lists
1411 @subsubsection GOP-PROP 6 - private mailing list
1413 Potentially sensitive or private matters will be referred to
1414 Graham. He will then decide who should discuss the matter on an
1415 ad-hoc basis, and forward or CC them on future emails.
1417 For emphasis, the project administrators are Han-Wen, Jan, and
1418 Graham; those three will always be CC'd on any important
1421 The lilypond-hackers mailing list will be removed.
1425 There is some unhappy history about this idea in our development
1429 @uref{http://lists.gnu.org/archive/html/lilypond-devel/2010-09/msg00178.html}
1430 @uref{http://news.lilynet.net/spip.php?article121}
1431 @uref{http://lists.gnu.org/archive/html/lilypond-devel/2010-11/msg00076.html}
1434 @subheading Other projects
1436 The idea of private mailing lists is hardly uncommon in
1437 open-source software. For example,
1440 @uref{http://lwn.net/Articles/394660/} about debian-private
1441 @uref{http://subversion.apache.org/mailing-lists.html} private@@
1442 @uref{http://www.freebsd.org/administration.html#t-core}
1443 @uref{http://foundation.gnome.org/legal/} board members pledge
1444 to keep certain matters confidential
1446 every security team of every GNU/Linux distribution and OS
1449 In fact, Karl Fogel's @qq{Producing Open Source Software}
1450 explicitly suggests a private mailing list for some circumstances:
1453 [on granting commit/push access to a contributor]
1455 But here is one of the rare instances where secrecy is
1456 appropriate. You can't have votes about potential committers
1457 posted to a public mailing list, because the candidate's feelings
1458 (and reputation) could be hurt.
1460 @uref{http://producingoss.com/en/consensus-democracy.html#electorate}
1463 @subheading Board of governers, voting, etc?
1465 Many projects have an official board of directors, or a list of
1466 @qq{core developers}, with set term limits and elections and
1469 I don't think that we're that big. I think we're still small
1470 enough, and there's enough trust and consensus decisions, that we
1471 can avoid that. I would rather that we kept on going with
1472 trust+consensus for at least the next 2-3 years, and spent more
1473 time+energy on bug fixes and new features instead of
1474 administrative stuff.
1476 Project administrators are Han-Wen, Jan, and Graham.
1478 @subsubheading Discussions
1481 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00783.html}
1482 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg01004.html}
1483 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00117.html}
1487 @node GOP-PROP 7 - developers as resources
1488 @subsubsection GOP-PROP 7 - developers as resources
1490 We shall treat developers (and contributors) as
1491 @strong{Independent volunteers}: each person does whatever they
1492 want, whenever they want. We have busy careers and lives; we make
1493 no expectations of action from anybody (with the exception of the
1494 6 people in @qq{Meister} positions).
1496 @subsubheading Discussions
1499 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg01092.html}
1500 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00087.html}
1501 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00497.html}
1505 @node GOP-PROP 8 - issue priorities
1506 @subsubsection GOP-PROP 8 - issue priorities
1508 We will delete the @qq{priority} field of the issue tracker
1509 altogether. The @qq{type} system will be tweaked.
1516 a reproducible failure to build either @code{make} or @code{make
1517 doc}, from an empty build tree, in a first run, if
1518 @code{configure} does not report any errors.
1521 any program behaviour which is @strong{unintentionally} worse than
1522 the previous stable version or the current development version.
1523 Developers may always use the @qq{this is intentional}, or even
1524 the @qq{this is an unavoidable effect of an improvement in another
1525 area}, reason to move this to a different type.
1528 anything which stops contributors from helping out (e.g.
1529 lily-git.tcl not working, source tree(s) not being available,
1530 LilyDev being unable to compile git master, inaccurate
1531 instructions in the Contributor's Guide 2 Quick start).
1533 To limit this scope of this point, we will assume that the
1534 contributor is using the latest LilyDev and has read the relevant
1535 part(s) of the Contributor's Guide. Problems in other chapters of
1536 the CG are not sufficient to qualify as Type-Critical.
1540 @subsubheading More new/changed types and labels
1542 Unless otherwise specified, the current types and labels will
1543 continue to be used. The new types introduced by this proposal
1549 Type-crash: any segfault, regardless of what the input file looks
1550 like or which options are given. Disclaimer: this might not be
1551 possible in some cases, for example certain guile programs (we
1552 certainly can't predict if a piece of scheme will ever stop
1553 running, i.e. the halting problem), or if we rely on other
1554 programs (i.e. ghostscript). If there are any such cases that
1555 make segfault-prevention impossible, we will document those
1556 exceptions (and the issue will remain as a "crash" instead of
1557 "documentation" until the warning has been pushed).
1560 Type-maintainability: anything which makes it difficult for
1561 serious contributors to help out (e.g. difficult to find the
1562 relevant source tree(s), confusing policies, problems with
1563 automatic indentation tools, etc).
1566 Type-ugly: replaces Type-collision, and it will include things
1567 like bad slurs in addition to actual collision.
1571 A new label will be added:
1575 (label) Needs_evidence: it is not clear what the correct output
1576 should look like. We need scans, references, examples, etc.
1580 @subheading Reminding users about stars
1582 We can remind users that they can @qq{star} an issue to indicate
1583 that they care about it. Since we resolved to treat developers as
1584 independent volunteers, there is no expectation that anybody will
1585 look at those stars, but if any developer want to organize their
1586 work schedule according to the stars, they are welcome to do so.
1588 @subsubheading Discussions
1591 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00019.html}
1592 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00277.html}
1593 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00413.html}
1594 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00624.html}
1599 @node GOP-PROP 9 - behavior of make doc
1600 @subsubsection GOP-PROP 9 - behavior of make doc
1602 If there are build problems, then it should be easier to find out
1603 why it's failing. This will be achieved with log files, as well
1604 as possibly including scripts which automatically display portions
1605 of those log files for a failing build.
1607 We will also add targets for building a specific manual (for
1608 quick+easy checking of doc work), as well as for building all
1609 documentation in a specific language (either English or a
1610 translated language).
1612 When you run @code{make doc},
1617 All output will be saved to various log files, with the exception
1618 of output directly from @code{make(1)}.
1620 Note that @code{make(1)} refers to a specific executable file on
1621 unix computers, and is not a general term for the build system.
1624 By default, no other output will be displayed on the console, with
1625 one exception: if a build fails, we might display some portion(s)
1626 of log file(s) which give useful clues about the reason for the
1629 The user may optionally request additional output to be printed;
1630 this is controlled with the @code{VERBOSE=x} flag. In such cases,
1631 all output will still be written to log files; the console output
1632 is strictly additional to the log files.
1635 Logfiles from calling lilypond (as part of lilypond-book) will go in
1637 @file{$LILYPOND_BUILD_DIR/out/lybook-db/12/lily-123456.log} file. All
1638 other logfiles will go in the @file{$LILYPOND_BUILD_DIR/logfiles/}
1641 A single @code{make doc} will therefore result in hundreds of log
1642 files. Log files produced from individual lilypond runs are not
1643 under our control; apart from that, I anticipate having one or two
1644 dozen log files. As long as it is clear which log file is
1645 associated with which operation(s), I think this is entirely
1646 appropriate. The precise implementation will be discussed for
1647 specific patches as they appear.
1650 Both stderr and stdout will be saved in @code{*.log}. The order
1651 of lines from these streams should be preserved.
1654 There will be no additional @qq{progress messages} during the
1655 build process. If you run @code{make --silent}, a non-failing
1656 build should print absolutely nothing to the screen.
1659 Assuming that the loglevels patch is accepted, lilypond (inside
1660 lilypond-book) will be run with --loglevel=WARN.
1661 @uref{http://codereview.appspot.com/4822055/}
1664 Ideally, a failing build should provide hints about the reason why
1665 it failed, or at least hints about which log file(s) to examine.
1669 If this proposal is accepted, none of these policies will be
1670 assumed to apply to any other aspect of the build system.
1671 Policies for any other aspect of the build system will be
1672 discussed in separate proposals.
1674 @subheading Don't cause more build problems
1676 However, there is a danger in this approach, that vital error
1677 messages can also be lost, thus preventing the cause of the
1678 failure of a make being found. We therefore need to be
1679 exceptionally careful to move cautiously, include plenty of tests,
1680 and give time for people to experiment/find problems in each stage
1681 before proceeding to the next stage.
1683 This will be done by starting from individual lilypond calls
1684 within lilypond-book, and slowly moving to @qq{larger} targets of
1685 the build system -- after the individual lilypond calls are are
1686 producing the appropriate amount of output and this is saved in
1687 the right place and we can automatically isolate parts of a
1688 failing build, we will work on lilypond-book in general, and only
1689 then will we look at the build system itself.
1691 @subheading Implementation notes
1693 There is an existing make variable QUIET_BUILD, which
1694 alter the amount of output being displayed
1696 http://lilypond.org/doc/v2.15/Documentation/contributor/useful-make-variables}
1697 ). We are not planning on keeping this make variable.
1699 The standard way for GNU packages to give more output is with a
1700 @code{V=x} option. Presumably this is done by increasing
1701 @code{x}? If we support this option, we should still write log
1702 files; we would simply print more of the info in those log files
1705 The command @code{tee} may be useful to write to a file and
1706 display to stdout (in the case of VERBOSE).
1709 @subsubheading Discussions
1712 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00378.html}
1713 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00703.html}
1718 @n ode GOP-PROP 10 - scheme indentation
1719 @s ubsubsection GOP-PROP 10 - scheme indentation
1721 still under discussion
1723 @subsubheading Discussions
1726 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00625.html}
1727 @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg01026.html}
1734 @node Grand LilyPond Input Syntax Standardization (GLISS)
1735 @section Grand LilyPond Input Syntax Standardization (GLISS)
1741 Start: sortly after 2.14 comes out, which is currently estimated
1742 to happen in January 2011.
1745 Length: 6-12 months. We're not going to rush this.
1748 Goal: define an input which we commit to being
1749 machine-updateable for the forseeable future. Any future patches
1750 which change the syntax in a non-convert-ly-able format will be
1751 rejected. (subject to the limitations, below)
1752 Once this is finished, we will release lilypond 3.0.
1757 @subheading The Problem
1759 One of the biggest complaints people have with lilypond -- other
1760 than silly thing like "there's no gui" -- is the changing syntax.
1761 Now, inventing a language or standards is difficult. If you set
1762 it in stone too soon, you risk being stuck with decisions which
1763 may limit matters. If you keep on updating the syntax,
1764 interaction with older data (and other programs!) becomes complex.
1766 @subheading Scope and Limitations
1770 tweaks will not be included. Anything with \override, \set,
1771 \overrideProperty, \tweak, \revert, \unset... including even those
1772 command names themselves... is still fair game for NOT_SMART
1776 other than that, everything is on the table. Is it a problem to
1777 have the tagline inside \header? What should the default behavior
1781 we need to get standards for command names. This will help users
1782 remember them, and reduce the options for future names (and
1783 potential renamings later on). \commandOn and \commandOff seem to
1784 work well (should we *always* have an Off command?), but what
1785 about the "command" part? Should it be \nounVerbOn, or
1786 \verbNounOn ? Or \verbNotesWithExtraInformationOn ?
1789 we need standards for the location of commands. Ligature
1790 brackets, I'm looking at you. (non-postfix notation must die!)
1793 this Grand Project doesn't affect whether we have a 2.16 or not.
1794 The main problem will be deciding what to do (with a bit of
1795 messiness anticipated for \tuplet); we should definitely release a
1796 2.16 before merging _any_ of these changes.
1799 we obviously can't /guarantee/ that we'll /never/ make any
1800 non-convert-ly changes in the basic format. But we *can*
1801 guarantee that such changes would force lilypond 4.0, and that we
1802 would only do so for overwhelmingly good reasons.
1806 @subheading Workflow
1810 We're going to have lots and lots of emails flying around. The
1811 vast majority won't really fit into either -devel or -user, so
1812 we'll use a list devoted to syntax issues.
1815 Once we have a serious proposal that gained general acceptance
1816 from the separate syntax mailing list, I'll bring it to -devel.
1817 We're not going to make any changes without discussing it on
1818 -devel, but if we're going to have huge threads about English
1819 grammar and silly ideas, and I don't want to clutter up -devel.
1820 Once whatever chaotic silliness on the syntax list is settled
1821 down, I'll bring the ideas to -devel.
1824 as with GDP, I'll moderate the discussion. Not as with mailist
1825 moderation, but rather by introducing issues at specific times.
1826 We don't want a free-for-all discussion of all parts of the syntax
1827 at once; nothing will get resolved.
1830 Whenever possible, we'll decide on policies at the highest level
1831 of abstraction. For example, consider \numericTimeSignature,
1832 \slurUp, \xNotesOn, \startTextSpan, and \verylongfermata. One of
1833 them starts with the name of the notation first (slur). One has
1834 an abbreviation (x instead of cross). One has the verb at the end
1835 (On), another has it at the beginning (start). The adjective can
1836 come at the beginning (numeric, x) or end (Up). Most are in
1837 camelCase, but one isn't (verylongfermata).
1840 Instead of arguing about each individual command, we'll decide on
1841 abstract questions. Should each command begin the notation-noun,
1842 or the verb? Should all commands be in camelCase, or should we
1843 make everything other than articulations in camelCase but make
1844 articulations all lower-case? Are abbreviations allowed?
1847 Once we've answered such fundamental questions, most of the syntax
1848 should fall into place pretty easily. There might be a few odd
1849 questions left ("is it a span, or a spanner?"), but those can be
1850 settled fairly quickly.
1854 @subheading Implementation
1856 Nothing until the project is finished, then we declare the next
1857 stable release (2.16.0 or 2.18.0 ?) to be the final 2.x version,
1858 release it, then apply all the GLISS syntax changes and start
1859 testing a beta for 3.0 a week or two later.
1861 @subheading Discussion
1863 Don't respond to any of the specifics yet. Yes, we all have our
1864 pet irritations (like "what's up with \paper and \layout?!").
1865 There will be plenty of time to discuss them once GLISS starts.
1867 That said, we have a list of specific items that people really
1868 wanted to have written down. See @ref{Specific GLISS issues}.
1871 * Specific GLISS issues::
1875 @node Specific GLISS issues
1876 @subsection Specific GLISS issues
1880 add regtests for every piece of syntax (not one-command-per-file,
1881 but making a few files which, between them, use every single piece
1882 of syntax.) This is a great test for convert-ly.
1885 should GLISS cover suggested conventions? (indentation,
1886 one-bar-per-line, etc -- the kind of stuff we list for the
1887 lilypond formatting in the docs ?)
1890 how much (if any) syntactic sugar should we add? i.e.
1892 \instrumentName #'foo
1894 \set Staff.instrumentName
1896 ? Carl: maybe yes, Neil: no. (for example, it fails for
1900 the values that are used as arguments to common used overrides.
1901 Sometimes they are a symbol (e.g. #'around), sometimes a
1902 predefined variable referring to a Scheme value or object (e.g.
1903 #LEFT, #all-visible ). The main trouble is that for novice users
1904 it is not clear when there should be an apostrophe and when not.
1907 When do we need -\command and when is it just \command ?
1911 Command-line options to the lilypond binary. -dfoo counts as a
1912 tweak; we won't be trying to pin those down.
1925 If would be pedagogically simpler to realize this difference if
1926 the syntax was separate if you define a context from scratch (as
1927 is the case with \RemoveEmptyStaffContext) or if it's defined by
1928 adding onto an existing context. For example, a syntax like
1932 % Copy the current settings of the Staff context:
1934 % do whatever additional settings
1936 %%% could be used to distinguish from
1938 % Take settings from a variable:
1940 % do whatever additional settings
1946 % Start from scratch:
1955 Capitalization of identifiers: \VoiceOne ?
1960 { music expression } * 4
1962 \repeat unfold 4 { music expression }
1967 @uref{http://lists.gnu.org/archive/html/lilypond-devel/2010-04/msg00467.html}
1971 Personally, I find it easier to understand when there's a repeated
1972 8 in the half-bar position; it's much easier to see that you have
1977 %%% instead of one group of eight:
1982 trivially simple bar-lines:
1986 encourage, allow, or discourage, or disallow?
1989 indentation of \\ inside a @{@} construct.
1993 barline checks at the end of line should be preceded by at least 2
1994 spaces? barline checks should line up if possible (i.e. if you
1995 can use less than 4, 8, X empty spaces before a barline check to
1999 Why doesn't \transpose respect \relative mode?
2003 on \score vs. \new Score
2005 But in the light of a consistent syntax and semantic, I see no
2006 reason (from the users POV) to disallow it. After all, the real
2007 top-level context is a \book @{@}, isn't it, and I don't see a point
2008 in disallowing a \new Score construct just like \new Staff.
2010 From a syntactical POV, I see the following pros for \new Score:
2011 - You can write \with @{ ... @} for every other context but \Score,
2012 which (for consistency) should also work with \new Score.
2013 - When there's a \new Foo Bar, there's also a \context Foo Bar,
2014 which makes the same as a parallel instantiation of all Bar's.
2015 - [Quoting Rune from
2016 @uref{http://www.mail-archive.com/lilypond-devel@@gnu.org/msg14713.html}
2017 "I know that the \score-statement is a syntactical construct,
2018 but I think it would be nice to hide this fact from the users. I
2019 think we could make the use of score-block much more intuitive if
2020 changing the syntax to \new \Score and adding an implicit
2021 sequential-statement to the score."
2026 @uref{http://code.google.com/p/lilypond/issues/detail?id=1322}
2027 about \new vs. \context.
2031 Let users add their own items to the parser? comment 11 on:
2032 @uref{http://code.google.com/p/lilypond/issues/detail?id=1322}
2035 should engravers be pluralized (note_heads_engraver) or not
2036 (note_head_engraver) ?
2039 should we allow numbers in identifier names? Issue:
2040 @uref{http://code.google.com/p/lilypond/issues/detail?id=1670}
2043 should we officially allow accented characters? in general, how
2044 do we feel about utf-8 stuff?
2047 for the sake of completeness/simplicity, what about *disallowing*
2048 the "one-note" form of a music expression? i.e. only allowing
2051 \transpose c d { e1 }
2052 \transpose c d << e1 >>
2061 What should be the officially encouraged way of writing music for
2062 transposing instruments? Maybe it should be simplified?
2063 See http://lists.gnu.org/archive/html/lilypond-user/2011-07/msg00130.html
2068 @node Unsorted policies
2069 @section Unsorted policies
2071 @subsubheading Language-specific mailing lists
2073 A translator can ask for an official lilypond-xy mailing list once
2074 they've finished all @qq{priority 1} translation items.
2076 @subsubheading Performing yearly copyright update (@qq{grand-replace})
2078 At the start of each year, copyright notices for all source files
2079 should be refreshed by running the following command from the top of
2086 Internally, this invokes the script @file{scripts/build/grand-replace.py},
2087 which performs a regular expression substitution for old-year -> new-year
2088 wherever it finds a valid copyright notice.
2090 Note that snapshots of third party files such as @file{texinfo.tex} should
2091 not be included in the automatic update; @file{grand-replace.py} ignores these
2092 files if they are listed in the variable @code{copied_files}.
2095 @subsubheading Push git access
2097 Git access is given out when a contributor has a significant
2098 record of patches being accepted without problems. If existing
2099 developers are tired of pushing patches for a contributor, we'll
2100 discuss giving them push access. Unsolicited requests from
2101 contributors for access will almost always be turned down.