[lilypond.git] / Documentation / contributor / administration.itexi

@node Administrative policies
@chapter Administrative policies

This chapter discusses miscellaneous administrative issues which
don't fit anywhere else.

@menu
* Meta-policy for this document::
* Environment variables::
* Meisters::
* Automated testing with Patchy::
* Administrative mailing list::
* Grand Organization Project (GOP)::
* Grand LilyPond Input Syntax Standardization (GLISS)::
* Unsorted policies::
@end menu

@node Meta-policy for this document
@section Meta-policy for this document

The Contributor's Guide as a whole is still a work in progress,
but some chapters are much more complete than others.  Chapters
which are @qq{almost finished} should not have major changes
without a discussion on @w{@code{-devel}}; in other chapters, a
disorganized @qq{wiki-style dump} of information is encouraged.

Do not change (other than spelling mistakes) without discussion:

@itemize

@item
@ref{Introduction to contributing}

@item
@ref{Working with source code}

@end itemize

Please dump info in an appropriate @@section within these manuals,
but discuss any large-scale reorganization:

@itemize

@item
@ref{Compiling}

@item
@ref{Documentation work}

@item
@ref{Issues}

@item
@ref{Regression tests}

@item
@ref{Programming work}


@end itemize

Totally disorganized; do whatever the mao you want:

@itemize

@item
@ref{Website work}

@item
@ref{LSR work}

@item
@ref{Release work}

@item
@ref{Administrative policies}

@end itemize


@node Environment variables
@section Environment variables

Some maintenance scripts and instructions in this guide rely on
the following environment variables.  They should be predefined in
LilyDev distribution (see @ref{LilyDev}); if you set up your own
development environment, you can set them by appending these settings to
your @file{~/.bashrc} (or whatever defines your default environment
variables for the user account for LilyPond development), then logging
out and in (adapt directories to your setup):

@example
LILYPOND_GIT=~/lilypond-git
export LILYPOND_GIT
LILYPOND_BUILD_DIR=~/lilypond-git/build
export LILYPOND_BUILD_DIR
@end example

The standard build and install procedure (with @code{autogen.sh},
@code{configure}, @code{make}, @code{make install}, @code{make doc}
@dots{}) does not rely on them.

In addition, for working on the website, @code{LILYPOND_WEB_MEDIA_GIT}
should be set to the repository lilypond-extra, see
@ref{lilypond-extra}.


@node Meisters
@section Meisters

We have four jobs for organizing a team of contributors:

@itemize

@item
Bug Meister: trains new Bug Squad volunteers, organizes who works
on which part of their job, checks to make sure that everything is
running smoothly, and has final say on our policy for Issues.

Currently: Colin H

@item
Doc Meister: trains new doc editors/writers, organizes who works
on which part of the job, checks to make sure that everything is
running smoothly, and has final say on our policy for
Documentation.  Also includes LSR work.

Currently: None

@item
Translation Meister: trains new translators, updates the
translation priority list, and handles merging branches (in both
directions).

Currently: Francisco

@end itemize


@node Automated testing with Patchy
@section Automated testing with Patchy

@menu
* Overview of Patchy::
* Patchy requirements::
* Installing Patchy::
* Configuring Patchy::
* Running the scripts::
* Automating Patchy::
* Troubleshooting Patchy::
@end menu

@node Overview of Patchy
@subsection Overview of Patchy

Patchy is a set of Python scripts used for testing patches or testing &
pushing new commits added to @code{remote/origin/staging} to
@code{remote/origin/master}.

No programmatic skill is required to run either of the scripts; although
knowledge of compiling LilyPond and its documentation along with
understanding how to configure the @var{PATH} environment of your
computer is required.  See @ref{Working with source code}.

The two scripts that are used for each function are:

@itemize
@item
@code{test-patches.py}. This script tests issues labelled as
@qq{Patch-New} on the LilyPond issue tracker.  Part of the testing
process involves running the regression tests, so this script always
requires some human intervention in order to visually check for any
differences that may be present after a successful test has occurred.

@item
@code{lilypond-patchy-staging.py}.  This script checks for any new
commits in @code{remote/origin/staging}, makes sure that the new HEAD
compiles along with all the LilyPond documentation.  Then finally
pushing to @code{remote/origin/master}.  This script can be run and left
unattended, requiring no human intervention.

@end itemize

Both of the scripts can be run independently of each other and it is not
necessary to be able to run both.  So if you wanted to contribute to
LilyPond development, for example by @emph{just} testing patches then
this would still be a helpful contribution to LilyPond's development.

Patchy can also be configured to send emails after each successful (or
unsuccessful) operation.  This is not a requirement and is turned off
by default.

@c Need to explain in more detail how to set up Patchy for email but
@c as I don't use myself it I have no experience - JL


@node Patchy requirements
@subsection Patchy requirements

@unnumberedsubsec Testing new patches

@itemize

@item
A full local copy of the source code.  See
@ref{Working with source code}.

@item
All the software needed for compiling LilyPond @emph{and} the
documentation.  Although being able to build the full set of LilyPond's
manuals is not mandatory for testing (most) patches, part of the patch
testing process requires that the regression tests are run and it is
this that requires the software normally used for compiling
documentation.  See @ref{Compiling}.

@item
Commit access is @emph{not} required to test patches, but a valid login
to @uref{http://code.google.com/} @emph{is}.

@end itemize


@unnumberedsubsec Testing & pushing new commits

@itemize

@item
A full local copy of the source code.  See
@ref{Working with source code}.

@item
All the software needed for compiling LilyPond @emph{and} the
documentation.  Unlike testing patches, being able to build the full set
of LilyPond's documentation is required to be able to test & push new
commits.  See @ref{Compiling}.

@item
Commit access @emph{is} required to test and push new commits, but a
valid login to @uref{http://code.google.com/} is @emph{not}.  See
@ref{Commit access}.

@end itemize


@node Installing Patchy
@subsection Installing Patchy

The Patchy scripts are not part of the LilyPond code base, but can be
downloaded from @uref{https://github.com/gperciva/lilypond-extra/}.  The
scripts and related Python libraries are all located in the
@file{patches/} directory.

Alternatively, use @code{git clone};

@example
git clone https://github.com/gperciva/lilypond-extra/
@end example

This makes it simpler to update the scripts if any changes are ever made
to them.  Finally, add the location of the @file{patches/} directory to
your @var{PATH}.


@node Configuring Patchy
@subsection Configuring Patchy

@warning{It is recommended to create a new user on your computer
specifically to run the Patchy scripts as a security precaution and that
this user should not have any administrative privileges.  Also do not
set password protection for your ssh key else you will not be able to
run the scripts unattended.}

@enumerate

@item
Make sure the environment variables @var{LILYPOND_GIT} and
@var{LILYPOND_BUILD_DIR} are configured appropriately.  See
@ref{Environment variables}.

@item
To save being prompted for your login and password to
@uref{http://code.google.com/} when testing patches. create a
@emph{plain-text} file in your Patchy user's @code{$HOME} directory
called @code{.lilypond-project-hosting-login} containing your login and
password, each on a separate line.

@example
joe_smith123@@gmail.com
mYp455w0rd!
@end example

@item
Manually run either the @code{test-patches.py} or
@code{lilypond-patchy-staging.py} scripts and when prompted:

@smallexample
Warning: using default config; please edit /home/joe/.lilypond-patchy-config
Are you sure that you want to continue with the default config? (y/[n])
@end smallexample

Answer @qq{@code{n}} and press enter.

The next time either of the scripts are run they will use the
@code{.lilypond-patchy-config} settings copied to your @code{$HOME}
directory.

@item
Manually edit the @file{.lilypond-patchy-config} file, located in your
@code{$HOME} directory to change any of the default settings.

@end enumerate

These include:

@itemize

@item
All @code{make} operations are run with;
@example
extra_make_options = -j3 CPU_COUNT=3
@end example

See @ref{Saving time with the -j option}

@item
A complete build of all the LilyPond documentation is @emph{not}
performed;
@example
patch_test_build_docs = no
@end example

@item
Each instance of either a patch test or commit test & push is logged in;
@example
auto_compile_results_dir = ~/lilypond-auto-compile-results/
@end example

@item
Both scripts will perform their build operations in;
@example
build_dir = /tmp/lilypond-autobuild/
@end example

@end itemize

Each completed patch test will also generate its own directory in
@file{/tmp/...} labelled with the tracker issue number prefixed by
@code{show-}.
@example
\tmp\show-3446
@end example

Both the scripts create clones of @code{staging} and @code{master}
branches (prefixed with @code{test-}) with a third branch, called
@code{test-master-lock} used as a check to protect against two or more
instances of Patchy being run locally at the same time.


@node Running the scripts
@subsection Running the scripts

@unnumberedsubsec Testing & pushing new commits

@code{lilypond-patchy-staging.py} is run @emph{without} any arguments.
It then checks to see if @code{remote/origin/staging} is
@qq{further ahead} than @code{remote/origin/master}.

@noindent
If there are no new differences between the two branches since the last
run check, the script will report something like this:

@smallexample
(UTC) Begin LilyPond compile, previous commit at 4726764cb591f622e7893407db0e7d42bcde90d9
Success:                No new commits in staging
@end smallexample

@noindent
If there are any differences between the two branches since the last
run check, (or if the script cannot for any reason, locate the last
instance of a commit that it checked) it will report something like
this:

@smallexample
(UTC) Begin LilyPond compile, previous commit at 4726764cb591f622e7893407db0e7d42bcde90d9
Merged staging, now at: 79e98a773b6570cfa28a15775a9dea3d3e54d6b5
        Success:                ./autogen.sh --noconfigure
        Success:                /tmp/lilypond-autobuild/configure --disable-optimising
...
@end smallexample

and proceed with running @code{make}, @code{make test} and a
@code{make doc}.  Unlike @code{test-patches.py} if all the tests pass,
the script then pushes the changes to @code{remote/origin/master}.

@smallexample
...
Success:                nice make clean
Success:                nice make -j7 CPU_COUNT=7
Success:                nice make test -j7 CPU_COUNT=7
Success:                nice make doc -j7 CPU_COUNT=7
To ssh://joe@@git.sv.gnu.org/srv/git/lilypond.git
   79e98a7..4726764  test-staging -> master
        Success:                pushed to master
@end smallexample

@warning{In the case where any of the @code{lilypond-patchy-staging.py}
tests fail, do not try to push your own fixes but report the failures to
the Developers List <lilypond-devel@@gnu.org> for advice.}


@unnumberedsubsec Testing new patches

When run without any argument, @code{test-patches.py} will check
@uref{http://code.google.com/p/lilypond/issues/list} for all tracker
issues that are marked with the label @code{Patch-new}.  It then scrapes
the issue, looking for the last Rietveld URL entered.  It then downloads
the patch file and applies it to @code{test-master}.

Here is an example where two tracker issues labeled as @code{Patch-new}
were detected:

@smallexample
...
issues [4007, 4008]
Trying issue 4007
Found url: http://codereview.appspot.com/112210043
Found patch: 4007,/home/joe/lilypond-git/issue112210043_1.diff,
Trying issue 4008
Found url: http://codereview.appspot.com/115770043
Found patch: 4008,/home/joe/lilypond-git/issue115770043_1.diff,
Fetching, cloning, compiling master.
...
@end smallexample

If run no tracker items with the @var{Patch-New} label are found it will
report:

@example
issues []
@end example

The script can also be run using the tracker issue number(s) as an
argument regardless if the @var{Patch-New} label has been assigned;

@example
test-patches.py 4006
@end example

or

@example
test-patches.py 4006 3992 4020
@end example

The script then checks to see if any previously
@code{make test-baseline}s have been generated and if the commit ID of
@code{remote/origin/master} is different from that previously completed
test.

@noindent
If no previous @code{make test-baseline} test is discovered or if the
commit ID of @code{remote/origin/master} has changed, then a new
@code{make test-baseline} will run first automatically before the patch
is tested:

This shows when the commit ID has changed:

@smallexample
...
(UTC) Begin LilyPond compile, previous commit at        3f92dcb2c81dcd2755542b57a0a5f2039f29a211
Merged master, now at:  4726764cb591f622e7893407db0e7d42bcde90d9
        Success:                ./autogen.sh --noconfigure
        Success:                /tmp/lilypond-autobuild/configure --disable-optimising
        Success:                nice make clean
        Success:                nice make -j3 CPU_COUNT=3
        Success:                nice make test-baseline -j3 CPU_COUNT=3
        Success:                nice make doc -j3 CPU_COUNT=3
        Success:                nice make doc-clean
...
@end smallexample

@noindent
If a previous regression test @emph{is} discovered @emph{and} if the
commit ID of @code{remote/origin/master} has not changed, then the patch
will be tested against the previous @code{make test-baseline} without
the need to re-generate a new one:

@smallexample
...
issues [4009]
Trying issue 4009
Found url: http://codereview.appspot.com/110540043
Found patch: 4009,/home/joe/lilypond-extra/patches/issue110540043_1.diff,
Fetching, cloning, compiling master.
(UTC) Begin LilyPond compile, previous commit at        4726764cb591f622e7893407db0e7d42bcde90d9
        Success:                No new commits in master
Using test baseline from previous build.
...
@end smallexample

The patch is then applied and a @code{make} and @code{make check} are
run.  A full @code{make doc} is also run if the
@file{.lilypond-patchy-config} file has been edited accordingly;

@smallexample
...
Issue 4009:
Issue 4009: Testing patch issue110540043_1.diff
        Success:                git apply --index /home/joe/lilypond-git/issue112210043_1.diff
        Success:                ./autogen.sh --noconfigure
        Success:                /tmp/lilypond-autobuild/configure --disable-optimising
        Success:                nice make clean
        Success:                nice make -j3 CPU_COUNT=3
        Success:                nice make check -j3 CPU_COUNT=3
        Success:                nice make doc -j3 CPU_COUNT=3
...
@end smallexample

Once all the tests have run (successfully or not), the script will clean
up from the previous patch and, if required, start testing the next
issue;

@smallexample
...
Issue 4007: Cleaning up
        Success:                nice make test-clean
        Success:                nice make doc-clean
        Success:                nice make clean
        Success:                git reset --hard
Issue 4007: Done.
Issue 4008:
Issue 4008: Testing patch issue115770043_1_diff
        Success:                git apply --index /home/joe/lilypond-git/issue115770043_1.diff
        Success:                ./autogen.sh --noconfigure
...
@end smallexample

and so on.

@unnumberedsubsubsec Checking the regression test results

Assuming the patch passed all the @code{make} tests, the regression
differences will be located in the @file{/test-results/} directory
within the build location for the patch issue number;

@example
/tmp/show-4007/test-results/
@end example

Open @file{index.html} in a browser of your choice to view any
differences.

@noindent
Alternatively if the Firefox browser is installed, then the regression
test results can be opened by calling the appropriate
@file{show-regtests-} file located in the auto-compile log location;

@example
sh ~/lilypond-auto-compile-results/show-regtests-4007
@end example

See @ref{Regression tests}.

@unnumberedsubsec Reporting test results

Once a patch has been tested and the regression tests have been
manually checked, the tracker can be updated manually by editing the
tracker issue directly in the web browser or by using two additional
python scripts that are included as part of the Patchy suite.

@unnumberedsubsubsec For patches that have passed

Use the @code{accept-patch.py} script and run it with the Google issue
tracker number (not the Rietveld issue number) as an argument;

@example
accept-patch.py 4007
@end example

This will automatically update the tracker issue with the phrase
@qq{Patchy the autobot says: passes tests.}.

@noindent
It is also possible to add additional information to the default
message by adding a second argument within double-quote marks.

@example
accept-patch.py 4007 "This also includes a full documentation build."
@end example

The tracker issue's label is then changed automatically to
@qq{Patch-review}.

@unnumberedsubsubsec Patches that have failed

Use the @code{reject-patch.py} script and run it with the Google issue
tracker number (not the Rietveld issue number) as an argument but you
@emph{must} also include a second argument, in double-quotes, stating
the reason the patch has been rejected;

@example
reject-patch.py 4007 "Fails the 'make check' test."
@end example

Once the @code{reject-patch.py} script has been run, the tracker issue's
label is changed automatically to @qq{Patch-Needs_work}.


@node Automating Patchy
@subsection Automating Patchy

To run as a cron job make sure you have;

@example
[notification]
notify_non_action = no
@end example

in @file{$HOME/.lilypond-patchy-config} to avoid any unintentional email
flooding:

Assuming that Patchy run a user @qq{patchy}, create a file called
@file{$HOME/lilypond-patchy.cron}, adapting it as necessary (the
@code{/2} means @qq{run this every 2 hours}):

@smallexample
02 0-23/2 * * * /home/patchy/lilypond-extra/patches/lilypond-patchy-staging.py
@end smallexample

@warning{@code{cron} will not inherit environment variables so you must
re-define any variables inside @file{$HOME/lilypond-patchy.cron}.  For
instance, @var{LILYPOND_GIT} may need to be defined if
@var{git_repository_dir} is not correctly set in
@file{$HOME/.lilypond-patchy-config}.}

Finally, apply the cron job (you may need superuser privileges for
this):

@example
crontab -u patchy /home/patchy/lilypond-patchy.cron
@end example


@node Troubleshooting Patchy
@subsection Troubleshooting Patchy

The following is a list of the most common messages that the scripts
may report with explanations.

@itemize

@item
@example
issues []
@end example

@itemize

@item
There are currently no tracker issues with the @code{Patch-New} status.

@item
If specific tracker issue number has been used as an argument when
running @code{test-patches.py}, then the issue contains no URL to
Rietveld.

@end itemize

@item
@smallexample
this Git revision has already been pushed by an operator other than this Patchy.
@end smallexample

@itemize

@item
Another, remote, machine has already tested and pushed the new commits
in staging.

@item
You may also see this on your local machine if the auto-build files
have been deleted and this computer has previously already pushed the
listed commit ID to @code{master}.

@end itemize



@item
@smallexample
Git revision has not changed but checksum of test baseline has, must rebuild.
@end smallexample

This occurs when Patchy detects that the commit ID has not changed
since the last test but it cannot locate the last
@code{make test-baseline} (usually because it has been deleted or moved)
and so a new @code{test-baseline} is rebuilt.

@item
@example
Last patch for issue xxxx already tested or under testing
by another Patchy instance, skipping.
@end example

@itemize

@item
There is another instance of Patchy running on your computer that is
testing the same tracker issue.

@item
A previous test attempt was unsuccessful for some reason and the scripts
were not able to tidy up after themselves (for example if you manually
halt the testing process by killing it or closing the terminal you may
have been running the script in).

@noindent
There is a hidden file located in the @code{$HOME} directory of the user
running Patchy called @code{.lilypond-patchy-cache} that records the
current patches that are being tested, have been tested and the commit
ID of @code{remote/origin/master} since the last test.  It will contain
a line like this:

@example
[3995]
issue105560044_120001_diff = testing
@end example

for any issue that it thinks is still in the process of being tested.

@noindent
Manually remove this entry and re-run the script.

@end itemize

@item
@example
test-master-lock and PID entry exist but previous Patchy
run (PID xxxxx) died, resetting test-master-lock anyway.
@end example

@item
A previous test attempt was unsuccessful for some reason and the scripts
were not able to tidy up after themselves (for example if you manually
halt the testing process by killing it or closing the terminal you may
have been running the script in).  The @code{test-master-lock} branch
was therefore not able to be deleted cleanly however, nothing needs to
be done the scripts will rebuild any tests it needs to.

@item
@example
fatal: A branch named 'test-master-lock' already exists.
@end example

@itemize

@item
There is another instance of Patchy running on your computer that is
testing the same tracker issue.

@item
A previous test attempt was unsuccessful for some reason and the scripts
were not able to tidy up after themselves (for example if you manually
halt the testing process by killing it or closing the terminal you may
have been running the script in).  The @code{test-master-lock} branch
was therefore not able to be deleted cleanly, in this case you must
manually delete the @code{test-master-lock} branch in your
@code{$LILYPOND_GIT} directory.

@example
git branch -d test-master-lock
@end example

@noindent
It may be wise to also manually delete @code{test-master} and
@code{test-staging} too, just to be safe.

@end itemize

@item
@example
*** FAILED STEP ***
        merge from staging
        Another instance (PID xxxxx) is already running.
@end example

@noindent
This occurs when trying to run @code{lilypond-patchy-staging.py} when
another instance of either script is already running locally.

@item
@example
Warning: something went wrong; omitting patch for issue 3976
@end example

@itemize

@item
The Rietveld URL as listed in the tracker is incorrect (e.g. missing or
incorrect issue number

@item
The patch on Rietveld is too large to download

@end itemize

@end itemize


@node Administrative mailing list
@section Administrative mailing list

A mailing list for administrative issues is maintained at
@code{lilypond-hackers@@gnu.org}.

This list is intended to be used for discussions that should be kept
private. Therefore, the archives are closed to the public.

Subscription to this list is limited to certain senior developers.

At the present time, the list is dormant.

Details about the criteria for membership, the types of discussion
to take place on the list, and other policies for the hackers list
will be finalized during the
@ref{Grand Organization Project (GOP)}.



@node Grand Organization Project (GOP)
@section Grand Organization Project (GOP)

GOP has two goals:

@itemize
@item
Clarify the various development tasks by writing down the policies
and techniques and/or simplifying the tasks directly.

@item
Get more people involved in development: specifically, find people
to do easy tasks to allow advanced developers to concentrate on
difficult tasks.

@end itemize

@menu
* Motivation::
* Ongoing jobs::
* Policy decisions::
* Policy decisions (finished)::
@end menu

@node Motivation
@subsection Motivation

Most readers are probably familiar with the LilyPond Grand
Documentation Project, which ran from Aug 2007 to Aug 2008. This
project involved over 20 people and resulted in an almost complete
rewrite of the documentation. Most of those contributors were
normal users who decided to volunteer their time and effort to
improve lilypond for everybody. By any measure, it was a great
success.

The Grand Organization Project aims to do the same thing with a
larger scope -- instead of focusing purely on documentation, the
project aims to improve all parts of LilyPond and its community.
Just as with GDP, the main goal is to encourage and train users to
become more involved.

If you have never contributed to an open-source project before --
especially if you use Windows or OSX and do not know how to
program or compile programs -- you may be wondering if there's
anything you can do. Rest assured that you @emph{can} help.

@subheading "Trickle-up" development

One of the reasons I'm organizing GOP is "trickle-up"
development.  The idea is this: doing easy tasks frees up advanced
developers to do harder tasks.  Don't ask "am I the @emph{best}
person for this job"; instead, ask "am I @emph{capable} of doing
this job, so that the current person can do stuff I @emph{can't}
do?".

For example, consider lilypond's poor handling of grace notes in
conjunction with clef and tempo changes. Fixing this will require
a fair amount of code rewriting, and would take an advanced
developer a few weeks to do. It's clearly beyond the scope of a
normal user, so we might as well sit back and do nothing, right?

No; we @emph{can} help, indirectly. Suppose that our normal user
starts answering more emails on lilypond-user. This in turn means
that documentation writers don't need to answer those emails, so
they can spend more time improving the docs. I've noticed that all
doc writers tackle harder and harder subjects, and when they start
writing docs on scheme programming and advanced tweaks, they start
contributing bug fixes to lilypond. Having people performing these
easy-to-moderate bug fixes frees up the advanced developers to
work on the really hard stuff... like rewriting the grace note
code.

Having 1 more normal user answering emails on lilypond-user won't
have a dramatic @q{trickle-up} effect all by itself, of course. But if
we had 8 users volunteering to answer emails, 6 users starting to
write documentation, and 2 users editing LSR... well, that would
free up a lot of current bug-fixing-capable contributors to focus
on that, and we could start to make a real dent in the number of
bugs in lilypond. Quite apart from the eased workload, having that
many new helpers will provide a great moral boost!

@node Ongoing jobs
@subsection Ongoing jobs

Although GOP is a short-term project, the main goal is to train
more people to handle ongoing jobs. The more people doing these
jobs, the lighter the work will be, and the more we can get done
with lilypond!

Also, it would be nice if we had at least one "replacement" /
"understudy" for each role -- too many tasks are only being done
by one person, so if that person goes on vacation or gets very
busy with other matters, work in that area grinds to a halt.

@subheading Jobs for normal users

@itemize
@item Consultant:
LilyPond is sometimes critized for not listening to users, but
whenever we ask for opinions about specific issues, we never get
enough feedback. This is somewhat aggravating.
We need a group of users to make a dedicated effort to test and
give feedback. If there's new documentation, read it. If there's
an experimental binary, download it and try compiling a score with
it. If we're trying to name a new command, think about it and give
serious suggestions.

@item lilypond-user support:
I think it would be nice if we had an official team of users
helping other users.

@item LilyPond Report:
Keeping a monthly newsletter running is a non-trivial task.  A lot
of work is needed to organize it; it would be great if we could
split up the work. One person could write the Snippet of the
Month, another person could do Quotes of the Month, another person
could do interviews, etc.

@item Documentation:
Although GDP (the Grand Documentation Project) did great work,
there's still many tasks remaining.

@item Translations:
Keeping the documentation translations is a monumental task; we
need all the help we can get!

@end itemize

@subheading Jobs for advanced users for developers

@itemize
@item Git help for writers:
We often receive reports of typos and minor text updates to the
documentation. It would be great if somebody could create
properly-formatted patches for these corrections.

Technical requirements: ability to run @ref{LilyDev}.

@item LSR editor:
LSR contains many useful examples of lilypond, but some snippets
are out of date and need updating. Other snippets need to be
advertized, and new snippets need to be sorted. We could use
another person to handle LSR.

Technical requirements: use of a web browser. LilyPond
requirements: you should be familiar with most of Notation
chapters 1 and 2 (or be willing to read the docs to find out).

@item Join the Frogs:
"Frogs" are a team of bug-fixers (because frogs eat bugs, and you
often find them in Ponds of Lilies) and new feature implementors.

Technical requirements: development environment (such as
@ref{LilyDev}), ability to read+write scheme and/or C++ code.

@end itemize


@node Policy decisions
@subsection Policy decisions

There are a number of policy decisions -- some of them fairly
important -- which we have been postponing for a few years.  We
are now discussing them slowly and thoroughly; agenda and exact
proposals are online:

@example
@uref{http://lilypond.org/~graham/gop/index.html}
@end example

Below is a list of policies which are not @qq{on the agenda} yet.

Note that the presence of an item on this list does @emph{not}
mean that everybody thinks that something needs to be done.
Inclusion in this simply means that one developer thinks that we
should discuss it.  We are not going to filter this list; if any
developer thinks we should discuss something, just add it to the
bottom of the list.  (the list is unsorted)

As GOP progresses, items from this list will be put on the agenda
and removed from this list.  I generally try to have one month's
discussion planned in advance, but I may shuffle things around to
respond to any immediate problems in the developer community.

There are some item(s) not displayed here; these are questions
that were posed to me privately, and I do not feel justified in
discussing them publicly without the consent of the person(s) that
brought them up. They will initially be discussed privately on the
lilypond-hackers mailing list -- but the first question will be
"do we absolutely need to do this privately", and if not, the
discussion will take place on lilypond-devel like the other items.

In most policy discussions in lilypond over the past few years,
the first half (or more) is wasted arguing on the basis of
incorrect or incomplete data; once all the relevant facts are
brought to light, the argument is generally resolved fairly
quickly.  In order to keep the GOP discussions focused, each topic
will be introduced with a collection of relevant facts and/or
proposals.  It is, of course, impossible to predict exactly which
facts will be relevant to the discussion -- but spending an hour
or two collecting information could still save hours of
discussion.

@warning{The estimated time required for "prep work", and the
following discussion, has been added to each item.  At the moment,
there is an estimated 30 hours of prep work and 140 hours of
discussion.}

@itemize
@item @strong{Patch reviewing}:
At the time of this writing, we have 23 (known) patches waiting
for review.  Some from main developers; some from new developers.
We desperately need more people helping with lilypond, but
ignoring patches is the best way to drive potential contributors
away.  This is not good.

(prep: 2 hours.  discuss: 10 hours)

@item @strong{Official links to other organizations?}:
There's something called the "software freedom conservancy", and
in general, there's a bunch of "umbrella organizations". Joining
some of these might give us more visibility, possibly leading to
more users, more developers, maybe even financial grants or use in
schools, etc.

(prep: 2 hours.  discuss: 5 hours)

@item @strong{Issue tracking with google code}:
We use the google issue tracker, but this means that we are
relying on a commercial entity for a large part of our
development. Would it be better (safer in the long run) to use the
savannah bug tracker?

(prep: 1 hour.  discuss: 5 hours)

@item @strong{Patch review tool}:
Reitveld is inconvenient in some respects: it requires a google
account, and there's no way to see all patches relating to
lilypond. Should we switch to something like gerritt?
@uref{http://code.google.com/p/lilypond/issues/detail?id=1184}

(prep: 5 hours.  discuss: 15 hours)

@item @strong{Clarity for sponsorships}:
We currently do not advertize bounties and sponsorships on the
webpage.  How much advertising do we want, and what type?
Should we change the "structure" / "framework" for bounties?

(prep: 2 hours.  discuss: 10 hours)

@item @strong{code readability}:
"Our aim when producing source code for Lilypond in whatever
language is that it should be totally comprehensible to a
relatively inexperienced developer at the second reading."

Rationale:
- aids maintainability of code base
- "second reading" so newer developers can look up unfamiliar
  stuff
- will help to keep things simple, even if the code is doing
  complex stuff discourages "secret squirrel" coding, e.g.  "how
  much functionality can I squeeze into as few characters as
  possible" "comments are for wimps"
- will aid not *discouraging* new developers to join the project

(prep: 2 hours.  discuss: 10 hours)

@item @strong{C++ vs. scheme}:
what should be in scheme, what should be in C++, what can/should
be ported from one to the other, etc.  Questions of
maintainability, speed (especially considering guile 2.0), and the
amount of current material in either form, are important.

(prep: 5 hours.  discuss: 15 hours)

@item @strong{always make an issue number for patches}:
there is a proposal that we should always have a google code issue
number for every patch.  This proposal is closely tied to our
choice of patch review tool; if we switch to a different tool (as
suggested in a different proposal), this proposal may become moot.

(prep: 1 hour.  discuss: 5 hours)

@item @strong{initalizer lists}:
shoudl we use initalizer lists for C++?  AFAIK they make no
difference for built-in types, but there's some weird case where
it's more efficient for objects, or something.

Probably not worth making this a weekly thing on its own, but we
can probably wrap it up with some other code-related questions.

(prep: 15 minutes.  discuss: 3 hours)

@end itemize

@node Policy decisions (finished)
@subsection Policy decisions (finished)

Here is a record the final decisions, along with links to the
discussions.

@menu
* GOP-PROP 1 - python formatting::
* GOP-PROP 2 - mentors and frogs::
* GOP-PROP 3 - C++ formatting::
* GOP-PROP 4 - lessons from 2.14::
* GOP-PROP 5 - build system output (not accepted)::
* GOP-PROP 6 - private mailing lists::
* GOP-PROP 7 - developers as resources::
* GOP-PROP 8 - issue priorities::
* GOP-PROP 9 - behavior of make doc::
@end menu

@node GOP-PROP 1 - python formatting
@subsubsection GOP-PROP 1 - python formatting

We will follow the indentation described in PEP-8.
@uref{http://www.python.org/dev/peps/pep-0008/}

@itemize
@item
use 4 spaces per indentation level

@item
never mix tabs and spaces (for indentation)

@item
Code indented with a mixture of tabs and spaces should be
converted to using spaces exclusively

Once this is done, we should add @code{python -tt} to the build
system to avoid such errors in the future.

@end itemize

There should be absolutely no tab characters for indentation in
any @code{.py} file in lilypond git.  All such files should be
converted to use spaces only.

@subsubheading Discussions

@smallexample
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00060.html}
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00084.html}
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00310.html}
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00574.html}
@end smallexample


@node GOP-PROP 2 - mentors and frogs
@subsubsection GOP-PROP 2 - mentors and frogs

Nothing much was decided.  The list of responsibilities was
slightly altered; see the new one in @ref{Mentors}.  We should
encourage more use of the Frogs mailing list.  There's a list of
contributor-mentor pairs in:

@smallexample
@uref{https://github.com/gperciva/lilypond-extra/blob/master/people/mentors.txt}
@end smallexample

That's pretty much it.

@subsubheading Discussions

@smallexample
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00311.html}
@uref{}
@uref{}
@end smallexample


@node GOP-PROP 3 - C++ formatting
@subsubsection GOP-PROP 3 - C++ formatting

Speaking academically, C++ code style is a "solved problem". Let's
pick one of the existing solutions, and let a computer deal with
this.  Humans should not waste their time, energy, and creativity
manually adding tabs or spaces to source code.

We have modified @code{fixcc.py} to use astyle, along with extra
regex tweaks.

@itemize
@item
the final script will be run @strong{blindly} on the lilypond
source code.  We will accept whatever formatting the final version
of this script produces, with no manual tweaking.

@item
patches which have been run through this tool will not be rejected
for style reasons.  Any code formatting @qq{desires} which are not
enforced by @code{fixcc.py} will not be considered grounds for
rejecting a patch.

@item
for now, this style will not be enforced.  It is not cause for
concern if patches which do not follow the formatting done by
@code{fixcc.py} are pushed.  From time to time, Graham will run
the formatter on the entire code base, and commit the resulting
changes.

In a few months, we will tighten up this policy item (with some
sort of automatic processing), but that is outside the scope of
this policy item and is a matter for later discussion.

@item
after the proposal is accepted, we will leave some time for
existing patches to be accepted and pushed.  The script was
run on the source code on @strong{2011 August 01}.

@end itemize

@subheading GNU code

LilyPond is a GNU project, so it makes sense to follow the GNU
coding standards.  These standards state:

@quotation
We don’t think of these recommendations as requirements, because
it causes no problems for users if two different programs have
different formatting styles.

But whatever style you use, please use it consistently, since a
mixture of styles within one program tends to look ugly. If you
are contributing changes to an existing program, please follow the
style of that program. 
@end quotation

(@uref{http://www.gnu.org/prep/standards/html_node/Formatting.html})

With that in mind, we do not think that we must blindly follow the
formatting given by the currrent version of Emacs.

@subheading Implementation notes

We can avoid some of the style change pollution in git history by
ignoring whitespaces changes:

@example
git diff -w
@end example

@subsubheading Discussions

@smallexample
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00526.html}
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00796.html}
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00200.html}
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00525.html}
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00751.html}
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00751.html}
@end smallexample


@node GOP-PROP 4 - lessons from 2.14
@subsubsection GOP-PROP 4 - lessons from 2.14

@subheading History

A brief history of releases:

@multitable @columnfractions .2 .2 .3
@headitem date (YYYY-MM-DD) @tab version @tab comment
@item 2008-10-28 @tab 2.11.63 @tab nobody checking regtests
@item 2008-11-17 @tab 2.11.64
@item 2008-11-29 @tab 2.11.65
@item 2008-12-23 @tab 2.12.0
@item 2009-01-01 @tab @tab somewhere around here, Graham becomes
officially release manager, but Han-Wen still builds the actual
releases
@item 2009-01-01 @tab 2.12.1
@item 2009-01-25 @tab 2.12.2
@item 2009-02-28 @tab 2.13.0
@item 2009-06-01 @tab 2.13.1 @tab note jump in time!
@item 2009-06-27 @tab 2.13.2 @tab first Graham release?
@item 2009-07-03 @tab 2.13.3
@item 2009-09-09 @tab @tab Graham arrives in Glasgow, gets a
powerful desktop computer, and begins serious work on GUB (sending
bug reports to Jan).  It takes approximately 100 hours until GUB
is stable enough to make regular releases.
@item 2009-09-24 @tab 2.13.4
@item 2009-10-02 @tab 2.13.5
@item 2009-10-22 @tab 2.13.6
@item 2009-11-05 @tab 2.13.7
@item ...
@item 2010-01-13 @tab 2.12.3
@item ...
@item 2010-03-19 @tab 2.13.16 @tab Bug squad starts doing a few
regtest comparisons, but IIRC the effort dies out after a few
weeks (BLUE)
@item ...
@item 2010-08-04 @tab 2.13.29 @tab Phil starts checking regtests (BLUE)
@item ...
@item 2011-01-12 @tab 2.13.46 @tab release candidate 1 (GREEN)
@item ...
@item 2011-05-30 @tab 2.13.63 @tab release candidate 7 (GREEN)
@item 2011-06-06 @tab 2.14.0
@end multitable

@c A graphical display of bugs:
@c 
@c @image{bugs-2.13-visualization,png}
@c @image{zoom-2.13-visualization,png}

@subheading Carl's analysis of the bugs

A @file{csv} spreadsheet is available.

@smallexample
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00852.html}
@end smallexample

@example
@uref{lilypond-issues-analysis.csv}
@uref{lilypond-issues-analysis-trim-duplicates.csv}
@end example

There 148 issues marked with Priority=Critical in the tracker.

I've done an analysis, and it looks to me like there was initially
a backlog of critical issues that weren't fixed, and little work
was being done to eliminate critical issues.

Somewhere about 2010-08-01, critical issues started to disappear,
but occasional new ones appeared.

There were a couple of major changes that introduced unanticipated
regressions (new spacing code, beam collision avoidance).  These
produced more than the expected number of regressions.

It appears to me that we didn't really get serious about
eliminating critical bugs until about 2010-06-15 or so.  After
that point, the number of critical bugs more-or-less steadily
decreased until we got to a release candidate.

Of particular interest, the first release candidate of 2.14 was
released on 2011-01-12.  Over the next 10 days, about a dozen bugs
were reported and fixed.  Release candidate 2 came out on
2011-02-09.   No surge of bugs occurred with this release.
Candidate 3 came out on 2011-03-13; we got 2 bugs per week.
Candidate 4 came out on 2011-03-29; 2 new bugs.  Candidate 6 came
out on 2011-04-07.  We got a couple of bugs per week.

@subheading Notes, commentary, and opinions

@example
Han-Wen: Overall, I think this cycle took too long
Mike: I agree
Graham: +1
@end example

@subsubheading Discussions

@smallexample
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00797.html}
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00364.html}
@uref{}
@end smallexample


@node GOP-PROP 5 - build system output (not accepted)
@subsubsection GOP-PROP 5 - build system output (not accepted)

This proposal was too broad; after a month of discussion, Graham
withdrew the proposal.  Portions of it will be introduced in later
proposals.

@subsubheading Discussions

@smallexample
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00320.html}
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00527.html}
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00753.html}
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg01042.html}
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00116.html}
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00310.html}
@end smallexample


@node GOP-PROP 6 - private mailing lists
@subsubsection GOP-PROP 6 - private mailing list

Potentially sensitive or private matters will be referred to
Graham.  He will then decide who should discuss the matter on an
ad-hoc basis, and forward or CC them on future emails.

For emphasis, the project administrators are Han-Wen, Jan, and
Graham; those three will always be CC'd on any important
discussions.

The lilypond-hackers mailing list will be removed.

@subheading History

There is some unhappy history about this idea in our development
community:

@example
@uref{http://lists.gnu.org/archive/html/lilypond-devel/2010-09/msg00178.html}
@uref{http://news.lilynet.net/spip.php?article121}
@uref{http://lists.gnu.org/archive/html/lilypond-devel/2010-11/msg00076.html}
@end example

@subheading Other projects

The idea of private mailing lists is hardly uncommon in
open-source software.  For example,

@example
@uref{http://lwn.net/Articles/394660/}   about debian-private
@uref{http://subversion.apache.org/mailing-lists.html}  private@@
@uref{http://www.freebsd.org/administration.html#t-core}
@uref{http://foundation.gnome.org/legal/}   board members pledge
to keep certain matters confidential

every security team of every GNU/Linux distribution and OS
@end example

In fact, Karl Fogel's @qq{Producing Open Source Software}
explicitly suggests a private mailing list for some circumstances:

@example
[on granting commit/push access to a contributor]

But here is one of the rare instances where secrecy is
appropriate. You can't have votes about potential committers
posted to a public mailing list, because the candidate's feelings
(and reputation) could be hurt.

@uref{http://producingoss.com/en/consensus-democracy.html#electorate}
@end example

@subheading Board of governers, voting, etc?

Many projects have an official board of directors, or a list of
@qq{core developers}, with set term limits and elections and
stuff.

I don't think that we're that big.  I think we're still small
enough, and there's enough trust and consensus decisions, that we
can avoid that.  I would rather that we kept on going with
trust+consensus for at least the next 2-3 years, and spent more
time+energy on bug fixes and new features instead of
administrative stuff.

Project administrators are Han-Wen, Jan, and Graham.

@subsubheading Discussions

@smallexample
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00783.html}
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg01004.html}
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00117.html}
@end smallexample


@node GOP-PROP 7 - developers as resources
@subsubsection GOP-PROP 7 - developers as resources

We shall treat developers (and contributors) as
@strong{Independent volunteers}: each person does whatever they
want, whenever they want.  We have busy careers and lives; we make
no expectations of action from anybody (with the exception of the
6 people in @qq{Meister} positions).

@subsubheading Discussions

@smallexample
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg01092.html}
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00087.html}
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00497.html}
@end smallexample


@node GOP-PROP 8 - issue priorities
@subsubsection GOP-PROP 8 - issue priorities

We will delete the @qq{priority} field of the issue tracker
altogether.  The @qq{type} system will be tweaked.

Type-critical:

@itemize

@item
a reproducible failure to build either @code{make} or @code{make
doc}, from an empty build tree, in a first run, if
@code{configure} does not report any errors.

@item
any program behaviour which is @strong{unintentionally} worse than
the previous stable version or the current development version.
Developers may always use the @qq{this is intentional}, or even
the @qq{this is an unavoidable effect of an improvement in another
area}, reason to move this to a different type.

@item
anything which stops contributors from helping out (e.g.
lily-git.tcl not working, source tree(s) not being available,
LilyDev being unable to compile git master, inaccurate
instructions in the Contributor's Guide 2 Quick start).

To limit this scope of this point, we will assume that the
contributor is using the latest LilyDev and has read the relevant
part(s) of the Contributor's Guide.  Problems in other chapters of
the CG are not sufficient to qualify as Type-Critical.

@end itemize

@subsubheading More new/changed types and labels

Unless otherwise specified, the current types and labels will
continue to be used.  The new types introduced by this proposal
are:

@itemize

@item
Type-crash: any segfault, regardless of what the input file looks
like or which options are given.  Disclaimer: this might not be
possible in some cases, for example certain guile programs (we
certainly can't predict if a piece of scheme will ever stop
running, i.e. the halting problem), or if we rely on other
programs (i.e. ghostscript).  If there are any such cases that
make segfault-prevention impossible, we will document those
exceptions (and the issue will remain as a "crash" instead of
"documentation" until the warning has been pushed).

@item
Type-maintainability: anything which makes it difficult for
serious contributors to help out (e.g. difficult to find the
relevant source tree(s), confusing policies, problems with
automatic indentation tools, etc).

@item
Type-ugly: replaces Type-collision, and it will include things
like bad slurs in addition to actual collision.

@end itemize

A new label will be added:

@itemize
@item
(label) Needs_evidence: it is not clear what the correct output
should look like.  We need scans, references, examples, etc.

@end itemize

@subheading Reminding users about stars

We can remind users that they can @qq{star} an issue to indicate
that they care about it.  Since we resolved to treat developers as
independent volunteers, there is no expectation that anybody will
look at those stars, but if any developer want to organize their
work schedule according to the stars, they are welcome to do so.

@subsubheading Discussions

@smallexample
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00019.html}
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00277.html}
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00413.html}
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00624.html}
@uref{}
@end smallexample


@node GOP-PROP 9 - behavior of make doc
@subsubsection GOP-PROP 9 - behavior of make doc

If there are build problems, then it should be easier to find out
why it's failing.  This will be achieved with log files, as well
as possibly including scripts which automatically display portions
of those log files for a failing build.

We will also add targets for building a specific manual (for
quick+easy checking of doc work), as well as for building all
documentation in a specific language (either English or a
translated language).

When you run @code{make doc},

@itemize

@item
All output will be saved to various log files, with the exception
of output directly from @code{make(1)}.

Note that @code{make(1)} refers to a specific executable file on
unix computers, and is not a general term for the build system.

@item
By default, no other output will be displayed on the console, with
one exception: if a build fails, we might display some portion(s)
of log file(s) which give useful clues about the reason for the
failure.

The user may optionally request additional output to be printed;
this is controlled with the @code{VERBOSE=x} flag.  In such cases,
all output will still be written to log files; the console output
is strictly additional to the log files.

@item
Logfiles from calling lilypond (as part of lilypond-book) will go in
the relevant
@file{$LILYPOND_BUILD_DIR/out/lybook-db/12/lily-123456.log} file.  All
other logfiles will go in the @file{$LILYPOND_BUILD_DIR/logfiles/}
directory.

A single @code{make doc} will therefore result in hundreds of log
files.  Log files produced from individual lilypond runs are not
under our control; apart from that, I anticipate having one or two
dozen log files.  As long as it is clear which log file is
associated with which operation(s), I think this is entirely
appropriate.  The precise implementation will be discussed for
specific patches as they appear.

@item
Both stderr and stdout will be saved in @code{*.log}.  The order
of lines from these streams should be preserved.

@item
There will be no additional @qq{progress messages} during the
build process.  If you run @code{make --silent}, a non-failing
build should print absolutely nothing to the screen.

@item
Assuming that the loglevels patch is accepted, lilypond (inside
lilypond-book) will be run with --loglevel=WARN.
@uref{http://codereview.appspot.com/4822055/}

@item
Ideally, a failing build should provide hints about the reason why
it failed, or at least hints about which log file(s) to examine.

@end itemize

If this proposal is accepted, none of these policies will be
assumed to apply to any other aspect of the build system.
Policies for any other aspect of the build system will be
discussed in separate proposals.

@subheading Don't cause more build problems

However, there is a danger in this approach, that vital error
messages can also be lost, thus preventing the cause of the
failure of a make being found.  We therefore need to be
exceptionally careful to move cautiously, include plenty of tests,
and give time for people to experiment/find problems in each stage
before proceeding to the next stage.

This will be done by starting from individual lilypond calls
within lilypond-book, and slowly moving to @qq{larger} targets of
the build system -- after the individual lilypond calls are are
producing the appropriate amount of output and this is saved in
the right place and we can automatically isolate parts of a
failing build, we will work on lilypond-book in general, and only
then will we look at the build system itself.

@subheading Implementation notes

There is an existing make variable QUIET_BUILD, which
alter the amount of output being displayed
(@uref{
http://lilypond.org/doc/v2.15/Documentation/contributor/useful-make-variables}
).  We are not planning on keeping this make variable.

The standard way for GNU packages to give more output is with a
@code{V=x} option.  Presumably this is done by increasing
@code{x}?  If we support this option, we should still write log
files; we would simply print more of the info in those log files
to screen.

The command @code{tee} may be useful to write to a file and
display to stdout (in the case of VERBOSE).


@subsubheading Discussions

@smallexample
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00378.html}
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00703.html}
@end smallexample


@ignore
@n ode GOP-PROP 10 - scheme indentation
@s ubsubsection GOP-PROP 10 - scheme indentation

still under discussion

@subsubheading Discussions

@smallexample
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00625.html}
@uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg01026.html}
@c @uref{}
@end smallexample
@end ignore



@node Grand LilyPond Input Syntax Standardization (GLISS)
@section Grand LilyPond Input Syntax Standardization (GLISS)

@subheading Summary

@itemize
@item
Start: sortly after 2.14 comes out, which is currently estimated
to happen in January 2011.

@item
Length: 6-12 months.  We're not going to rush this.

@item
Goal: define an input which we commit to being
machine-updateable for the forseeable future.  Any future patches
which change the syntax in a non-convert-ly-able format will be
rejected.  (subject to the limitations, below)
Once this is finished, we will release lilypond 3.0.

@end itemize


@subheading The Problem

One of the biggest complaints people have with lilypond -- other
than silly thing like "there's no gui" -- is the changing syntax.
Now, inventing a language or standards is difficult.  If you set
it in stone too soon, you risk being stuck with decisions which
may limit matters.  If you keep on updating the syntax,
interaction with older data (and other programs!) becomes complex.

@subheading Scope and Limitations

@itemize
@item
tweaks will not be included.  Anything with \override, \set,
\overrideProperty, \tweak, \revert, \unset... including even those
command names themselves... is still fair game for NOT_SMART
convert-ly updates.

@item
other than that, everything is on the table.  Is it a problem to
have the tagline inside \header?  What should the default behavior
of \include be?

@item
we need to get standards for command names.  This will help users
remember them, and reduce the options for future names (and
potential renamings later on).  \commandOn and \commandOff seem to
work well (should we *always* have an Off command?), but what
about the "command" part?  Should it be \nounVerbOn, or
\verbNounOn ?  Or \verbNotesWithExtraInformationOn ?

@item
we need standards for the location of commands.  Ligature
brackets, I'm looking at you.  (non-postfix notation must die!)

@item
this Grand Project doesn't affect whether we have a 2.16 or not.
The main problem will be deciding what to do (with a bit of
messiness anticipated for \tuplet); we should definitely release a
2.16 before merging _any_ of these changes.

@item
we obviously can't /guarantee/ that we'll /never/ make any
non-convert-ly changes in the basic format.  But we *can*
guarantee that such changes would force lilypond 4.0, and that we
would only do so for overwhelmingly good reasons.

@end itemize

@subheading Workflow

@itemize
@item
We're going to have lots and lots of emails flying around.  The
vast majority won't really fit into either -devel or -user, so
we'll use a list devoted to syntax issues.

@item
Once we have a serious proposal that gained general acceptance
from the separate syntax mailing list, I'll bring it to -devel.
We're not going to make any changes without discussing it on
-devel, but if we're going to have huge threads about English
grammar and silly ideas, and I don't want to clutter up -devel.
Once whatever chaotic silliness on the syntax list is settled
down, I'll bring the ideas to -devel.

@item
as with GDP, I'll moderate the discussion.  Not as with mailist
moderation, but rather by introducing issues at specific times.
We don't want a free-for-all discussion of all parts of the syntax
at once; nothing will get resolved.

@item
Whenever possible, we'll decide on policies at the highest level
of abstraction.  For example, consider \numericTimeSignature,
\slurUp, \xNotesOn, \startTextSpan, and \verylongfermata.  One of
them starts with the name of the notation first (slur).  One has
an abbreviation (x instead of cross).  One has the verb at the end
(On), another has it at the beginning (start).  The adjective can
come at the beginning (numeric, x) or end (Up).  Most are in
camelCase, but one isn't (verylongfermata).

@item
Instead of arguing about each individual command, we'll decide on
abstract questions.  Should each command begin the notation-noun,
or the verb?  Should all commands be in camelCase, or should we
make everything other than articulations in camelCase but make
articulations all lower-case?  Are abbreviations allowed?

@item
Once we've answered such fundamental questions, most of the syntax
should fall into place pretty easily.  There might be a few odd
questions left ("is it a span, or a spanner?"), but those can be
settled fairly quickly.

@end itemize

@subheading Implementation

Nothing until the project is finished, then we declare the next
stable release (2.16.0 or 2.18.0 ?) to be the final 2.x version,
release it, then apply all the GLISS syntax changes and start
testing a beta for 3.0 a week or two later.

@subheading Discussion

Don't respond to any of the specifics yet.  Yes, we all have our
pet irritations (like "what's up with \paper and \layout?!").
There will be plenty of time to discuss them once GLISS starts.

That said, we have a list of specific items that people really
wanted to have written down.  See @ref{Specific GLISS issues}.

@menu
* Specific GLISS issues::
@end menu


@node Specific GLISS issues
@subsection Specific GLISS issues

@itemize
@item
add regtests for every piece of syntax (not one-command-per-file,
but making a few files which, between them, use every single piece
of syntax.)  This is a great test for convert-ly.

@item
should GLISS cover suggested conventions?  (indentation,
one-bar-per-line, etc -- the kind of stuff we list for the
lilypond formatting in the docs ?)

@item
how much (if any) syntactic sugar should we add?  i.e.
@example
  \instrumentName #'foo
% instead of
  \set Staff.instrumentName
@end example
?  Carl: maybe yes, Neil: no.  (for example, it fails for
pianostaff)

@item
the values that are used as arguments to common used overrides.
Sometimes they are a symbol (e.g. #'around), sometimes a
predefined variable referring to a Scheme value or object (e.g.
#LEFT, #all-visible ). The main trouble is that for novice users
it is not clear when there should be an apostrophe and when not.

@item
When do we need -\command and when is it just \command ?


@item
Command-line options to the lilypond binary.  -dfoo counts as a
tweak; we won't be trying to pin those down.

@item
@verbatim
\layout {
  \context { \Score
% vs.
\layout {
  \context {
    \Score
@end verbatim

@item
If would be pedagogically simpler to realize this difference if
the syntax was separate if you define a context from scratch (as
is the case with \RemoveEmptyStaffContext) or if it's defined by
adding onto an existing context. For example, a syntax like

@verbatim
\context{
 % Copy the current settings of the Staff context:
 \use Staff
 % do whatever additional settings
}
%%% could be used to distinguish from
\context{
 % Take settings from a variable:
 \Variable
 % do whatever additional settings
}

%%% and

\context{
 % Start from scratch:
 \type ...
 \name ...
 \consists ...
 ...
}
@end verbatim

@item
Capitalization of identifiers: \VoiceOne ?

@item
@verbatim
%%% Allow
{ music expression } * 4
%%% instead of
\repeat unfold 4 { music expression }
@end verbatim

?  patch here:
@smallexample
@uref{http://lists.gnu.org/archive/html/lilypond-devel/2010-04/msg00467.html}
@end smallexample

@item
Personally, I find it easier to understand when there's a repeated
8 in the half-bar position; it's much easier to see that you have
two groups of 4:

@example
c8 c c c c8 c c c
%%% instead of one group of eight:
c8 c c c c c c c
@end example

@item
trivially simple bar-lines:

c1 | c1 |

encourage, allow, or discourage, or disallow?

@item
indentation of \\ inside a @{@} construct.


@item
barline checks at the end of line should be preceded by at least 2
spaces?  barline checks should line up if possible (i.e.  if you
can use less than 4, 8, X empty spaces before a barline check to
make them line up?)

@item
Why doesn't \transpose respect \relative mode?


@item
on \score vs. \new Score

But in the light of a consistent syntax and semantic, I see no
reason (from the users POV) to disallow it.  After all, the real
top-level context is a \book @{@}, isn't it, and I don't see a point
in disallowing a \new Score construct just like \new Staff.

From a syntactical POV, I see the following pros for \new Score:
- You can write \with @{ ... @} for every other context but \Score,
which (for consistency) should also work with \new Score.
- When there's a \new Foo Bar, there's also a \context Foo Bar,
  which makes the same as a parallel instantiation of all Bar's.
- [Quoting Rune from
@uref{http://www.mail-archive.com/lilypond-devel@@gnu.org/msg14713.html}
  "I know that the \score-statement is a syntactical construct,
but I think it would be nice to hide this fact from the users.  I
think we could make the use of score-block much more intuitive if
changing the syntax to \new \Score and adding an implicit
sequential-statement to the score."


@item
Discussion on
@uref{http://code.google.com/p/lilypond/issues/detail?id=1322}
about \new vs. \context.


@item
Let users add their own items to the parser?  comment 11 on:
@uref{http://code.google.com/p/lilypond/issues/detail?id=1322}

@item
should engravers be pluralized (note_heads_engraver) or not
(note_head_engraver) ?

@item
should we allow numbers in identifier names?  Issue:
@uref{http://code.google.com/p/lilypond/issues/detail?id=1670}

@item
should we officially allow accented characters?  in general, how
do we feel about utf-8 stuff?

@item
for the sake of completeness/simplicity, what about *disallowing*
the "one-note" form of a music expression?  i.e. only allowing
stuff like
@verbatim
  \transpose c d { e1 }
  \transpose c d << e1 >>
@end verbatim

and never allowing
@verbatim
  \transpose c d e1
@end verbatim

@item
What should be the officially encouraged way of writing music for
transposing instruments? Maybe it should be simplified?
See http://lists.gnu.org/archive/html/lilypond-user/2011-07/msg00130.html

@end itemize


@node Unsorted policies
@section Unsorted policies

@subsubheading Language-specific mailing lists

A translator can ask for an official lilypond-xy mailing list once
they've finished all @qq{priority 1} translation items.

@subsubheading Performing yearly copyright update (@qq{grand-replace})

At the start of each year, copyright notices for all source files
should be refreshed by running the following command from the top of
the source tree:

@example
make grand-replace
@end example

Internally, this invokes the script @file{scripts/build/grand-replace.py},
which performs a regular expression substitution for old-year -> new-year
wherever it finds a valid copyright notice.

Note that snapshots of third party files such as @file{texinfo.tex} should
not be included in the automatic update; @file{grand-replace.py} ignores these
files if they are listed in the variable @code{copied_files}.


@subsubheading Push git access

Git access is given out when a contributor has a significant
record of patches being accepted without problems.  If existing
developers are tired of pushing patches for a contributor, we'll
discuss giving them push access.  Unsolicited requests from
contributors for access will almost always be turned down.