@c -*- coding: utf-8; mode: texinfo; -*- @node Administrative policies @chapter Administrative policies This chapter discusses miscellaneous administrative issues which don't fit anywhere else. @menu * Meta-policy for this document:: * Meisters:: * Patchy:: * Administrative mailing list:: * Grand Organization Project (GOP):: * Grand LilyPond Input Syntax Standardization (GLISS):: * Unsorted policies:: @end menu @node Meta-policy for this document @section Meta-policy for this document The Contributor's Guide as a whole is still a work in progress, but some chapters are much more complete than others. Chapters which are @qq{almost finished} should not have major changes without a discussion on @w{@code{-devel}}; in other chapters, a disorganized @qq{wiki-style dump} of information is encouraged. Do not change (other than spelling mistakes) without discussion: @itemize @item @ref{Introduction to contributing} @item @ref{Working with source code} @end itemize Please dump info in an appropriate @@section within these manuals, but discuss any large-scale reorganization: @itemize @item @ref{Compiling} @item @ref{Documentation work} @item @ref{Issues} @item @ref{Regression tests} @item @ref{Programming work} @end itemize Totally disorganized; do whatever the mao you want: @itemize @item @ref{Website work} @item @ref{LSR work} @item @ref{Release work} @item @ref{Administrative policies} @end itemize @node Meisters @section Meisters We have four jobs for organizing a team of contributors: @itemize @item Bug Meister: trains new Bug Squad volunteers, organizes who works on which part of their job, checks to make sure that everything is running smoothly, and has final say on our policy for Issues. Currently: Phil @item Doc Meister: trains new doc editors/writers, organizes who works on which part of the job, checks to make sure that everything is running smoothly, and has final say on our policy for Documentation. Also includes LSR work. Currently: Graham @item Translation Meister: trains new translators, updates the translation priority list, and handles merging branches (in both directions). Currently: Francisco @item Frog Meister: is responsible for code patches from (relatively) inexperienced contributors. Keeps track of patches, does initial reviewing of those patches, sends them to @w{@code{-devel}} when they've had some initial review on the Frog list, pesters the @w{@code{-devel}} community into actually reviewing said patches, and finally pushes the patches once they're accepted. This person is @emph{not} responsible for training new programmers, because that would be far too much work -- his/her job is @qq{only} to guide completed patches through our process. Currently: Carl @end itemize @node Patchy @section Patchy @subheading Introduction Patchy is a set of Python scripts to automate two administrative tasks: @itemize @item @code{lilypond-patchy-staging.py}: checks that new commits in @code{staging} can compile the regtests and documentation before merging @code{staging} into @code{master}. (completely automatic) @item @code{test-patches.py}: checks that patches apply to git master, compile, and lets a human check that there are no big unintended changes to the regtests. (requires some human input) @end itemize @subheading Installing patchy To install patchy, you should do the following: @enumerate @item Create a new user on your box to run patchy; this is a security step for your own protection. It is recommended that this should not be an administrator. New users are created from System; Administration; Users and Groups. @item Get the patchy scripts from @example @uref{https://github.com/gperciva/lilypond-extra/} @end example Patchy is in the @file{patches/} directory. @item Put the scripts in a sensible place on your system @item Create a new git repository with @example git clone git://git.sv.gnu.org/lilypond.git @end example This will create a directory called lilypond with the repo in it. Make sure it's where you want it and name it lilypond-git (assuming you want to follow the standard naming conventions). @item Create an environment variable called LILYPOND_GIT and make it equal to the location of your new git repo. You can do this by editing @file{$HOME/.profile} and adding the line: @example export LILYPOND_GIT=~/lilypond-git @end example then logging out and in. @item Run patchy once to set up config files. Cancel this build (ctrl-c). @item Edit @file{$HOME/.lilypond-patchy-config} to provide working directories for your build directory, your results directory, compiler options and notification method. If you don't want to use email notification, then delete everything after @code{smtp_command:}. @item Ensure that your new user has git push access. Follow the instructions in the CG at @ref{Commit access}. Do not set password protection for the key - if you do you will not be able to run patchy unattended. @end enumerate @subheading lilypond-patchy-staging.py lilypond-patchy-staging.py is run with @example python lilypond-patchy-staging.py @end example Not much appears to happen except you can see a lot of CPU gets used if you open System Monitor. There's not much point running @code{lilypond-patchy-staging.py} unless there is something in staging to be merged to master, however, if there's nothing in staging then the script won't waste resources by compiling anything. The script fetches the current patches in staging and runs @code{make}, @code{make test} and @code{make doc} to ensure that all of these complete error-free. If you have set patchy up to use email, it emails its results to you. If you haven't, then you can view them in a logfile. It also merges staging into master. @subheading test-patches.py test-patches prepares a regtest comparison for a human to quickly glance at, to determine if the patch is ready for a review. After looking at the comparison (or the lack of a comparison in the case of problems), run @code{accept-patch.py} or @code{reject-patch.py}. Once a patch has gotten a "LGTM" from Patchy, it should be reviewed by relevant developers, and if it passes this, it can be considered for countdown (see @ref{Commits and patches}) and pushing to staging (see @ref{Pushing to staging}). @node Administrative mailing list @section Administrative mailing list A mailing list for administrative issues is maintained at @code{lilypond-hackers@@gnu.org}. This list is intended to be used for discussions that should be kept private. Therefore, the archives are closed to the public. Subscription to this list is limited to certain senior developers. At the present time, the list is dormant. Details about the criteria for membership, the types of discussion to take place on the list, and other policies for the hackers list will be finalized during the @ref{Grand Organization Project (GOP)}. @node Grand Organization Project (GOP) @section Grand Organization Project (GOP) GOP has two goals: @itemize @item Clarify the various development tasks by writing down the policies and techniques and/or simplifying the tasks directly. @item Get more people involved in development: specifically, find people to do easy tasks to allow advanced developers to concentrate on difficult tasks. @end itemize @menu * Motivation:: * Ongoing jobs:: * Policy decisions:: * Policy decisions (finished):: @end menu @node Motivation @subsection Motivation Most readers are probably familiar with the LilyPond Grand Documentation Project, which ran from Aug 2007 to Aug 2008. This project involved over 20 people and resulted in an almost complete rewrite of the documentation. Most of those contributors were normal users who decided to volunteer their time and effort to improve lilypond for everybody. By any measure, it was a great success. The Grand Organization Project aims to do the same thing with a larger scope -- instead of focusing purely on documentation, the project aims to improve all parts of LilyPond and its community. Just as with GDP, the main goal is to encourage and train users to become more involved. If you have never contributed to an open-source project before -- especially if you use Windows or OSX and do not know how to program or compile programs -- you may be wondering if there's anything you can do. Rest assured that you @emph{can} help. @subheading "Trickle-up" development One of the reasons I'm organizing GOP is "trickle-up" development. The idea is this: doing easy tasks frees up advanced developers to do harder tasks. Don't ask "am I the @emph{best} person for this job"; instead, ask "am I @emph{capable} of doing this job, so that the current person can do stuff I @emph{can't} do?". For example, consider lilypond's poor handling of grace notes in conjunction with clef and tempo changes. Fixing this will require a fair amount of code rewriting, and would take an advanced developer a few weeks to do. It's clearly beyond the scope of a normal user, so we might as well sit back and do nothing, right? No; we @emph{can} help, indirectly. Suppose that our normal user starts answering more emails on lilypond-user. This in turn means that documentation writers don't need to answer those emails, so they can spend more time improving the docs. I've noticed that all doc writers tackle harder and harder subjects, and when they start writing docs on scheme programming and advanced tweaks, they start contributing bug fixes to lilypond. Having people performing these easy-to-moderate bug fixes frees up the advanced developers to work on the really hard stuff... like rewriting the grace note code. Having 1 more normal user answering emails on lilypond-user won't have a dramatic @q{trickle-up} effect all by itself, of course. But if we had 8 users volunteering to answer emails, 6 users starting to write documentation, and 2 users editing LSR... well, that would free up a lot of current bug-fixing-capable contributors to focus on that, and we could start to make a real dent in the number of bugs in lilypond. Quite apart from the eased workload, having that many new helpers will provide a great moral boost! @node Ongoing jobs @subsection Ongoing jobs Although GOP is a short-term project, the main goal is to train more people to handle ongoing jobs. The more people doing these jobs, the lighter the work will be, and the more we can get done with lilypond! Also, it would be nice if we had at least one "replacement" / "understudy" for each role -- too many tasks are only being done by one person, so if that person goes on vacation or gets very busy with other matters, work in that area grinds to a halt. @subheading Jobs for normal users @itemize @item Consultant: LilyPond is sometimes critized for not listening to users, but whenever we ask for opinions about specific issues, we never get enough feedback. This is somewhat aggravating. We need a group of users to make a dedicated effort to test and give feedback. If there's new documentation, read it. If there's an experimental binary, download it and try compiling a score with it. If we're trying to name a new command, think about it and give serious suggestions. @item lilypond-user support: I think it would be nice if we had an official team of users helping other users. @item LilyPond Report: Keeping a monthly newsletter running is a non-trivial task. A lot of work is needed to organize it; it would be great if we could split up the work. One person could write the Snippet of the Month, another person could do Quotes of the Month, another person could do interviews, etc. @item Documentation: Although GDP (the Grand Documentation Project) did great work, there's still many tasks remaining. @item Translations: Keeping the documentation translations is a monumental task; we need all the help we can get! @end itemize @subheading Jobs for advanced users for developers @itemize @item Git help for writers: We often receive reports of typos and minor text updates to the documentation. It would be great if somebody could create properly-formatted patches for these corrections. Technical requirements: ability to run @ref{LilyDev}. @item LSR editor: LSR contains many useful examples of lilypond, but some snippets are out of date and need updating. Other snippets need to be advertized, and new snippets need to be sorted. We could use another person to handle LSR. Technical requirements: use of a web browser. LilyPond requirements: you should be familiar with most of Notation chapters 1 and 2 (or be willing to read the docs to find out). @item Join the Frogs: "Frogs" are a team of bug-fixers (because frogs eat bugs, and you often find them in Ponds of Lilies) and new feature implementors. Technical requirements: development environment (such as @ref{LilyDev}), ability to read+write scheme and/or C++ code. @end itemize @node Policy decisions @subsection Policy decisions There are a number of policy decisions -- some of them fairly important -- which we have been postponing for a few years. We are now discussing them slowly and thoroughly; agenda and exact proposals are online: @example @uref{http://lilypond.org/~graham/gop/index.html} @end example Below is a list of policies which are not @qq{on the agenda} yet. Note that the presence of an item on this list does @emph{not} mean that everybody thinks that something needs to be done. Inclusion in this simply means that one developer thinks that we should discuss it. We are not going to filter this list; if any developer thinks we should discuss something, just add it to the bottom of the list. (the list is unsorted) As GOP progresses, items from this list will be put on the agenda and removed from this list. I generally try to have one month's discussion planned in advance, but I may shuffle things around to respond to any immediate problems in the developer community. There are some item(s) not displayed here; these are questions that were posed to me privately, and I do not feel justified in discussing them publicly without the consent of the person(s) that brought them up. They will initially be discussed privately on the lilypond-hackers mailing list -- but the first question will be "do we absolutely need to do this privately", and if not, the discussion will take place on lilypond-devel like the other items. In most policy discussions in lilypond over the past few years, the first half (or more) is wasted arguing on the basis of incorrect or incomplete data; once all the relevant facts are brought to light, the argument is generally resolved fairly quickly. In order to keep the GOP discussions focused, each topic will be introduced with a collection of relevant facts and/or proposals. It is, of course, impossible to predict exactly which facts will be relevant to the discussion -- but spending an hour or two collecting information could still save hours of discussion. @warning{The estimated time required for "prep work", and the following discussion, has been added to each item. At the moment, there is an estimated 30 hours of prep work and 140 hours of discussion.} @itemize @item @strong{Patch reviewing}: At the time of this writing, we have 23 (known) patches waiting for review. Some from main developers; some from new developers. We desperately need more people helping with lilypond, but ignoring patches is the best way to drive potential contributors away. This is not good. (prep: 2 hours. discuss: 10 hours) @item @strong{Official links to other organizations?}: There's something called the "software freedom conservancy", and in general, there's a bunch of "umbrella organizations". Joining some of these might give us more visibility, possibly leading to more users, more developers, maybe even financial grants or use in schools, etc. (prep: 2 hours. discuss: 5 hours) @item @strong{Issue tracking with google code}: We use the google issue tracker, but this means that we are relying on a commercial entity for a large part of our development. Would it be better (safer in the long run) to use the savannah bug tracker? (prep: 1 hour. discuss: 5 hours) @item @strong{Patch review tool}: Reitveld is inconvenient in some respects: it requires a google account, and there's no way to see all patches relating to lilypond. Should we switch to something like gerritt? @uref{http://code.google.com/p/lilypond/issues/detail?id=1184} (prep: 5 hours. discuss: 15 hours) @item @strong{Clarity for sponsorships}: We currently do not advertize bounties and sponsorships on the webpage. How much advertising do we want, and what type? Should we change the "structure" / "framework" for bounties? (prep: 2 hours. discuss: 10 hours) @item @strong{code readability}: "Our aim when producing source code for Lilypond in whatever language is that it should be totally comprehensible to a relatively inexperienced developer at the second reading." Rationale: - aids maintainability of code base - "second reading" so newer developers can look up unfamiliar stuff - will help to keep things simple, even if the code is doing complex stuff discourages "secret squirrel" coding, e.g. "how much functionality can I squeeze into as few characters as possible" "comments are for wimps" - will aid not *discouraging* new developers to join the project (prep: 2 hours. discuss: 10 hours) @item @strong{C++ vs. scheme}: what should be in scheme, what should be in C++, what can/should be ported from one to the other, etc. Questions of maintainability, speed (especially considering guile 2.0), and the amount of current material in either form, are important. (prep: 5 hours. discuss: 15 hours) @item @strong{always make an issue number for patches}: there is a proposal that we should always have a google code issue number for every patch. This proposal is closely tied to our choice of patch review tool; if we switch to a different tool (as suggested in a different proposal), this proposal may become moot. (prep: 1 hour. discuss: 5 hours) @item @strong{initalizer lists}: shoudl we use initalizer lists for C++? AFAIK they make no difference for built-in types, but there's some weird case where it's more efficient for objects, or something. Probably not worth making this a weekly thing on its own, but we can probably wrap it up with some other code-related questions. (prep: 15 minutes. discuss: 3 hours) @end itemize @node Policy decisions (finished) @subsection Policy decisions (finished) Here is a record the final decisions, along with links to the discussions. @menu * GOP-PROP 1 - python formatting:: * GOP-PROP 2 - mentors and frogs:: * GOP-PROP 3 - C++ formatting:: * GOP-PROP 4 - lessons from 2.14:: * GOP-PROP 5 - build system output (not accepted):: * GOP-PROP 6 - private mailing lists:: * GOP-PROP 7 - developers as resources:: * GOP-PROP 8 - issue priorities:: * GOP-PROP 9 - behavior of make doc:: @end menu @node GOP-PROP 1 - python formatting @subsubsection GOP-PROP 1 - python formatting We will follow the indentation described in PEP-8. @uref{http://www.python.org/dev/peps/pep-0008/} @itemize @item use 4 spaces per indentation level @item never mix tabs and spaces (for indentation) @item Code indented with a mixture of tabs and spaces should be converted to using spaces exclusively Once this is done, we should add @code{python -tt} to the build system to avoid such errors in the future. @end itemize There should be absolutely no tab characters for indentation in any @code{.py} file in lilypond git. All such files should be converted to use spaces only. @subsubheading Discussions @smallexample @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00060.html} @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00084.html} @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00310.html} @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00574.html} @end smallexample @node GOP-PROP 2 - mentors and frogs @subsubsection GOP-PROP 2 - mentors and frogs Nothing much was decided. The list of responsibilities was slightly altered; see the new one in @ref{Mentors}. We should encourage more use of the Frogs mailing list. There's a list of contributor-mentor pairs in: @smallexample @uref{https://github.com/gperciva/lilypond-extra/blob/master/people/mentors.txt} @end smallexample That's pretty much it. @subsubheading Discussions @smallexample @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00311.html} @uref{} @uref{} @end smallexample @node GOP-PROP 3 - C++ formatting @subsubsection GOP-PROP 3 - C++ formatting Speaking academically, C++ code style is a "solved problem". Let's pick one of the existing solutions, and let a computer deal with this. Humans should not waste their time, energy, and creativity manually adding tabs or spaces to source code. We have modified @code{fixcc.py} to use astyle, along with extra regex tweaks. @itemize @item the final script will be run @strong{blindly} on the lilypond source code. We will accept whatever formatting the final version of this script produces, with no manual tweaking. @item patches which have been run through this tool will not be rejected for style reasons. Any code formatting @qq{desires} which are not enforced by @code{fixcc.py} will not be considered grounds for rejecting a patch. @item for now, this style will not be enforced. It is not cause for concern if patches which do not follow the formatting done by @code{fixcc.py} are pushed. From time to time, Graham will run the formatter on the entire code base, and commit the resulting changes. In a few months, we will tighten up this policy item (with some sort of automatic processing), but that is outside the scope of this policy item and is a matter for later discussion. @item after the proposal is accepted, we will leave some time for existing patches to be accepted and pushed. The script was run on the source code on @strong{2011 August 01}. @end itemize @subheading GNU code LilyPond is a GNU project, so it makes sense to follow the GNU coding standards. These standards state: @quotation We don’t think of these recommendations as requirements, because it causes no problems for users if two different programs have different formatting styles. But whatever style you use, please use it consistently, since a mixture of styles within one program tends to look ugly. If you are contributing changes to an existing program, please follow the style of that program. @end quotation (@uref{http://www.gnu.org/prep/standards/html_node/Formatting.html}) With that in mind, we do not think that we must blindly follow the formatting given by the currrent version of Emacs. @subheading Implementation notes We can avoid some of the style change pollution in git history by ignoring whitespaces changes: @example git diff -w @end example @subsubheading Discussions @smallexample @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00526.html} @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00796.html} @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00200.html} @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00525.html} @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00751.html} @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00751.html} @end smallexample @node GOP-PROP 4 - lessons from 2.14 @subsubsection GOP-PROP 4 - lessons from 2.14 @subheading History A brief history of releases: @multitable @columnfractions .2 .2 .3 @headitem date (YYYY-MM-DD) @tab version @tab comment @item 2008-10-28 @tab 2.11.63 @tab nobody checking regtests @item 2008-11-17 @tab 2.11.64 @item 2008-11-29 @tab 2.11.65 @item 2008-12-23 @tab 2.12.0 @item 2009-01-01 @tab @tab somewhere around here, Graham becomes officially release manager, but Han-Wen still builds the actual releases @item 2009-01-01 @tab 2.12.1 @item 2009-01-25 @tab 2.12.2 @item 2009-02-28 @tab 2.13.0 @item 2009-06-01 @tab 2.13.1 @tab note jump in time! @item 2009-06-27 @tab 2.13.2 @tab first Graham release? @item 2009-07-03 @tab 2.13.3 @item 2009-09-09 @tab @tab Graham arrives in Glasgow, gets a powerful desktop computer, and begins serious work on GUB (sending bug reports to Jan). It takes approximately 100 hours until GUB is stable enough to make regular releases. @item 2009-09-24 @tab 2.13.4 @item 2009-10-02 @tab 2.13.5 @item 2009-10-22 @tab 2.13.6 @item 2009-11-05 @tab 2.13.7 @item ... @item 2010-01-13 @tab 2.12.3 @item ... @item 2010-03-19 @tab 2.13.16 @tab Bug squad starts doing a few regtest comparisons, but IIRC the effort dies out after a few weeks (BLUE) @item ... @item 2010-08-04 @tab 2.13.29 @tab Phil starts checking regtests (BLUE) @item ... @item 2011-01-12 @tab 2.13.46 @tab release candidate 1 (GREEN) @item ... @item 2011-05-30 @tab 2.13.63 @tab release candidate 7 (GREEN) @item 2011-06-06 @tab 2.14.0 @end multitable @c A graphical display of bugs: @c @c @image{bugs-2.13-visualization,png} @c @image{zoom-2.13-visualization,png} @subheading Carl's analysis of the bugs A @file{csv} spreadsheet is available. @smallexample @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00852.html} @end smallexample @example @uref{lilypond-issues-analysis.csv} @uref{lilypond-issues-analysis-trim-duplicates.csv} @end example There 148 issues marked with Priority=Critical in the tracker. I've done an analysis, and it looks to me like there was initially a backlog of critical issues that weren't fixed, and little work was being done to eliminate critical issues. Somewhere about 2010-08-01, critical issues started to disappear, but occasional new ones appeared. There were a couple of major changes that introduced unanticipated regressions (new spacing code, beam collision avoidance). These produced more than the expected number of regressions. It appears to me that we didn't really get serious about eliminating critical bugs until about 2010-06-15 or so. After that point, the number of critical bugs more-or-less steadily decreased until we got to a release candidate. Of particular interest, the first release candidate of 2.14 was released on 2011-01-12. Over the next 10 days, about a dozen bugs were reported and fixed. Release candidate 2 came out on 2011-02-09. No surge of bugs occurred with this release. Candidate 3 came out on 2011-03-13; we got 2 bugs per week. Candidate 4 came out on 2011-03-29; 2 new bugs. Candidate 6 came out on 2011-04-07. We got a couple of bugs per week. @subheading Notes, commentary, and opinions @example Han-Wen: Overall, I think this cycle took too long Mike: I agree Graham: +1 @end example @subsubheading Discussions @smallexample @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00797.html} @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00364.html} @uref{} @end smallexample @node GOP-PROP 5 - build system output (not accepted) @subsubsection GOP-PROP 5 - build system output (not accepted) This proposal was too broad; after a month of discussion, Graham withdrew the proposal. Portions of it will be introduced in later proposals. @subsubheading Discussions @smallexample @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00320.html} @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00527.html} @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00753.html} @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg01042.html} @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00116.html} @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00310.html} @end smallexample @node GOP-PROP 6 - private mailing lists @subsubsection GOP-PROP 6 - private mailing list Potentially sensitive or private matters will be referred to Graham. He will then decide who should discuss the matter on an ad-hoc basis, and forward or CC them on future emails. For emphasis, the project administrators are Han-Wen, Jan, and Graham; those three will always be CC'd on any important discussions. The lilypond-hackers mailing list will be removed. @subheading History There is some unhappy history about this idea in our development community: @example @uref{http://lists.gnu.org/archive/html/lilypond-devel/2010-09/msg00178.html} @uref{http://news.lilynet.net/spip.php?article121} @uref{http://lists.gnu.org/archive/html/lilypond-devel/2010-11/msg00076.html} @end example @subheading Other projects The idea of private mailing lists is hardly uncommon in open-source software. For example, @example @uref{http://lwn.net/Articles/394660/} about debian-private @uref{http://subversion.apache.org/mailing-lists.html} private@@ @uref{http://www.freebsd.org/administration.html#t-core} @uref{http://foundation.gnome.org/legal/} board members pledge to keep certain matters confidential every security team of every linux distribution and OS @end example In fact, Karl Fogel's @qq{Producing Open Source Software} explicitly suggests a private mailing list for some circumstances: @example [on granting commit/push access to a contributor] But here is one of the rare instances where secrecy is appropriate. You can't have votes about potential committers posted to a public mailing list, because the candidate's feelings (and reputation) could be hurt. @uref{http://producingoss.com/en/consensus-democracy.html#electorate} @end example @subheading Board of governers, voting, etc? Many projects have an official board of directors, or a list of @qq{core developers}, with set term limits and elections and stuff. I don't think that we're that big. I think we're still small enough, and there's enough trust and consensus decisions, that we can avoid that. I would rather that we kept on going with trust+consensus for at least the next 2-3 years, and spent more time+energy on bug fixes and new features instead of administrative stuff. Project administrators are Han-Wen, Jan, and Graham. @subsubheading Discussions @smallexample @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00783.html} @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg01004.html} @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00117.html} @end smallexample @node GOP-PROP 7 - developers as resources @subsubsection GOP-PROP 7 - developers as resources We shall treat developers (and contributors) as @strong{Independent volunteers}: each person does whatever they want, whenever they want. We have busy careers and lives; we make no expectations of action from anybody (with the exception of the 6 people in @qq{Meister} positions). @subsubheading Discussions @smallexample @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg01092.html} @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00087.html} @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00497.html} @end smallexample @node GOP-PROP 8 - issue priorities @subsubsection GOP-PROP 8 - issue priorities We will delete the @qq{priority} field of the issue tracker altogether. The @qq{type} system will be tweaked. Type-critical: @itemize @item a reproducible failure to build either @code{make} or @code{make doc}, from an empty build tree, in a first run, if @code{configure} does not report any errors. @item any program behaviour which is @strong{unintentionally} worse than the previous stable version or the current development version. Developers may always use the @qq{this is intentional}, or even the @qq{this is an unavoidable effect of an improvement in another area}, reason to move this to a different type. @item anything which stops contributors from helping out (e.g. lily-git.tcl not working, source tree(s) not being available, LilyDev being unable to compile git master, inaccurate instructions in the Contributor's Guide 2 Quick start). To limit this scope of this point, we will assume that the contributor is using the latest LilyDev and has read the relevant part(s) of the Contributor's Guide. Problems in other chapters of the CG are not sufficient to qualify as Type-Critical. @end itemize @subsubheading More new/changed types and labels Unless otherwise specified, the current types and labels will continue to be used. The new types introduced by this proposal are: @itemize @item Type-crash: any segfault, regardless of what the input file looks like or which options are given. Disclaimer: this might not be possible in some cases, for example certain guile programs (we certainly can't predict if a piece of scheme will ever stop running, i.e. the halting problem), or if we rely on other programs (i.e. ghostscript). If there are any such cases that make segfault-prevention impossible, we will document those exceptions (and the issue will remain as a "crash" instead of "documentation" until the warning has been pushed). @item Type-maintainability: anything which makes it difficult for serious contributors to help out (e.g. difficult to find the relevant source tree(s), confusing policies, problems with automatic indentation tools, etc). @item Type-ugly: replaces Type-collision, and it will include things like bad slurs in addition to actual collision. @end itemize A new label will be added: @itemize @item (label) Needs_evidence: it is not clear what the correct output should look like. We need scans, references, examples, etc. @end itemize @subheading Reminding users about stars We can remind users that they can @qq{star} an issue to indicate that they care about it. Since we resolved to treat developers as independent volunteers, there is no expectation that anybody will look at those stars, but if any developer want to organize their work schedule according to the stars, they are welcome to do so. @subsubheading Discussions @smallexample @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00019.html} @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00277.html} @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00413.html} @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00624.html} @uref{} @end smallexample @node GOP-PROP 9 - behavior of make doc @subsubsection GOP-PROP 9 - behavior of make doc If there are build problems, then it should be easier to find out why it's failing. This will be achieved with log files, as well as possibly including scripts which automatically display portions of those log files for a failing build. We will also add targets for building a specific manual (for quick+easy checking of doc work), as well as for building all documentation in a specific language (either English or a translated language). When you run @code{make doc}, @itemize @item All output will be saved to various log files, with the exception of output directly from @code{make(1)}. Note that @code{make(1)} refers to a specific executable file on unix computers, and is not a general term for the build system. @item By default, no other output will be displayed on the console, with one exception: if a build fails, we might display some portion(s) of log file(s) which give useful clues about the reason for the failure. The user may optionally request additional output to be printed; this is controlled with the @code{VERBOSE=x} flag. In such cases, all output will still be written to log files; the console output is strictly additional to the log files. @item Logfiles from calling lilypond (as part of lilypond-book) will go in the relevant @file{build/out/lybook-db/12/lily-123456.log} file. All other logfiles will go in the @file{build/logfiles/} directory. A single @code{make doc} will therefore result in hundreds of log files. Log files produced from individual lilypond runs are not under our control; apart from that, I anticipate having one or two dozen log files. As long as it is clear which log file is associated with which operation(s), I think this is entirely appropriate. The precise implementation will be discussed for specific patches as they appear. @item Both stderr and stdout will be saved in @code{*.log}. The order of lines from these streams should be preserved. @item There will be no additional @qq{progress messages} during the build process. If you run @code{make --silent}, a non-failing build should print absolutely nothing to the screen. @item Assuming that the loglevels patch is accepted, lilypond (inside lilypond-book) will be run with --loglevel=WARN. @uref{http://codereview.appspot.com/4822055/} @item Ideally, a failing build should provide hints about the reason why it failed, or at least hints about which log file(s) to examine. @end itemize If this proposal is accepted, none of these policies will be assumed to apply to any other aspect of the build system. Policies for any other aspect of the build system will be discussed in separate proposals. @subheading Don't cause more build problems However, there is a danger in this approach, that vital error messages can also be lost, thus preventing the cause of the failure of a make being found. We therefore need to be exceptionally careful to move cautiously, include plenty of tests, and give time for people to experiment/find problems in each stage before proceeding to the next stage. This will be done by starting from individual lilypond calls within lilypond-book, and slowly moving to @qq{larger} targets of the build system -- after the individual lilypond calls are are producing the appropriate amount of output and this is saved in the right place and we can automatically isolate parts of a failing build, we will work on lilypond-book in general, and only then will we look at the build system itself. @subheading Implementation notes There is an existing make variable QUIET_BUILD, which alter the amount of output being displayed (@uref{ http://lilypond.org/doc/v2.15/Documentation/contributor/useful-make-variables} ). We are not planning on keeping this make variable. The standard way for GNU packages to give more output is with a @code{V=x} option. Presumably this is done by increasing @code{x}? If we support this option, we should still write log files; we would simply print more of the info in those log files to screen. The command @code{tee} may be useful to write to a file and display to stdout (in the case of VERBOSE). @subsubheading Discussions @smallexample @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00378.html} @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00703.html} @end smallexample @ignore @n ode GOP-PROP 10 - scheme indentation @s ubsubsection GOP-PROP 10 - scheme indentation still under discussion @subsubheading Discussions @smallexample @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00625.html} @uref{https://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg01026.html} @c @uref{} @end smallexample @end ignore @node Grand LilyPond Input Syntax Standardization (GLISS) @section Grand LilyPond Input Syntax Standardization (GLISS) @subheading Summary @itemize @item Start: sortly after 2.14 comes out, which is currently estimated to happen in January 2011. @item Length: 6-12 months. We're not going to rush this. @item Goal: define an input which we commit to being machine-updateable for the forseeable future. Any future patches which change the syntax in a non-convert-ly-able format will be rejected. (subject to the limitations, below) Once this is finished, we will release lilypond 3.0. @end itemize @subheading The Problem One of the biggest complaints people have with lilypond -- other than silly thing like "there's no gui" -- is the changing syntax. Now, inventing a language or standards is difficult. If you set it in stone too soon, you risk being stuck with decisions which may limit matters. If you keep on updating the syntax, interaction with older data (and other programs!) becomes complex. @subheading Scope and Limitations @itemize @item tweaks will not be included. Anything with \override, \set, \overrideProperty, \tweak, \revert, \unset... including even those command names themselves... is still fair game for NOT_SMART convert-ly updates. @item other than that, everything is on the table. Is it a problem to have the tagline inside \header? What should the default behavior of \include be? When we abolish \times, do we move to \tuplet 3:2 or \tuplet 2/3 or what (for typical triplets in 4/4 time)? @item we need to get standards for command names. This will help users remember them, and reduce the options for future names (and potential renamings later on). \commandOn and \commandOff seem to work well (should we *always* have an Off command?), but what about the "command" part? Should it be \nounVerbOn, or \verbNounOn ? Or \verbNotesWithExtraInformationOn ? @item we need standards for the location of commands. Ligature brackets, I'm looking at you. (non-postfix notation must die!) @item this Grand Project doesn't affect whether we have a 2.16 or not. The main problem will be deciding what to do (with a bit of messiness anticipated for \tuplet); we should definitely release a 2.16 before merging _any_ of these changes. @item we obviously can't /guarantee/ that we'll /never/ make any non-convert-ly changes in the basic format. But we *can* guarantee that such changes would force lilypond 4.0, and that we would only do so for overwhelmingly good reasons. @end itemize @subheading Workflow @itemize @item We're going to have lots and lots of emails flying around. The vast majority won't really fit into either -devel or -user, so we'll use a list devoted to syntax issues. @item Once we have a serious proposal that gained general acceptance from the separate syntax mailing list, I'll bring it to -devel. We're not going to make any changes without discussing it on -devel, but if we're going to have huge threads about English grammar and silly ideas, and I don't want to clutter up -devel. Once whatever chaotic silliness on the syntax list is settled down, I'll bring the ideas to -devel. @item as with GDP, I'll moderate the discussion. Not as with mailist moderation, but rather by introducing issues at specific times. We don't want a free-for-all discussion of all parts of the syntax at once; nothing will get resolved. @item Whenever possible, we'll decide on policies at the highest level of abstraction. For example, consider \numericTimeSignature, \slurUp, \xNotesOn, \startTextSpan, and \verylongfermata. One of them starts with the name of the notation first (slur). One has an abbreviation (x instead of cross). One has the verb at the end (On), another has it at the beginning (start). The adjective can come at the beginning (numeric, x) or end (Up). Most are in camelCase, but one isn't (verylongfermata). @item Instead of arguing about each individual command, we'll decide on abstract questions. Should each command begin the notation-noun, or the verb? Should all commands be in camelCase, or should we make everything other than articulations in camelCase but make articulations all lower-case? Are abbreviations allowed? @item Once we've answered such fundamental questions, most of the syntax should fall into place pretty easily. There might be a few odd questions left ("is it a span, or a spanner?"), but those can be settled fairly quickly. @end itemize @subheading Implementation Nothing until the project is finished, then we declare the next stable release (2.16.0 or 2.18.0 ?) to be the final 2.x version, release it, then apply all the GLISS syntax changes and start testing a beta for 3.0 a week or two later. @subheading Discussion Don't respond to any of the specifics yet. Yes, we all have our pet irritations (like "what's up with \paper and \layout?!"). There will be plenty of time to discuss them once GLISS starts. That said, we have a list of specific items that people really wanted to have written down. See @ref{Specific GLISS issues}. @menu * Specific GLISS issues:: @end menu @node Specific GLISS issues @subsection Specific GLISS issues @itemize @item add regtests for every piece of syntax (not one-command-per-file, but making a few files which, between them, use every single piece of syntax.) This is a great test for convert-ly. @item should GLISS cover suggested conventions? (indentation, one-bar-per-line, etc -- the kind of stuff we list for the lilypond formatting in the docs ?) @item how much (if any) syntactic sugar should we add? i.e. @example \instrumentName #'foo % instead of \set Staff.instrumentName @end example ? Carl: maybe yes, Neil: no. (for example, it fails for pianostaff) @item the values that are used as arguments to common used overrides. Sometimes they are a symbol (e.g. #'around), sometimes a predefined variable referring to a Scheme value or object (e.g. #LEFT, #all-visible ). The main trouble is that for novice users it is not clear when there should be an apostrophe and when not. @item When do we need -\command and when is it just \command ? @item Command-line options to the lilypond binary. -dfoo counts as a tweak; we won't be trying to pin those down. @item @verbatim \layout { \context { \Score % vs. \layout { \context { \Score @end verbatim @item If would be pedagogically simpler to realize this difference if the syntax was separate if you define a context from scratch (as is the case with \RemoveEmptyStaffContext) or if it's defined by adding onto an existing context. For example, a syntax like @verbatim \context{ % Copy the current settings of the Staff context: \use Staff % do whatever additional settings } %%% could be used to distinguish from \context{ % Take settings from a variable: \Variable % do whatever additional settings } %%% and \context{ % Start from scratch: \type ... \name ... \consists ... ... } @end verbatim @item Capitalization of identifiers: \VoiceOne ? @item @verbatim %%% Allow { music expression } * 4 %%% instead of \repeat unfold 4 { music expression } @end verbatim ? patch here: @smallexample @uref{http://lists.gnu.org/archive/html/lilypond-devel/2010-04/msg00467.html} @end smallexample @item Personally, I find it easier to understand when there's a repeated 8 in the half-bar position; it's much easier to see that you have two groups of 4: @example c8 c c c c8 c c c %%% instead of one group of eight: c8 c c c c c c c @end example @item trivially simple bar-lines: c1 | c1 | encourage, allow, or discourage, or disallow? @item indentation of \\ inside a @{@} construct. @item barline checks at the end of line should be preceded by at least 2 spaces? barline checks should line up if possible (i.e. if you can use less than 4, 8, X empty spaces before a barline check to make them line up?) @item Why doesn't \transpose respect \relative mode? @item on \score vs. \new Score But in the light of a consistent syntax and semantic, I see no reason (from the users POV) to disallow it. After all, the real top-level context is a \book @{@}, isn't it, and I don't see a point in disallowing a \new Score construct just like \new Staff. From a syntactical POV, I see the following pros for \new Score: - You can write \with @{ ... @} for every other context but \Score, which (for consistency) should also work with \new Score. - When there's a \new Foo Bar, there's also a \context Foo Bar, which makes the same as a parallel instantiation of all Bar's. - [Quoting Rune from @uref{http://www.mail-archive.com/lilypond-devel@@gnu.org/msg14713.html} "I know that the \score-statement is a syntactical construct, but I think it would be nice to hide this fact from the users. I think we could make the use of score-block much more intuitive if changing the syntax to \new \Score and adding an implicit sequential-statement to the score." @item Discussion on @uref{http://code.google.com/p/lilypond/issues/detail?id=1322} about \new vs. \context. @item Let users add their own items to the parser? comment 11 on: @uref{http://code.google.com/p/lilypond/issues/detail?id=1322} @item should engravers be pluralized (note_heads_engraver) or not (note_head_engraver) ? @item should we allow numbers in identifier names? Issue: @uref{http://code.google.com/p/lilypond/issues/detail?id=1670} @item should we officially allow accented characters? in general, how do we feel about utf-8 stuff? @item for the sake of completeness/simplicity, what about *disallowing* the "one-note" form of a music expression? i.e. only allowing stuff like @verbatim \transpose c d { e1 } \transpose c d << e1 >> @end verbatim and never allowing @verbatim \transpose c d e1 @end verbatim @item What should be the officially encouraged way of writing music for transposing instruments? Maybe it should be simplified? See http://lists.gnu.org/archive/html/lilypond-user/2011-07/msg00130.html @end itemize @node Unsorted policies @section Unsorted policies @subsubheading Language-specific mailing lists A translator can ask for an official lilypond-xy mailing list once they've finished all @qq{priority 1} translation items. @subsubheading Performing yearly copyright update (@qq{grand-replace}) At the start of each year, copyright notices for all source files should be refreshed by running the following command from the top of the source tree: @example make grand-replace @end example Internally, this invokes the script @file{scripts/build/grand-replace.py}, which performs a regular expression substitution for old-year -> new-year wherever it finds a valid copyright notice. Note that snapshots of third party files such as @file{texinfo.tex} should not be included in the automatic update; @file{grand-replace.py} ignores these files if they are listed in the variable @code{copied_files}. @subsubheading Push git access Git access is given out when a contributor has a significant record of patches being accepted without problems. If existing developers are tired of pushing patches for a contributor, we'll discuss giving them push access. Unsolicited requests from contributors for access will almost always be turned down.