X-Git-Url: https://git.donarmstrong.com/?a=blobdiff_plain;f=Documentation%2Fcontributor%2Fadministration.itexi;h=9af951ee8e6602429b58974a2f7ad33f4fb51194;hb=HEAD;hp=fca8f01deaba015f9acc80f3edf184c47cc151e3;hpb=6f6ea6500252a574985dd7550e7f01a84827cdd7;p=lilypond.git diff --git a/Documentation/contributor/administration.itexi b/Documentation/contributor/administration.itexi index fca8f01dea..9af951ee8e 100644 --- a/Documentation/contributor/administration.itexi +++ b/Documentation/contributor/administration.itexi @@ -1,4 +1,3 @@ -@c -*- coding: utf-8; mode: texinfo; -*- @node Administrative policies @chapter Administrative policies @@ -7,7 +6,9 @@ don't fit anywhere else. @menu * Meta-policy for this document:: +* Environment variables:: * Meisters:: +* Managing Staging and Master branches with Patchy:: * Administrative mailing list:: * Grand Organization Project (GOP):: * Grand LilyPond Input Syntax Standardization (GLISS):: @@ -77,55 +78,464 @@ Totally disorganized; do whatever the mao you want: @end itemize +@node Environment variables +@section Environment variables + +Some maintenance scripts and instructions in this guide rely on +the following environment variables. They should be predefined in +LilyDev distribution (see @ref{LilyDev}); if you set up your own +development environment, you can set them by appending these settings to +your @file{~/.bashrc} (or whatever defines your default environment +variables for the user account for LilyPond development), then logging +out and in (adapt directories to your setup): + +@example +LILYPOND_GIT=~/lilypond-git +export LILYPOND_GIT +LILYPOND_BUILD_DIR=~/lilypond-git/build +export LILYPOND_BUILD_DIR +@end example + +The standard build and install procedure (with @code{autogen.sh}, +@code{configure}, @code{make}, @code{make install}, @code{make doc} +@dots{}) does not rely on them. + +In addition, for working on the website, @code{LILYPOND_WEB_MEDIA_GIT} +should be set to the repository lilypond-extra, see +@ref{lilypond-extra}. + @node Meisters @section Meisters -We have four jobs for organizing a team of contributors: +We have four primary jobs to help organize all our contributors: + +@unnumberedsubsec The Bug Meister + +The Bug Meister's responsibilities are: + +@itemize + +@item +To organize the individual Bug Squad volunteers, making sure that +each member is aware of their responsibilities. See +@ref{The Bug Squad}. + +@item +To train new Bug Squad volunteers in the Issue Tracker process. See +@ref{Issues}. + +@item +To have the final say on our policies for Issues and their +classification. See @ref{Issue classification}. + +@end itemize + +Current Bug Meister: Colin Hall @email{bug-lilypond@@gnu.org} + + +@unnumberedsubsec The Doc Meister + +The Doc Meister's responsibilities are: @itemize @item -Bug Meister: trains new Bug Squad volunteers, organizes who works -on which part of their job, checks to make sure that everything is -running smoothly, and has final say on our policy for Issues. +To train new volunteers in our Documentation style and policy, +including organizing LilyPond Snippet Repository (LSR) work. + +@item +To organize the individual volunteers -- who does what on which job -- +and to check that everything is running smoothly. + +@item +To have final say on any Documentation policy. See +@ref{Documentation policy}. + +@end itemize + +Current Doc Meister: None + + +@unnumberedsubsec The Patch Meister -Currently: Phil +The Patch Meister's responsibilities are: + +@itemize @item -Doc Meister: trains new doc editors/writers, organizes who works -on which part of the job, checks to make sure that everything is -running smoothly, and has final say on our policy for -Documentation. Also includes LSR work. +To keep track of all patches submitted for testing and review. This +includes scanning the bug and dev email lists looking for any patches +submitted by @q{random} contributors and advising them on how to submit +a patch for testing and review. See @ref{Uploading a patch for review} +and @ref{The patch review cycle}. -Currently: Graham +@item +To makes sure that any patch submitted has a corresponding Issue Tracker +and Rietveld Issue created for it before it enters the testing and +review process. See @ref{Issues}. @item -Translation Meister: trains new translators, updates the -translation priority list, and handles merging branches (in both -directions). +Updates all Issue statuses for all patches that are currently in the +testing and review process periodically -- currently every 3 - 4 days. +See @ref{Patch handling}. + +@end itemize + +@warning{The Patch Meister's role is a purely administrative one and no +programming skill or judgement is assumed or required.} + +Currently: James Lowe @email{pkx@@gnu.org} + + +@unnumberedsubsec The Translation Meister + +The Translation Meister's responsibilities are: -Currently: Francisco +@itemize + +@item +To train new documentation translators in the translation process. See +@ref{Translating the documentation}. @item -Frog Meister: is responsible for code patches from (relatively) -inexperienced contributors. Keeps track of patches, does initial -reviewing of those patches, sends them to @w{@code{-devel}} when -they've had some initial review on the Frog list, pesters the -@w{@code{-devel}} community into actually reviewing said patches, and -finally pushes the patches once they're accepted. This person is -@emph{not} responsible for training new programmers, because that -would be far too much work -- he job is @qq{only} to guide -completed patches through our process. +To update the translation priority list and handle the merging of the +translation branches (in both directions). -Currently: Carl +@item +To have final say on any Translation management policies. See +@ref{Translating the documentation}. @end itemize +Currently: Francisco Vila @email{translations@@lilynet.net} + + + +@node Managing Staging and Master branches with Patchy +@section Managing Staging and Master branches with Patchy + +@ignore +The script 'test-patches.py' does not currently work with Allura. +@end ignore + +@menu +* Overview of Patchy:: +* Patchy requirements:: +* Installing Patchy:: +* Configuring Patchy:: +* Running the script:: +* Automating Patchy:: +* Troubleshooting Patchy:: +@end menu + +@node Overview of Patchy +@subsection Overview of Patchy + +No programmatic skill is required to run Patchy; although knowledge of +compiling LilyPond and its documentation along with understanding how to +configure the @var{PATH} environment of your computer is required. See +@ref{Working with source code}. + +The script @code{lilypond-patchy-staging.py} checks for any new commits +in @code{remote/origin/staging}, makes sure that the new HEAD compiles +along with all the LilyPond documentation. Then finally pushing to +@code{remote/origin/master}. This script can be run and left +unattended, requiring no human intervention. + +Patchy can also be configured to send emails after each successful (or +unsuccessful) operation. This is not a requirement and is turned off +by default. + +@c Need to explain in more detail how to set up Patchy for email but +@c as I don't use myself it I have no experience - JL + + +@node Patchy requirements +@subsection Patchy requirements + +@itemize + +@item +A full local copy of the source code. See +@ref{Working with source code}. + +@item +All the software needed for compiling LilyPond @emph{and} the +documentation. Unlike testing patches, being able to build the full set +of LilyPond's documentation is required to be able to test & push new +commits. See @ref{Compiling}. + +@item +Commit access @emph{is} required to test and push new commits, but a +valid login to @uref{https://sourceforge.net} is @emph{not}. See +@ref{Commit access}. + +@end itemize + + +@node Installing Patchy +@subsection Installing Patchy + +The Patchy scripts are not part of the LilyPond code base, but can be +downloaded from @uref{https://github.com/gperciva/lilypond-extra/}. The +scripts and related Python libraries are all located in the +@file{patches/} directory. + +Alternatively, use @code{git clone}; + +@example +git clone https://github.com/gperciva/lilypond-extra/ +@end example + +This makes it simpler to update the scripts if any changes are ever made +to them. Finally, add the location of the @file{patches/} directory to +your @var{PATH}. + + +@node Configuring Patchy +@subsection Configuring Patchy + +@warning{It is recommended to create a new user on your computer +specifically to run the Patchy scripts as a security precaution and that +this user should not have any administrative privileges. Also do not +set password protection for your ssh key else you will not be able to +run the scripts unattended.} + +@enumerate + +@item +Make sure the environment variables @var{LILYPOND_GIT} and +@var{LILYPOND_BUILD_DIR} are configured appropriately. See +@ref{Environment variables}. + +@item +Manually run either the @code{lilypond-patchy-staging.py} script and +when prompted: + +@smallexample +Warning: using default config; please edit /home/joe/.lilypond-patchy-config +Are you sure that you want to continue with the default config? (y/[n]) +@end smallexample + +Answer @qq{@code{n}} and press enter. + +The next time either of the scripts are run they will use the +@code{.lilypond-patchy-config} settings copied to your @code{$HOME} +directory. + +@item +Manually edit the @file{.lilypond-patchy-config} file, located in your +@code{$HOME} directory to change any of the default settings. + +@end enumerate + +These include: + +@itemize + +@item +All @code{make} operations are run with; +@example +extra_make_options = -j3 CPU_COUNT=3 +@end example + +See @ref{Saving time with the -j option} + +@item +A complete build of all the LilyPond documentation is @emph{not} +performed; +@example +patch_test_build_docs = no +@end example + +@item +Each instance of either a patch test or commit test & push is logged in; +@example +auto_compile_results_dir = ~/lilypond-auto-compile-results/ +@end example + +@item +Both scripts will perform their build operations in; +@example +build_dir = /tmp/lilypond-autobuild/ +@end example + +@end itemize + +The script creates a clones of @code{staging} and @code{master} +branches (prefixed with @code{test-}) with a third branch, called +@code{test-master-lock} used as a check to protect against two or more +instances of Patchy being run locally at the same time. + + +@node Running the script +@subsection Running the script + +@code{lilypond-patchy-staging.py} is run @emph{without} any arguments. +It then checks to see if @code{remote/origin/staging} is +@qq{further ahead} than @code{remote/origin/master}. + +@noindent +If there are no new differences between the two branches since the last +run check, the script will report something like this: + +@smallexample +(UTC) Begin LilyPond compile, previous commit at 4726764cb591f622e7893407db0e7d42bcde90d9 +Success: No new commits in staging +@end smallexample + +@noindent +If there are any differences between the two branches since the last +run check, (or if the script cannot for any reason, locate the last +instance of a commit that it checked) it will report something like +this: + +@smallexample +(UTC) Begin LilyPond compile, previous commit at 4726764cb591f622e7893407db0e7d42bcde90d9 +Merged staging, now at: 79e98a773b6570cfa28a15775a9dea3d3e54d6b5 + Success: ./autogen.sh --noconfigure + Success: /tmp/lilypond-autobuild/configure --disable-optimising +... +@end smallexample + +and proceed with running @code{make}, @code{make test} and a +@code{make doc}. Unlike @code{test-patches.py} if all the tests pass, +the script then pushes the changes to @code{remote/origin/master}. + +@smallexample +... +Success: nice make clean +Success: nice make -j7 CPU_COUNT=7 +Success: nice make test -j7 CPU_COUNT=7 +Success: nice make doc -j7 CPU_COUNT=7 +To ssh://joe@@git.sv.gnu.org/srv/git/lilypond.git + 79e98a7..4726764 test-staging -> master + Success: pushed to master +@end smallexample + +@warning{In the case where any of the @code{lilypond-patchy-staging.py} +tests fail, do not try to push your own fixes but report the failures to +the Developers List for advice.} + + +@node Automating Patchy +@subsection Automating Patchy + +To run as a cron job make sure you have; + +@example +[notification] +notify_non_action = no +@end example + +in @file{$HOME/.lilypond-patchy-config} to avoid any unintentional email +flooding: + +Assuming that Patchy run a user @qq{patchy}, create a file called +@file{$HOME/lilypond-patchy.cron}, adapting it as necessary (the +@code{/2} means @qq{run this every 2 hours}): + +@smallexample +02 0-23/2 * * * /home/patchy/lilypond-extra/patches/lilypond-patchy-staging.py +@end smallexample + +@warning{@code{cron} will not inherit environment variables so you must +re-define any variables inside @file{$HOME/lilypond-patchy.cron}. For +instance, @var{LILYPOND_GIT} may need to be defined if +@var{git_repository_dir} is not correctly set in +@file{$HOME/.lilypond-patchy-config}.} + +Finally, apply the cron job (you may need superuser privileges for +this): + +@example +crontab -u patchy /home/patchy/lilypond-patchy.cron +@end example + + +@node Troubleshooting Patchy +@subsection Troubleshooting Patchy + +The following is a list of the most common messages that the scripts +may report with explanations. + +@smallexample +this Git revision has already been pushed by an operator other than this Patchy. +@end smallexample + +@itemize + +@item +Another, remote, machine has already tested and pushed the new commits +in staging. + +@item +You may also see this if the auto-build files have been deleted and this +computer has previously already pushed the listed commit ID to +@code{master}. + +@end itemize + +@example +test-master-lock and PID entry exist but previous Patchy +run (PID xxxxx) died, resetting test-master-lock anyway. +@end example + +@noindent +A previous attempt was unsuccessful for some reason and the scripts were +not able to tidy up after themselves (for example if you manually halt +the process by killing it or closing the terminal you may have been +running the script in). The @code{test-master-lock} branch was +therefore not able to be deleted cleanly however, nothing needs to be +done the scripts will rebuild any tests it needs to. + +@example +fatal: A branch named 'test-master-lock' already exists. +@end example + +@itemize + +@item +There is another instance of Patchy running on your computer that is +testing the same tracker issue. + +@item +A previous test attempt was unsuccessful for some reason and the scripts +were not able to tidy up after themselves (for example if you manually +halt the testing process by killing it or closing the terminal you may +have been running the script in). The @code{test-master-lock} branch +was therefore not able to be deleted cleanly, in this case you must +manually delete the @code{test-master-lock} branch in your +@code{$LILYPOND_GIT} directory. + +@example +git branch -d test-master-lock +@end example + +@noindent +It may be wise to also manually delete @code{test-master} and +@code{test-staging} too, just to be safe. + +@end itemize + +@example +*** FAILED STEP *** + merge from staging + Another instance (PID xxxxx) is already running. +@end example + +@noindent +This occurs when trying to run @code{lilypond-patchy-staging.py} when +another instance of either script is already running locally. + + + + @node Administrative mailing list @section Administrative mailing list -An mailing list for administrative issues is maintained at +A mailing list for administrative issues is maintained at @code{lilypond-hackers@@gnu.org}. This list is intended to be used for discussions that should be kept @@ -149,7 +559,7 @@ GOP has two goals: @itemize @item -Clarify the various development tasks by writing down the polices +Clarify the various development tasks by writing down the policies and techniques and/or simplifying the tasks directly. @item @@ -215,7 +625,7 @@ work on the really hard stuff... like rewriting the grace note code. Having 1 more normal user answering emails on lilypond-user won't -have a dramatic trick-up affect all by himself, of course. But if +have a dramatic @q{trickle-up} effect all by itself, of course. But if we had 8 users volunteering to answer emails, 6 users starting to write documentation, and 2 users editing LSR... well, that would free up a lot of current bug-fixing-capable contributors to focus @@ -228,7 +638,7 @@ many new helpers will provide a great moral boost! Although GOP is a short-term project, the main goal is to train more people to handle ongoing jobs. The more people doing these -jobs, the ligher the work will be, and the more we can get done +jobs, the lighter the work will be, and the more we can get done with lilypond! Also, it would be nice if we had at least one "replacement" / @@ -278,7 +688,7 @@ We often receive reports of typos and minor text updates to the documentation. It would be great if somebody could create properly-formatted patches for these corrections. -Technical requirements: ability to run @ref{Lilydev}. +Technical requirements: ability to run @ref{LilyDev}. @item LSR editor: LSR contains many useful examples of lilypond, but some snippets @@ -295,7 +705,7 @@ chapters 1 and 2 (or be willing to read the docs to find out). often find them in Ponds of Lilies) and new feature implementors. Technical requirements: development environment (such as -@ref{Lilydev}), ability to read+write scheme and/or C++ code. +@ref{LilyDev}), ability to read+write scheme and/or C++ code. @end itemize @@ -360,48 +770,6 @@ away. This is not good. (prep: 2 hours. discuss: 10 hours) -@item @strong{Future release policy}: -(how) should we change any policies pertaining to releases? Should -an undocumented new feature count as release-blocking? - -(prep: 1 hour. discuss: 15 hours) - -@item @strong{lilypond-hackers mailing list}: -Should we have a private mailing list for senior developers? If -so, who should be on it? - -(prep: 2 hours+3 weeks. discuss: 10 hours) - -@item @strong{Hackers B}: - - -@item @strong{Git repository(s)}: -We currently have a web/ branch in our main repo; this seems -misleading to new developers. More generally, should we have -branches that aren't related to the master? i.e. should we -restrict a git branch to code which is an actual "branch" of -development? Also, some of our code (notably the windows and osx -lilypad) isn't in a git repository at all. -We can add new repositories very easily; should make repositories -like -@example -git://git.sv.gnu.org/lilypond/gub.git -git://git.sv.gnu.org/lilypond/lilypad.git -git://git.sv.gnu.org/lilypond/misc.git -@end example -? More information here: -@uref{http://code.google.com/p/lilypond/issues/detail?id=980} - -(prep: 2 hours. discuss: 10 hours) - -@item @strong{Roadmap of future development}: -Many projects have a roadmap of planned (or desired) future work. -Should we use one? If so, what should go on it, bearing in mind -our volunteer status? Is there any way of having a roadmap that -isn't vaporware? - -(prep: 1 hour. discuss: 5 hours) - @item @strong{Official links to other organizations?}: There's something called the "software freedom conservancy", and in general, there's a bunch of "umbrella organizations". Joining @@ -411,14 +779,6 @@ schools, etc. (prep: 2 hours. discuss: 5 hours) -@item @strong{Mailing lists}: -We currently have a mix of official GNU mailing lists and lilynet -lists. Is there a strong rationale for having separate mailing -list servers? Why not pick one place, and put all our lists there? -(or at least, all "permanent" lists?) - -(prep: 1 hour. discuss: 5 hours) - @item @strong{Issue tracking with google code}: We use the google issue tracker, but this means that we are relying on a commercial entity for a large part of our @@ -431,27 +791,10 @@ savannah bug tracker? Reitveld is inconvenient in some respects: it requires a google account, and there's no way to see all patches relating to lilypond. Should we switch to something like gerritt? -@uref{http://code.google.com/p/lilypond/issues/detail?id=1184} +@uref{https://sourceforge.net/p/testlilyissues/issues/1184/} (prep: 5 hours. discuss: 15 hours) -@item @strong{Subdomains of *.lilypond.org}: -Unless Jan has a really weird DNS hosting setup, there are no -technical barriers to having names like lsr.lilypond.org, -frog.lilypond.org, or news.lilypond.org. Is this something that we -want to do? - -(prep: 1 hours+2 weeks. discuss: 5 hours) - -@item @strong{Authorship in source files}: -Our documentation currently does not attempt to track individual -authors of each file, while our source code makes a confused and -jumbled attempt to track this. A number of guidelines for F/OSS -projects explicitly recommends _not_ tracking this in individual -files, since the code repository will track that for you. - -(prep: 2 hours. discuss: 15 hours) - @item @strong{Clarity for sponsorships}: We currently do not advertize bounties and sponsorships on the webpage. How much advertising do we want, and what type? @@ -459,29 +802,8 @@ Should we change the "structure" / "framework" for bounties? (prep: 2 hours. discuss: 10 hours) -@item @strong{Separate branches for active development}: -it might be good to have @emph{everybody} working on separate -branches. This complicates the git setup, but with sufficient -logic in lily-git.tcl, we can probably make it transparent to -newbies. However, we'd need a reliable person to handle all the -required merging and stuff. - -(prep: 2 hours. discuss: 10 hours) - -@item @strong{When do we add regtests?}: -There is a discrepancy between our stated policy on adding -regtests, and our actual practice in handling bugs and patches. -Clarify. - -There is also a wider question how to organize the regtests, such -as where to put interesting-console-output regtests, including -stuff like lilypond-book and midi2ly in a sensible manner, and -possibly including regtests for currently-broken functionality. - -(prep: 2 hours. discuss: 5 hours) - @item @strong{code readability}: -"Our aim when producing source code for Lilypond in whatever +"Our aim when producing source code for LilyPond in whatever language is that it should be totally comprehensible to a relatively inexperienced developer at the second reading." @@ -830,7 +1152,7 @@ community: @example @uref{http://lists.gnu.org/archive/html/lilypond-devel/2010-09/msg00178.html} -@uref{http://news.lilynet.net/spip.php?article121} +@uref{http://web.archive.org/web/20110325004849/http://news.lilynet.net/spip.php?article121} @uref{http://lists.gnu.org/archive/html/lilypond-devel/2010-11/msg00076.html} @end example @@ -846,7 +1168,7 @@ open-source software. For example, @uref{http://foundation.gnome.org/legal/} board members pledge to keep certain matters confidential -every security team of every linux distribution and OS +every security team of every GNU/Linux distribution and OS @end example In fact, Karl Fogel's @qq{Producing Open Source Software} @@ -930,11 +1252,11 @@ area}, reason to move this to a different type. @item anything which stops contributors from helping out (e.g. lily-git.tcl not working, source tree(s) not being available, -lilydev being unable to compile git master, inaccurate +LilyDev being unable to compile git master, inaccurate instructions in the Contributor's Guide 2 Quick start). To limit this scope of this point, we will assume that the -contributor is using the latest lilydev and has read the relevant +contributor is using the latest LilyDev and has read the relevant part(s) of the Contributor's Guide. Problems in other chapters of the CG are not sufficient to qualify as Type-Critical. @@ -1035,9 +1357,10 @@ all output will still be written to log files; the console output is strictly additional to the log files. @item -Logfiles from calling lilypond (as part of lilypond-book) will go -in the relevant @file{build/out/lybook-db/12/lily-123456.log} -file. All other logfiles will go in the @file{build/logfiles/} +Logfiles from calling lilypond (as part of lilypond-book) will go in +the relevant +@file{$LILYPOND_BUILD_DIR/out/lybook-db/12/lily-123456.log} file. All +other logfiles will go in the @file{$LILYPOND_BUILD_DIR/logfiles/} directory. A single @code{make doc} will therefore result in hundreds of log @@ -1177,8 +1500,7 @@ convert-ly updates. @item other than that, everything is on the table. Is it a problem to have the tagline inside \header? What should the default behavior -of \include be? When we abolish \times, do we move to \tuplet 3:2 -or \tuplet 2/3 or what (for typical triplets in 4/4 time)? +of \include be? @item we need to get standards for command names. This will help users @@ -1426,13 +1748,13 @@ sequential-statement to the score." @item Discussion on -@uref{http://code.google.com/p/lilypond/issues/detail?id=1322} +@uref{https://sourceforge.net/p/testlilyissues/issues/1322/} about \new vs. \context. @item Let users add their own items to the parser? comment 11 on: -@uref{http://code.google.com/p/lilypond/issues/detail?id=1322} +@uref{https://sourceforge.net/p/testlilyissues/issues/1322/} @item should engravers be pluralized (note_heads_engraver) or not @@ -1440,7 +1762,7 @@ should engravers be pluralized (note_heads_engraver) or not @item should we allow numbers in identifier names? Issue: -@uref{http://code.google.com/p/lilypond/issues/detail?id=1670} +@uref{https://sourceforge.net/p/testlilyissues/issues/1670/} @item should we officially allow accented characters? in general, how