-@subsubsubheading Testing new patches
-
-When run without any argument, @code{test-patches.py} will check
-@uref{http://code.google.com/p/lilypond/issues/list} for all tracker
-issues that are marked with the label @code{Patch-new}. It then scrapes
-the issue, looking for the last Rietveld URL entered. It then downloads
-the patch file and applies it to @code{test-master}.
-
-Here is an example where two tracker issues labeled as @code{Patch-new}
-were detected:
-
-@smallexample
-...
-issues [4007, 4008]
-Trying issue 4007
-Found url: http://codereview.appspot.com/112210043
-Found patch: 4007,/home/joe/lilypond-git/issue112210043_1.diff,
-Trying issue 4008
-Found url: http://codereview.appspot.com/115770043
-Found patch: 4008,/home/joe/lilypond-git/issue115770043_1.diff,
-Fetching, cloning, compiling master.
-...
-@end smallexample
-
-If run no tracker items with the @var{Patch-New} label are found it will
-report:
-
-@example
-issues []
-@end example
-
-The script can also be run using the tracker issue number(s) as an
-argument regardless if the @var{Patch-New} label has been assigned;
-
-@example
-test-patches.py 4006
-@end example
-
-or
-
-@example
-test-patches.py 4006 3992 4020
-@end example
-
-The script then checks to see if any previously
-@code{make test-baseline}s have been generated and if the commit ID of
-@code{remote/origin/master} is different from that previously completed
-test.
-
-@noindent
-If no previous @code{make test-baseline} test is discovered or if the
-commit ID of @code{remote/origin/master} has changed, then a new
-@code{make test-baseline} will run first automatically before the patch
-is tested:
-
-This shows when the commit ID has changed:
-
-@smallexample
-...
-(UTC) Begin LilyPond compile, previous commit at 3f92dcb2c81dcd2755542b57a0a5f2039f29a211
-Merged master, now at: 4726764cb591f622e7893407db0e7d42bcde90d9
- Success: ./autogen.sh --noconfigure
- Success: /tmp/lilypond-autobuild/configure --disable-optimising
- Success: nice make clean
- Success: nice make -j3 CPU_COUNT=3
- Success: nice make test-baseline -j3 CPU_COUNT=3
- Success: nice make doc -j3 CPU_COUNT=3
- Success: nice make doc-clean
-...
-@end smallexample
-
-@noindent
-If a previous regression test @emph{is} discovered @emph{and} if the
-commit ID of @code{remote/origin/master} has not changed, then the patch
-will be tested against the previous @code{make test-baseline} without
-the need to re-generate a new one:
-
-@smallexample
-...
-issues [4009]
-Trying issue 4009
-Found url: http://codereview.appspot.com/110540043
-Found patch: 4009,/home/joe/lilypond-extra/patches/issue110540043_1.diff,
-Fetching, cloning, compiling master.
-(UTC) Begin LilyPond compile, previous commit at 4726764cb591f622e7893407db0e7d42bcde90d9
- Success: No new commits in master
-Using test baseline from previous build.
-...
-@end smallexample
-
-The patch is then applied and a @code{make} and @code{make check} are
-run. A full @code{make doc} is also run if the
-@file{.lilypond-patchy-config} file has been edited accordingly;
-
-@smallexample
-...
-Issue 4009:
-Issue 4009: Testing patch issue110540043_1.diff
- Success: git apply --index /home/joe/lilypond-git/issue112210043_1.diff
- Success: ./autogen.sh --noconfigure
- Success: /tmp/lilypond-autobuild/configure --disable-optimising
- Success: nice make clean
- Success: nice make -j3 CPU_COUNT=3
- Success: nice make check -j3 CPU_COUNT=3
- Success: nice make doc -j3 CPU_COUNT=3
-...
-@end smallexample
-
-Once all the tests have run (successfully or not), the script will clean
-up from the previous patch and, if required, start testing the next
-issue;
-
-@smallexample
-...
-Issue 4007: Cleaning up
- Success: nice make test-clean
- Success: nice make doc-clean
- Success: nice make clean
- Success: git reset --hard
-Issue 4007: Done.
-Issue 4008:
-Issue 4008: Testing patch issue115770043_1_diff
- Success: git apply --index /home/joe/lilypond-git/issue115770043_1.diff
- Success: ./autogen.sh --noconfigure
-...
-@end smallexample
-
-and so on.
-
-@subsubsubheading Checking the regression test results
-
-Assuming the patch passed all the @code{make} tests, the regression
-differences will be located in the @file{/test-results/} directory
-within the build location for the patch issue number;
-
-@example
-/tmp/show-4007/test-results/
-@end example
-
-Open @file{index.html} in a browser of your choice to view any
-differences.
-
-@noindent
-Alternatively if the Firefox browser is installed, then the regression
-test results can be opened by calling the appropriate
-@file{show-regtests-} file located in the auto-compile log location;
-
-@example
-sh ~/lilypond-auto-compile-results/show-regtests-4007
-@end example
-
-See @ref{Regression tests}.
-
-@subsubsubheading Reporting test results
-
-Once a patch has been tested and the regression tests have been
-manually checked, the tracker can be updated manually by editing the
-tracker issue directly in the web browser or by using two additional
-python scripts that are included as part of the Patchy suite.
-
-@unnumberedsubsubsec For patches that have passed
-
-Use the @code{accept-patch.py} script and run it with the Google issue
-tracker number (not the Rietveld issue number) as an argument;
-
-@example
-accept-patch.py 4007
-@end example
-
-This will automatically update the tracker issue with the phrase
-@qq{Patchy the autobot says: passes tests.}.
-
-@noindent
-It is also possible to add additional information to the default
-message by adding a second argument within double-quote marks.
-
-@example
-accept-patch.py 4007 "This also includes a full documentation build."
-@end example
-
-The tracker issue's label is then changed automatically to
-@qq{Patch-review}.
-
-@unnumberedsubsubsec Patches that have failed
-
-Use the @code{reject-patch.py} script and run it with the Google issue
-tracker number (not the Rietveld issue number) as an argument but you
-@emph{must} also include a second argument, in double-quotes, stating
-the reason the patch has been rejected;
-
-@example
-reject-patch.py 4007 "Fails the 'make check' test."
-@end example
-
-Once the @code{reject-patch.py} script has been run, the tracker issue's
-label is changed automatically to @qq{Patch-Needs_work}.
-
-