@c -*- coding: utf-8; mode: texinfo; -*-
@node Regression tests
@chapter Regression tests

@menu
* Introduction to regression tests::
* Current regtest output::
* Comparison regtest output::
* MusicXML tests::
@end menu


@node Introduction to regression tests
@section Introduction to regression tests

LilyPond has a complete suite of regression tests that are used
to ensure that changes to the code do not break existing behavior.
These regression tests comprise small LilyPond snippets that test
the functionality of each part of LilyPond.

Regression tests are added when new functionality is added to
LilyPond.  They are also added when bugs are identified.  The
snippet that causes the bug becomes a regression test to verify
that the bug has been fixed.

The regression tests are automatically compiled using special @code{make}
targets.  The output of the regression tests is also automatically
checked to identify changes in LilyPond output.

The output of the regression tests is available on the website
for every stable version of LilyPond.  This allows the comparison
of different versions to see when bugs appeared.


@node Current regtest output
@section Current regtest output


@TODO: To be checked and completed -vv

The easiest way to see the @q{current} regtest output (meaning,
the ouput of the latest stable or development version) is
to look at the precompiled regtest
@uref{../../input/regression/collated-files.html, HTML page} or its
@uref{../../input/regression/collated-files.pdf, pdf version}.

However, depending on how many changes have been made to the code
since the latest release, this page may not reflect the latest
features, bugfixes... or new bugs that may have been introduced!

Therefore, if you have an appropriate environment to build LilyPond
yourself, it is recommended that you compile the software yourself.

The first step is to download the latest available source code,
as explained in @ref{Working with source code}.  Then you will need
to build the LilyPond binary@footnote{Uninstalling the previous
LilyPond version is not necessary, nor is running @code{make install},
since the tests will automatically be compiled with the LilyPond binary
you have just built in your source directory.}: see
@rcontrib{Compiling LilyPond}.

From this point, compiling the regtests is as simple as running

@example
  make test
@end example

However, as there are many snippets to compile, if you have a multi-core
machine it is highly recommended to use the @option{-j} option, as
described in @ref{Saving time with the @option{-j} option}.  Another
useful optimization is to set the @var{CPU_COUNT} variable; for a
quad-core processor the complete command would look like

@example
  make -j5 CPU_COUNT=4 test
@end example

The regtest output will then be available in one of the
@file{input/regression/out-*} directories, depending on the
exact command you used.


@node Comparison regtest output
@section Comparison regtest output


@node MusicXML tests
@section MusicXML tests