(Introduction): rewrite

[lilypond.git] / Documentation / user / introduction.itely

@c -*-texinfo-*-


@node Introduction
@chapter Introduction

There are a lot of programs that let you print sheet music with a
computer. Unfortunately, most of them do not do good job.  If you
regularly from both recent and old music, you can probably agree with
us on that.  Most computer printouts have a bland, mechanical look,
and are unpleasant to play from.  Musicians are usually more absorbed
with performing the music than with studying its looks, so they
usually cannot pinpoint what exactly is so disconcerting about
computer prints.

However, it only takes a little attention to see for yourself what is
what the differences are. The procedure is easy. Find two editions of
the same piece, an old edition a reputable publishing house, and one
which is formatted with a computer. If you are unsure, both can be
told apart easily. Traditional hand-work has slight variations in
symbol placements, while computer programs repeat all their errors in
placement with iron rigidity. To illustrate this, we show two scanned
fragments of the same piece, the first Cello Suite by Johann Sebastian
Bach. The top one was hand made (published in 1950 by
B@"{a}renreiter), while the bottom one was computer made (published in
2000 by Henle).

@ignore
@iftex
@image{baer-suite1-line,15cm}

@image{henle-suite1-line,15.3cm}
@end iftex

@ifhtml
@html
<img src="baer-suite1-line.png"> 
<p>
<img src="henle-suite1-line.png">
<p>
@end html
@end ifhtml
@end ignore


The fragments come from different editions, so there are some
editorial differences regarding the slurs. However, we want to compare
the typography of both.  We can see that the Henle edition looks
mechanical. This is caused by lack of variation in the spacing: the
note heads in both systems are on the same horizontal positions, as is
the bar line. A second difference is in the blackness of the page: the
B@"{a}renreiter has a stronger look, simply because it uses a heavier
font, and thicker staff lines. In the printed editions, the Henle
version is 7 mm wider (approximately 4 %). Often, this difference is
larger: computer scores tend to be spaced very widely, thus putting
less information on a page, on average. Besides these global
characteristics, software also make errors in the details.  Beams
should normally completely cover staff lines. In this case, the
programmers have tried to do so, but forgot to take into account the
thickness of the staff lines. The result is that the endings of all
sloped beams are wedge-formed. The effect is subtle, it may not show
up in this reproduction due to limits of the resolution, but it is
certainly noticeable in the original.

Looking at such detail to the formatting of a score certainly is
nitpicky.  However, all these details serve a purpose, and that is to
help the musician perform better: variation in spacing give each line
a unique, which helps you remember where you are when you look
away. Boldly print notes stand out more clearly when viewed from a
distance. Tightly spaced scores take up less pages, which reduces the
frequency of page turns. Correctly formatted beams and slurs indicate
the direction of music, and without distracting the reader with
unnecessary clusters of ink. Clearly, good music typography is
desirable.

What is wrong with other computer printed scores?

What is engraving?

Music notation  vs. Engraving. (vs. composing)

Why do you care about engraving?

Why would I care about engraving?

You say your program is special; you must be trying to sell it?

Freedom

Preserving/reinventing the art of engraving 

Typography in LilyPond.

Input format.  Issue is overrated.




************


LilyPond is a program to print sheet music.  If you have used notation
programs before, then the way to use this program might be surprising
at first sight: in order to print music you have to enter musical
codes in a file.  Then you run the program on the file, and the music
is produced without any further intervention.  For example, something
like this:


@lilypond[fragment,verbatim, relative 1, intertext="produces this:

"]
\key c \minor r8 c16 b c8 g as c16 b c8 d | g,4
@end lilypond

@cindex encoding music

Encoding music using letters and digits may appear strange,
intimidating or even clumsy at first. Nevertheless, when you take the
effort to learn the codes and the program you will find that it is
easier than it seems.  Entering music can be done quickly, and you
never have to remember how you made the program do something
complicated: it is all in the input code, and you only have to read
the file to see how it works. Moreover, you are rewarded with very
nicely looking output.

In this chapter, we will explain the reasoning behind this unusual
design, and how this approach affects you as a user.



@menu
* Music engraving::             
* Computerized typography::     
* Music representation::        
* Example applications::        
* About this manual::           
@end menu

@node Music engraving
@section Music engraving



@cindex engraving
@cindex typography

Making sheet music may seem trivial at first (``you print 5 lines, and
then put in the notes at different heights''),


Hier over notatie.

@emph{music engraving}, i.e. professional music typography, is in
another ballpark.  The term `music engraving' derives from the
traditional process of music printing.  Only a few decades ago, sheet
music was made by cutting and stamping the music into zinc or pewter
plates, mirrored. The plate would be inked, and the depressions caused
by the cutting and stamping would hold ink.  An image was formed by
pressing paper to the plate. Stamping and cutting was completely done
by hand. Making corrections was cumbersome, so engraving had to be
done correctly in one go. As you can imagine this was a highly
specialized skill, much more so than the traditional process of
printing books.
@cindex craftsmanship
@cindex master
In the traditional German craftsmanship six years of full-time
training, more than any other craft, were required before a student
could call himself a master of the art. After that many more years of
practical experience were needed to become an established music
engraver.  Even today, with the use of high-speed computers and
advanced software, music requires lots of manual fine tuning before it
is acceptable for publication.

When we wanted to write a computer program to create music typography,
we encountered the first problem: there were no sets of musical
symbols available: either they were not available freely, or they did
not look well to our taste. Not let down, we decided to try font
design ourselves. We created a font of musical symbols, relying on
nice printouts of hand-engraved music.  The experience helped develop
a typographical taste, and it made us appreciate subtle design
details. Without that experience, we would not have realized the
shortcomings of the fonts were that we admired at first.


@lilypond[noindent]
#(define magfact 3.0)
\score {  \notes { as'2 r4 }
          \paper {
                 raggedright = ##t
              \translator {
                  \ScoreContext
                  AccidentalPlacement \override #'right-padding = #3.0
                  StaffSymbol \override #'transparent =   ##t
                  Clef \override #'transparent = ##t
                  TimeSignature \override #'transparent = ##t
                  Accidental \override #'font-magnification = #magfact
                  Rest \override #'font-magnification = #magfact
                  NoteHead \override #'font-magnification = #magfact
                  Stem \override #'transparent = ##t
                  } } }
@end lilypond

@cindex musical symbols
@cindex font
@cindex blackness
@cindex balance


The figure above shows a few notable glyphs.  For example, the
vertical stem of the flat symbol should be brushed slightly,
i.e. becoming wider at the top. the half-notehead is not elliptic but
slightly diamond shaped.  Fine endings, such as the one on the bottom
of the quarter rest, should not end in sharp points, but rather in
rounded shapes.  Taken together, the blackness of the font must be
carefully tuned together with the thickness of lines, beams and slurs
to give a strong yet balanced overall impression.

Producing a strong and balanced look is the real challenge of music
engraving. It is a recurring theme with many variations.  In spacing,
strength and balance are in layout that is `heavy' enough---without
big gaps of space--- and without big clusters of black.  The
distribution of space should reflect the character of the music.

Spacing is an example of a subtlety of formatting music. The distances
between notes should reflect the durations between notes, but adhering
with mathematical precision to the duration will lead to a poor
result. Shown here is an example of a motive, printed twice. It is
printed using exact mathematical spacing, and with some
corrections. Can you spot which fragment is which?


@cindex optical spacing
@lilypond[noindent]
    \score { \notes {
      \property Staff.NoteSpacing \set #'stem-spacing-correction
        = #0.6
      c'4 e''4 e'4 b'4 |
       \stemDown b'4 e''4 a'4 e''4| \stemBoth
      \property Staff.NoteSpacing \override #'stem-spacing-correction
      = #0.0
      \property Staff.StaffSpacing \override #'stem-spacing-correction
      = #0.0
      c'4 e''4 e'4 b'4 |
      \stemDown b'4 e''4 a'4 e''4|
    }
    \paper { raggedright = ##t } }
@end lilypond

@cindex regular rhythms
@cindex regular spacing

The fragment that was printed uses only quarter notes: notes that are
played in a constant rhythm. The spacing should reflect
that. Unfortunately, the eye deceives us a little: the eye not only
notices the distance between note heads, but also between consecutive
stems. As a result, the notes of a up-stem/down-stem combination
should be put farther apart, and the notes of a down-up combination
should be put closer together, all depending on the combined vertical
positions of the notes. The first two measures are printed with this
correction, the last two measures without. The notes in the last two
measures form down-stem/up-stems clumps of notes.

@node Computerized typography
@section Computerized typography

The example in the previous section is one illustration of how subtle
music engraving can be is a subtle. Producing good engraving requires
skill and knowledge.


It was our challenge to see if we could put typographical knowledge
into a computer program. Capturing that knowledge has two aspects:
first, it has to be acquired. Then, it has to be encoded in
data-structures and algorithms.  As the previous example shows, there
is a lot of subtlety involved in music engraving, and unfortunately,
only a small fraction of these tiny details are documented.

One reason for the time that it takes to become a master engraver, is
that all these details must be learned either from experience or from
other engravers: as an engraver gets older and wiser, he will be able
to produce better and more complex pieces.  A similar situation is
present when putting typography into computer programs.  It is not
possible to come up with a final solution for a problem at the first
try. Instead, we start out with simple solution that might cover 75%
of the cases, and gradually refine that solution over the course of
months or years, so that 90 or 95 % of the cases are handled.

This has an important implication for the design of the
program. During development, almost every piece of formatting code
must be considered as temporary. When the need arises, is to be
replaced a solution that will cover even more cases.  A clean way to
accomplish this, is a ``plug-in'' architecture: an architecture where
new pieces of code can be inserted in the program dynamically.  In
such a program, a new solution can be developed along-side the
existing code. It can be perfected separately until it is better than
the existing solution, at which point, the new solution is switched on
by default, and the old one is removed.

Until that time, users must have a way to deal with imperfections:
these 25%, 10% or 5% of the cases that are not handled
automatically. In these cases, a user must be able to override
formatting decisions. A way to accomplish this, is to store decisions
in generic variables, and let the user manipulate these variables.
For example, consider the following fragment of notation.

@lilypond
\score { \notes {
    g'4-\f g4
        }
\paper { raggedright = ##t }
     }
@end lilypond

@noindent
The position of the forte symbol is slightly awkward, because it is
next to the low note, whereas dynamics should be below notes in
general. This may be remedied by inserting extra space between the
high note and the `f', as shown in this example

@lilypond
\score { \notes {
    \once\property Voice. DynamicLineSpanner  \override #'padding = #4.0 
    g'4-\f g4
        }
\paper { raggedright = ##t }
     }
@end lilypond

This was achieved with the input statement
@example
    \property Voice. DynamicLineSpanner  \override #'padding = #4.0 
@end example
which increases the amount of space (@code{padding}) between the note
and the dynamic symbol to 4.0 (which is measured in staff space, so
4.0 equals the height of a staff).

Both design aspects, a plug-in architecture, and formatting variables,
are built on top of GUILE, an interpreter for the programming language
Scheme, which is a member of the LISP family. Variables are stored as
Scheme objects, and attached to graphical objects such as note heads
and stems. The variables are a means to adjust formatting details in
individual cases, but they are used in a more general manner.

Consider the case of a publisher that is not satisfied with the in the
default layout, and wants heavier stems. Normally, they are @code{1.3}
times the thickness of staff lines, but suppose that their editions
require them to be twice the thickness of the staff lines. The same
mechanism can be used to adjust a setting globally. By issuing
@example
    \property Score.Stem \override #'thickness = #2.0 
@end example
the entire piece is formatted with thick stems:
@lilypond
\score { \notes {
    \property Score.Stem \override #'thickness = #2.0 
    \once\property Voice. DynamicLineSpanner  \override #'padding = #4.0 
    g'4-\f g4
        }
\paper { raggedright = ##t }
     }
@end lilypond

@noindent
In effect, by setting these variables, users can define their own
layout styles.

``Plug-ins'' are also implemented using Scheme.  A formatting
``plug-in'' takes the form of a function written in Scheme (or a C++
function made available as a Scheme function), and it is also stored
in a variable.  For example, the placement of the forte symbol in the
example above is calculated by the function
@code{Side_position_interface::aligned_side}.  If we want to replace
this function by a more advanced one, we could issue
@example
    \property Voice.DynamicLineSpanner \override #'Y-offset-callbacks
       = #`(,gee-whiz-gadget)
@end example

@noindent
Now, the formatting process will trigger a call to our new
@code{gee-whiz-gadget} function when the position of the f symbol has
to be determined.

The full scope of this functionality certainly is intimidating, but
there is no need to fear: normally, it is not necessary to define
style-sheets or rewrite formatting functions. In fact, LilyPond gets a
lot of formatting right automatically, so adjusting individual layout
situations is not needed very often at all.


@node Music representation
@section Music representation


One of the big questions when writing batch programs, is what kind of
input the program should expect. Many music notation programs offer a
graphical interface that shows notation, and allow you to enter the
music by placing notes on a staff. From our point of view, this design
is a form of cheating. After all, the core message of a piece of music
notation simply is the music itself. If you start by offering notation
to the user, you have already skipped one conversion, even if it is
implicit. If we want to generate music notation from something else,
then the obvious candidate for the source is the music itself.

On paper this theory sounds very good. In practice, it opens a can of
worms. What really @emph{is} music? Many philosophical treatises must
have been written on the subject.  Instead of losing ourselves in
philosophical arguments over the essence of music, we have reversed
the question to yield a more practical approach. Our assumption is
that the printed score contains all of the music of piece. We build a
program that uses some input format to produce such a score. Over the
course of time, the program evolves. While this happens, we can remove
more and more elements of the input format: as the program improves,
it can fill in irrelevant details of the input by itself. At some
(hypothetical) point, the program is finished: there is no possibility
to remove any more elements from the syntax.  What we have left is by
definition exactly the musical meaning of the score.

There are also more practical concerns.  Our users have to key in the
music into the file directly, so the input format should have a
friendly syntax. As programmers and scientists, we want a
clean formal definition. After all, producing music notation is a
difficult problem, and in the scientific world, problems can only be
solved if they are well-specified. Moreover, formally defined formats
are easier to write programs for.

These ideas shaped our music representation: it is a compact format
that can easily be typed by hand. It complex musical constructs from
simple entities like notes and rests, in much the same way that one
builds complex formulas from simple expressions such as numbers and
mathematical operators.

@node Example applications
@section Example applications

As programmers and hedonists we enjoy beauty in code, and code that
produces beautiful typeset music, but nevertheless this program can
applied to do useful things. In this section, we show a few small
examples of what is possible.

The simplest application is printing notes.

@lilypond[relative=1]
  \time 2/4 c4 c g'4 g a4 a g2  
@end lilypond

To these notes, chord names and lyrics  may be added, yielding
a lead sheet.

@lilypond[raggedright]
\score { <
  \context ChordNames \chords  { c2 c f2 c }
  \notes \relative c' { \time 2/4 c4 c g'4 g a4 a g2 }
  \context Lyrics \lyrics  { twin4 kle twin kle lit tle star2 } > }
@end lilypond

The following example combines some more exotic uses of notation

@lilypondfile{screech-boink.ly}

The fragments shown above have all been written by hand, but that is not
a requirement. Since the formatting engine is mostly automatic, it can
serve as an output means for other programs that manipulate music. It
can also be used to convert databases of musical fragments to images for
use on websites on multimedia presentations.

This manual also shows an application: the input format is plain text,
and can therefore be easily embedded in other text-based formats, such
as La@TeX{}, HTML or in the case of this manual, Texinfo.  By means of a
special program, the input fragments can be replaced by music images in
the resulting PostScript or HTML output files. This makes it easy to
mix music and text in documents.



@node About this manual
@section About this manual

The manual is divided into the following chapters
@itemize @bullet
@item
@ifhtml The 
@end ifhtml
@emph{@ref{Tutorial}}
gives a  gentle introduction into typesetting music.
First time users should start here. 
@item
@ifhtml
The
@end ifhtml
@emph{@ref{Notation manual}}
discusses topics grouped by notation construct.
@item
@ifhtml
 The
 @end ifhtml
@emph{@ref{Technical manual}}
@c
discusses the general design of the program, and how to extend 
functionality.
@item
@ifhtml
 The chapter
@end ifhtml
@emph{@ref{Invoking LilyPond}}  explains how to run LilyPond and its helper
programs.
@end itemize

Once you are experienced, you can simply use the manual as reference:
there is an extensive index@footnote{If you are looking for something,
and you cannot find it by using the index, that is considered a bug.
In that case, please file a bug report.}, but the document is also
available in
@ifnothtml
a big HTML page,
@end ifnothtml 
@ifhtml
@uref{../lilypond.html, a big HTML page}
@end ifhtml
which can be searched easily using the search facility of a web
browser.
@cindex search in manual
@cindex using the manual

@c  TODO: advise to buy a book on notation? 

If you are not familiar with music notation, or music terminology
(especially if you are a foreigner), then it is advisable to consult
the glossary as well. The glossary explains musical terms, and
includes translations to various languages. It is a
@ifhtml
@uref{../glossary.html,separate document}
@end ifhtml
@ifnothtml
separate document, available in HTML and PDF and can be printed as
well.
@end ifnothtml
@cindex idiom
@cindex jargon
@cindex terminology
@cindex foreign languages
@cindex language


This manual is not complete without a number of other documents. They
are not available in print, but should be included with the
documentation package for your platform

@itemize @bullet
@item
Generated internal documentation.
@ifhtml
available @uref{../lilypond-internals/lilypond-internals.html,here}
@end ifhtml

Almost all formatting functionality that is used internally, is
available directly to the user. For example, all variables that
control thicknesses, distances, etc, can be changed in input
files. There are a huge number of formatting options, and it would be
impossible to describe them all in a hand-written manual. The
generated internal documentation is a heavily crosslinked HTML
document, produced directly from the formatting definitions used.  It
documents the nit-gritty details of each and every LilyPond class, object and
function.

Each section of the reference manual has a @b{See also}
subsection, with links (in the HTML document, at least) to the
generated documentation.

@item
  Templates
@ifhtml
(available @uref{../../../input/templates/out-www/collated-files.html,here})
@end ifhtml

When you have gone through the tutorial, in theory you are able to
start writing input files. In practice, writing files from scratch
turns out to be intimidating.  To give a headstart, we have collected
a number of often-used formats in example files.  These files can be
used as a start, by copying the template, and adding notes in the
appropriate places.

@item
  Various input examples
@ifhtml
available @uref{../../../input/test/out-www/collated-files.html,here}
@end ifhtml
@cindex snippets

These small files show various tips and tricks, and are available as a
big HTML document, with pictures and explanatory texts included.


@item
  The regression test
@ifhtml
available @uref{../../../input/regression/out-www/collated-files.html,here}
@end ifhtml

We strive to test each feature in one test file. This collection of is
primarily to help us debug problems, but it can be instructive to see
how we excercise the program. The format is like the input examples.

@end itemize


The location of the documentation files that are mentioned here can
vary from system to system.  On occasion, this manual refers to
initialization and example files.  Throughout this manual, we refer to
input files relative to the top-directory of the source archive. For
example, @file{input/test/bla.ly} may refer to the file
@file{lilypond-1.7.19/input/test/bla.ly}.  On binary packages for the
Unix platform, the documentation and examples can typically be found
somewhere below @file{/usr/share/doc/lilypond/}. Initialization files,
for example @file{scm/lily.scm}, or @file{ly/engraver-init.ly}, are
usually found in the directory @file{/usr/share/lilypond/}.

@cindex adjusting output
@cindex variables
@cindex properties
@cindex lilypond-internals
@cindex internal documentation
@cindex Scheme
@cindex extending lilypond
@cindex bugreport
@cindex index

Finally, this and all other manuals, are available online both as PDF
files for print and HTML files for browsing. They are available from
the web site, which can be found at @uref{http://www.lilypond.org/}.

@cindex website 
@cindex URL