Doc-de: updates from master to NR

[lilypond.git] / Documentation / essay / engraving.itely

@c -*- coding: utf-8; mode: texinfo; -*-

@ignore
    Translation of GIT committish: FILL-IN-HEAD-COMMITTISH

    When revising a translation, copy the HEAD committish of the
    version that you are working on.  See TRANSLATION for details.
@end ignore

@c \version "2.13.4"

@node Music engraving
@chapter Music engraving

This section covers the overall goals and architecture of
LilyPond.

@menu
* Engraving::                   
* Automated engraving::         
* What symbols to engrave?::    
* Music representation::        
* Example applications::        
@end menu


@node Engraving
@unnumberedsec Engraving

@cindex engraving
@cindex typography, music
@cindex music typography
@cindex plate engraving
@cindex music engraving

The art of music typography is called @emph{(plate) engraving}.
The term derives from the traditional process of music printing.
Just a few decades ago, sheet music was made by cutting and
stamping the music into a zinc or pewter plate in mirror image.
The plate would be inked, and the depressions caused by the cutting
and stamping would hold ink.  An image was formed by pressing
paper to the plate.  The stamping and cutting was done completely
by hand.  Making a correction was cumbersome, if possible at all,
so the engraving had to be perfect in one go.  Engraving was a
highly specialized skill; a craftsman had to complete around five
years of training before earning the title of master engraver, and
another five years of experience were necessary to become truly
skilled.

Nowadays, all newly printed music is produced with computers.
This has obvious advantages: prints are cheaper to make, and
editorial work can be delivered by email.  Unfortunately, the
pervasive use of computers has also decreased the graphical
quality of scores.  Computer printouts have a bland, mechanical
look, which makes them unpleasant to play from.


@c introduce illustrating aspects of engraving, font...
The images below illustrate the difference between traditional
engraving and typical computer output, and the third picture shows
how LilyPond mimics the traditional look.  The left picture shows
a scan of a flat symbol from an edition published in 2000.  The
center depicts a symbol from a hand-engraved Bärenreiter edition
of the same music.  The left scan illustrates typical flaws of
computer print: the staff lines are thin, the weight of the flat
symbol matches the light lines and it has a straight layout with
sharp corners.  By contrast, the Bärenreiter flat has a bold,
almost voluptuous rounded look.  Our flat symbol is designed
after this one, among others.  It is rounded, and its weight
harmonizes with the thickness of our staff lines, which are also
much thicker than lines in the computer edition.

@multitable @columnfractions .125 .25 .25 .25 .125
@item @tab
@ifnotinfo
@iftex
@image{pictures/henle-flat-gray,,4cm}
@end iftex
@ifnottex
@image{pictures/henle-flat-gray,,,png}
@end ifnottex

@tab
@iftex
@image{pictures/baer-flat-gray,,4cm}
@end iftex
@ifnottex
@image{pictures/baer-flat-gray,,,png}
@end ifnottex

@tab
@iftex
@image{pictures/lily-flat-bw,,4cm}
@end iftex
@ifnottex
@image{pictures/lily-flat-bw,,,png}
@end ifnottex
@end ifnotinfo
@ifinfo
@image{lilypond/pictures/henle-flat-bw,,,,png} @image{lilypond/pictures/baer-flat-bw,,,,png}
@image{lilypond/pictures/lily-flat-bw,,,,png}
@end ifinfo

@item @tab
Henle (2000)
@tab
Bärenreiter (1950)
@tab
LilyPond Feta font (2003)

@end multitable


@cindex musical symbols
@cindex font
@cindex blackness
@cindex balance

@c introduce illustrating aspects of engraving, spacing...
In spacing, the distribution of space should reflect the durations
between notes.  However, many modern scores adhere to the
durations with mathematical precision, which leads to poor
results.  In the next example a motive is printed twice: once
using exact mathematical spacing, and once with corrections.  Can
you spot which fragment is which?

@cindex optical spacing
@c file spacing-optical.
@c need to include it here,  because we want two images.
@lilypond
\paper {
  ragged-right = ##t
  indent = #0.0
}

music = {
   c'4 e''4 e'4 b'4 |
   \stemDown
   b'8[ e'' a' e'']
   \stemNeutral
   e'8[ e'8 e'8 e'8]
}

\score
{
  \music
  \layout {
    \context {
      \Staff
      \override NoteSpacing #'stem-spacing-correction = #0.6
    }
  }
}
@end lilypond

@lilypond
\paper {
  ragged-right = ##t
  indent = #0.0
}

music = {
   c'4 e''4 e'4 b'4 |
   \stemDown
   b'8[ e'' a' e'']
   \stemNeutral
   e'8[ e'8 e'8 e'8]
}
\score
{
  \music
  \layout {
    \context {
      \Staff
      \override NoteSpacing #'stem-spacing-correction = #0.0
      \override NoteSpacing #'same-direction-correction = #0.0
      \override StaffSpacing #'stem-spacing-correction = #0.0
    }
  }
}
@end lilypond

@cindex regular rhythms
@cindex regular spacing
@cindex spacing, regular

Each bar in the fragment only uses notes that are played in a
constant rhythm.  The spacing should reflect that.  Unfortunately,
the eye deceives us a little; not only does it notice the distance
between note heads, it also takes into account the distance
between consecutive stems.  As a result, the notes of an
up-stem/@/down-stem combination should be put farther apart, and
the notes of a down-stem/@/up-stem combination should be put
closer together, all depending on the combined vertical positions
of the notes.  The upper two measures are printed with this
correction, the lower two measures, however, form
down-stem/@/up-stem clumps of notes.


Musicians are usually more absorbed with performing than with
studying the looks of a piece of music, so nitpicking
typographical details may seem academic.  But it is not.  In
larger pieces with monotonous rhythms, spacing corrections lead to
subtle variations in the layout of every line, giving each one a
distinct visual signature.  Without this signature all lines would
look the same, and they become like a labyrinth.  A distinct visual
signature helps to keep musicians from losing their place on the
page when they look away or have a lapse in concentration.

Similarly, the strong visual look of bold symbols on heavy staff
lines stands out better when the music is far away from the
reader: for example, if it is on a music stand.  A careful
distribution of white space allows music to be set very tightly
without crowding symbols together.  The result minimizes the
number of page turns, which is a great advantage.

This is a common characteristic of typography.  Layout should be
pretty, not only for its own sake, but especially because it helps
the reader in her task.  For performance material like sheet
music, this is of double importance: musicians have a limited
amount of attention.  The less attention they need for reading,
the more they can focus on playing the music.  In other words,
better typography translates to better performances.

These examples demonstrate that music typography is an art that is
subtle and complex, and that producing it requires considerable
expertise, which musicians usually do not have.  LilyPond is our
effort to bring the graphical excellence of hand-engraved music to
the computer age, and make it available to normal musicians.  We
have tuned our algorithms, font-designs, and program settings to
produce prints that match the quality of the old editions we love
to see and love to play from.


@node Automated engraving
@unnumberedsec Automated engraving

@cindex engraving, automated
@cindex automated engraving

How do we go about implementing typography?  If craftsmen need
over ten years to become true masters, how could we simple hackers
ever write a program to take over their jobs?

The answer is: we cannot.  Typography relies on human judgment of
appearance, so people cannot be replaced completely.  However,
much of the dull work can be automated.  If LilyPond solves most
of the common situations correctly, this will be a huge
improvement over existing software.  The remaining cases can be
tuned by hand.  Over the course of years, the software can be
refined to do more and more things automatically, so manual
overrides are less and less necessary.

When we started, we wrote the LilyPond program entirely in the C++
programming language; the program's functionality was set in stone
by the developers.  That proved to be unsatisfactory for a number
of reasons:

@itemize

@item When LilyPond makes mistakes, users need to override
formatting decisions.  Therefore, the user must have access to the
formatting engine.  Hence, rules and settings cannot be fixed by
us at compile-time but must be accessible for users at run-time.

@item Engraving is a matter of visual judgment, and therefore a
matter of taste.  As knowledgeable as we are, users can disagree
with our personal decisions.  Therefore, the definitions of
typographical style must also be accessible to the user.

@item Finally, we continually refine the formatting algorithms, so
we need a flexible approach to rules.  The C++ language forces a
certain method of grouping rules that cannot readily be applied to
formatting music notation.

@end itemize

@cindex Scheme programming language

These problems have been addressed by integrating an interpreter
for the Scheme programming language and rewriting parts of
LilyPond in Scheme.  The current formatting architecture is built
around the notion of graphical objects, described by Scheme
variables and functions.  This architecture encompasses formatting
rules, typographical style and individual formatting decisions.
The user has direct access to most of these controls.

Scheme variables control layout decisions.  For example, many
graphical objects have a direction variable that encodes the
choice between up and down (or left and right).  Here you see two
chords, with accents and arpeggios.  In the first chord, the
graphical objects have all directions down (or left).  The second
chord has all directions up (right).

@lilypond[quote,ragged-right]
\new Score \with {
   \override SpacingSpanner #'spacing-increment = #3
   \override TimeSignature #'transparent = ##t
} \relative c' {
   \stemDown <e g b>4_>-\arpeggio
   \override Arpeggio #'direction = #RIGHT
   \stemUp <e g b>4^>-\arpeggio
}
@end lilypond

@cindex score formatting
@cindex formatting a score
@cindex formatting rules

@noindent
The process of formatting a score consists of reading and writing
the variables of graphical objects.  Some variables have a preset
value.  For example, the thickness of many lines -- a
characteristic of typographical style -- is a variable with a
preset value.  You are free to alter this value, giving your score
a different typographical impression.

@lilypond[quote,ragged-right]
fragment = {
   \clef bass f8 as8
   c'4-~ c'16 as g f e16 g bes c' des'4
}
<<
   \new Staff \fragment
   \new Staff \with {
      \override Beam #'beam-thickness = #0.3
      \override Stem #'thickness = #0.5
      \override Bar #'thickness = #3.6
      \override Tie #'thickness = #2.2
      \override StaffSymbol #'thickness = #3.0
      \override Tie #'extra-offset = #'(0 .  0.3)
      }
      \fragment
>>
@end lilypond

Formatting rules are also preset variables: each object has
variables containing procedures.  These procedures perform the
actual formatting, and by substituting different ones, we can
change the appearance of objects.  In the following example, the
rule governing which note head objects are used to produce the
note head symbol is changed during the music fragment.

@lilypond[quote,ragged-right]
#(set-global-staff-size 30)

#(define (mc-squared grob orig current)
  (let* ((interfaces (ly:grob-interfaces grob))
         (pos (ly:grob-property grob 'staff-position)))
    (if (memq 'note-head-interface interfaces)
        (begin
          (ly:grob-set-property! grob 'stencil
            (grob-interpret-markup grob
              (make-lower-markup 0.5
                (case pos
                  ((-5) "m")
                  ((-3) "c ")
                  ((-2) (make-smaller-markup (make-bold-markup "2")))
                  (else "bla")))))))))

\new Voice \relative c' {
  \stemUp
  \set autoBeaming = ##f
  \time 2/4
  <d f g>4
  \once \override NoteHead #'stencil = #note-head::brew-ez-stencil
  \once \override NoteHead #'font-size = #-7
  \once \override NoteHead #'font-family = #'sans
  \once \override NoteHead #'font-series = #'bold
  <d f g>4
  \once \override NoteHead #'style = #'cross
  <d f g>4
  \applyOutput #'Voice #mc-squared
  <d f g>4
  <<
    { d8[ es-( fis^^ g] fis2-) }
    \repeat unfold 5 { \applyOutput #'Voice #mc-squared s8 }
  >>
}
@end lilypond


@node What symbols to engrave?
@unnumberedsec What symbols to engrave?

@cindex engraving
@cindex typography
@cindex engraver
@cindex plug-in

The formatting process decides where to place symbols.  However,
this can only be done once it is decided @emph{what} symbols
should be printed -- in other words, what notation to use.

Common music notation is a system of recording music that has
evolved over the past 1000 years.  The form that is now in common
use dates from the early Renaissance.  Although the basic form
(i.e., note heads on a 5-line staff) has not changed, the details
still evolve to express the innovations of contemporary notation.
Hence, common music notation encompasses some 500 years of music.
Its applications range from monophonic melodies to monstrous
counterpoints for a large orchestra.

How can we get a grip on such a seven-headed beast, and force it
into the confines of a computer program?  Our solution is to break
up the problem of notation (as opposed to engraving, i.e.,
typography) into digestible and programmable chunks: every type of
symbol is handled by a separate module, a so-called plug-in.  Each
plug-in is completely modular and independent, so each can be
developed and improved separately.  Such plug-ins are called
@code{engraver}s, by analogy with craftsmen who translate musical
ideas to graphic symbols.

In the following example, we start out with a plug-in for note
heads, the @code{Note_heads_engraver}.

@lilypond[quote,ragged-right]
\include "engraver-example.ily"

\score {
  \topVoice
  \layout {
    \context {
      \Voice
      \remove "Stem_engraver"
      \remove "Phrasing_slur_engraver"
      \remove "Slur_engraver"
      \remove "Script_engraver"
      \remove "Beam_engraver"
      \remove "Auto_beam_engraver"
    }
    \context {
      \Staff
      \remove "Accidental_engraver"
      \remove "Key_engraver"
      \remove "Clef_engraver"
      \remove "Bar_engraver"
      \remove "Time_signature_engraver"
      \remove "Staff_symbol_engraver"
      \consists "Pitch_squash_engraver"
    }
  }
}
@end lilypond

@noindent
Then a @code{Staff_symbol_engraver} adds the staff,

@lilypond[quote,ragged-right]
\include "engraver-example.ily"

\score {
  \topVoice
  \layout {
    \context {
      \Voice
      \remove "Stem_engraver"
      \remove "Phrasing_slur_engraver"
      \remove "Slur_engraver"
      \remove "Script_engraver"
      \remove "Beam_engraver"
      \remove "Auto_beam_engraver"
    }
    \context {
      \Staff
      \remove "Accidental_engraver"
      \remove "Key_engraver"
      \remove "Clef_engraver"
      \remove "Bar_engraver"
      \consists "Pitch_squash_engraver"
      \remove "Time_signature_engraver"
    }
  }
}
@end lilypond

@noindent
the @code{Clef_engraver} defines a reference point for the staff,

@lilypond[quote,ragged-right]
\include "engraver-example.ily"

\score {
  \topVoice
  \layout {
    \context {
      \Voice
      \remove "Stem_engraver"
      \remove "Phrasing_slur_engraver"
      \remove "Slur_engraver"
      \remove "Script_engraver"
      \remove "Beam_engraver"
      \remove "Auto_beam_engraver"
    }
    \context {
      \Staff
      \remove "Accidental_engraver"
      \remove "Key_engraver"
      \remove "Bar_engraver"
      \remove "Time_signature_engraver"
    }
  }
}
@end lilypond

@noindent
and the @code{Stem_engraver} adds stems.

@lilypond[quote,ragged-right]
\include "engraver-example.ily"

\score {
  \topVoice
  \layout {
    \context {
      \Voice
      \remove "Phrasing_slur_engraver"
      \remove "Slur_engraver"
      \remove "Script_engraver"
      \remove "Beam_engraver"
      \remove "Auto_beam_engraver"
    }
    \context {
      \Staff
      \remove "Accidental_engraver"
      \remove "Key_engraver"
      \remove "Bar_engraver"
      \remove "Time_signature_engraver"
    }
  }
}
@end lilypond

@noindent
The @code{Stem_engraver} is notified of any note head coming
along.  Every time one (or more, for a chord) note head is seen, a
stem object is created and connected to the note head.  By adding
engravers for beams, slurs, accents, accidentals, bar lines, time
signature, and key signature, we get a complete piece of notation.

@lilypond[quote,ragged-right]
\include "engraver-example.ily"
\score { \topVoice }
@end lilypond

@cindex polyphony
@cindex engraving multiple voices
@cindex contexts

This system works well for monophonic music, but what about
polyphony?  In polyphonic notation, many voices can share a staff.

@lilypond[quote,ragged-right]
\include "engraver-example.ily"
\new Staff << \topVoice \\ \botVoice >>
@end lilypond

In this situation, the accidentals and staff are shared, but the
stems, slurs, beams, etc., are private to each voice.  Hence,
engravers should be grouped.  The engravers for note heads, stems,
slurs, etc., go into a group called @q{Voice context,} while the
engravers for key, accidental, bar, etc., go into a group called
@q{Staff context.}  In the case of polyphony, a single Staff
context contains more than one Voice context.  Similarly, multiple
Staff contexts can be put into a single Score context.  The Score
context is the top level notation context.


@seealso
Internals Reference: @rinternals{Contexts}.


@lilypond[quote,ragged-right]
\include "engraver-example.ily"
\score {
   <<
      \new Staff << \topVoice \\ \botVoice >>
      \new Staff << \pah \\ \hoom >>
   >>
}
@end lilypond


@node Music representation
@unnumberedsec Music representation

@cindex syntax
@cindex recursive structures

Ideally, the input format for any high-level formatting system is
an abstract description of the content.  In this case, that would
be the music itself.  This poses a formidable problem: how can we
define what music really is? Instead of trying to find an answer,
we have reversed the question.  We write a program capable of
producing sheet music, and adjust the format to be as lean as
possible.  When the format can no longer be trimmed down, by
definition we are left with content itself.  Our program serves as
a formal definition of a music document.

The syntax is also the user-interface for LilyPond, hence it is
easy to type:

@example
@{
  c'4 d'8
@}
@end example

@noindent
to create a quarter note on middle C (C1) and an eighth note on
the D above middle C (D1).

@lilypond[quote]
{
  c'4 d'8
}
@end lilypond

On a microscopic scale, such syntax is easy to use.  On a larger
scale, syntax also needs structure.  How else can you enter
complex pieces like symphonies and operas?  The structure is
formed by the concept of music expressions: by combining small
fragments of music into larger ones, more complex music can be
expressed.  For example

@lilypond[quote,verbatim,fragment,relative=1]
f4
@end lilypond

@noindent
Simultaneous notes can be constructed by enclosing them with
@code{<<} and @code{>>}:

@example
<<c4 d4 e4>>
@end example

@lilypond[quote,fragment,relative=1]
\new Voice { <<c4 d4 e>> }
@end lilypond

@noindent
This expression is put in sequence by enclosing it in curly braces
@code{@{@tie{}@dots{}@tie{}@}}:

@example
@{ f4 <<c4 d4 e4>> @}
@end example

@lilypond[quote,relative=1,fragment]
{ f4 <<c d e4>> }
@end lilypond

@noindent
The above is also an expression, and so it may be combined again
with another simultaneous expression (a half note) using
@code{<<}, @code{\\}, and @code{>>}:

@example
<< g2 \\ @{ f4 <<c4 d4 e4>> @} >>
@end example

@lilypond[quote,fragment,relative=2]
\new Voice { << g2 \\ { f4 <<c d e>> } >> }
@end lilypond

Such recursive structures can be specified neatly and formally in
a context-free grammar.  The parsing code is also generated from
this grammar.  In other words, the syntax of LilyPond is clearly
and unambiguously defined.

User-interfaces and syntax are what people see and deal with most.
They are partly a matter of taste, and also the subject of much
discussion.  Although discussions on taste do have their merit,
they are not very productive.  In the larger picture of LilyPond,
the importance of input syntax is small: inventing neat syntax is
easy, while writing decent formatting code is much harder.  This
is also illustrated by the line-counts for the respective
components: parsing and representation take up less than 10% of
the source code.


@node Example applications
@unnumberedsec Example applications

@cindex simple examples
@cindex examples, simple

We have written LilyPond as an experiment of how to condense the
art of music engraving into a computer program.  Thanks to all
that hard work, the program can now be used to perform useful
tasks.  The simplest application is printing notes.

@lilypond[quote,relative=1]
{
  \time 2/4
  c4 c g'4 g a4 a g2
}
@end lilypond

@noindent
By adding chord names and lyrics we obtain a lead sheet.

@lilypond[quote,ragged-right]
<<
   \chords { c2 c f2 c }
   \new Staff
   \relative c' {
     \time 2/4
     c4 c g' g a a g2
   }
   \addlyrics { twin -- kle twin -- kle lit -- tle star }
>>
@end lilypond

Polyphonic notation and piano music can also be printed.  The
following example combines some more exotic constructs.

@lilypond[quote]
\header {
  title = "Screech and boink"
  subtitle = "Random complex notation"
  composer = "Han-Wen Nienhuys"
}

\score {
  \context PianoStaff <<
    \new Staff = "up" {
      \time 4/8
      \key c \minor
      << {
        \revert Stem #'direction
        \change Staff = down
        \set subdivideBeams = ##t
        g16.[
          \change Staff = up
          c'''32
          \change Staff = down
          g32
          \change Staff = up
          c'''32
          \change Staff = down
          g16]
        \change Staff = up
        \stemUp
        \set followVoice = ##t
        c'''32([ b''16 a''16 gis''16 g''32)]
      } \\ {
        s4 \times 2/3 { d'16[ f' g'] } as'32[ b''32 e'' d'']
      } \\ {
        s4 \autoBeamOff d''8.. f''32
      } \\ {
        s4 es''4
      } >>
    }

    \new Staff = "down" {
      \clef bass
      \key c \minor
      \set subdivideBeams = ##f
      \override Stem  #'french-beaming = ##t
      \override Beam  #'beam-thickness = #0.3
      \override Stem  #'thickness = #4.0
      g'16[ b16 fis16 g16]
      << \makeClusters {
        as16 <as b>
        <g b>
        <g cis>
      } \\ {
        \override Staff.Arpeggio  #'arpeggio-direction =#down
        <cis, e, gis, b, cis>4\arpeggio
      }
    >> }
  >>
  \midi {
    \context {
      \Score
      tempoWholesPerMinute = #(ly:make-moment 60 8)
    }
  }
  \layout {
    \context {
      \Staff
      \consists Horizontal_bracket_engraver
    }
  }
}
@end lilypond

The fragments shown above have all been written by hand, but that
is not a requirement.  Since the formatting engine is mostly
automatic, it can serve as an output means for other programs that
manipulate music.  For example, it can also be used to convert
databases of musical fragments to images for use on websites and
multimedia presentations.

This manual also shows an application: the input format is text,
and can therefore be easily embedded in other text-based formats
such as @LaTeX{}, HTML, or in the case of this manual, Texinfo.
By means of a special program, the input fragments can be replaced
by music images in the resulting PDF or HTML output files.  This
makes it easy to mix music and text in documents.


TODO: add extra chapter for computer aesthetics?