@node Music engraving
@chapter Music engraving
-insert essay here.
+This section covers the overall goals and architecture of
+LilyPond.
-TODO: add extra chapter for computer aesthetics?
+@menu
+* Engraving::
+* Automated engraving::
+* What symbols to engrave?::
+* Music representation::
+* Example applications::
+@end menu
+
+
+@node Engraving
+@unnumberedsec Engraving
+
+@cindex engraving
+@cindex typography, music
+@cindex music typography
+@cindex plate engraving
+@cindex music engraving
+
+The art of music typography is called @emph{(plate) engraving}.
+The term derives from the traditional process of music printing.
+Just a few decades ago, sheet music was made by cutting and
+stamping the music into a zinc or pewter plate in mirror image.
+The plate would be inked, and the depressions caused by the cutting
+and stamping would hold ink. An image was formed by pressing
+paper to the plate. The stamping and cutting was done completely
+by hand. Making a correction was cumbersome, if possible at all,
+so the engraving had to be perfect in one go. Engraving was a
+highly specialized skill; a craftsman had to complete around five
+years of training before earning the title of master engraver, and
+another five years of experience were necessary to become truly
+skilled.
+
+Nowadays, all newly printed music is produced with computers.
+This has obvious advantages: prints are cheaper to make, and
+editorial work can be delivered by email. Unfortunately, the
+pervasive use of computers has also decreased the graphical
+quality of scores. Computer printouts have a bland, mechanical
+look, which makes them unpleasant to play from.
+
+
+@c introduce illustrating aspects of engraving, font...
+The images below illustrate the difference between traditional
+engraving and typical computer output, and the third picture shows
+how LilyPond mimics the traditional look. The left picture shows
+a scan of a flat symbol from an edition published in 2000. The
+center depicts a symbol from a hand-engraved Bärenreiter edition
+of the same music. The left scan illustrates typical flaws of
+computer print: the staff lines are thin, the weight of the flat
+symbol matches the light lines and it has a straight layout with
+sharp corners. By contrast, the Bärenreiter flat has a bold,
+almost voluptuous rounded look. Our flat symbol is designed
+after this one, among others. It is rounded, and its weight
+harmonizes with the thickness of our staff lines, which are also
+much thicker than lines in the computer edition.
+
+@multitable @columnfractions .125 .25 .25 .25 .125
+@item @tab
+@ifnotinfo
+@iftex
+@image{henle-flat-gray,,4cm}
+@end iftex
+@ifnottex
+@image{henle-flat-gray,,,png}
+@end ifnottex
+
+@tab
+@iftex
+@image{baer-flat-gray,,4cm}
+@end iftex
+@ifnottex
+@image{baer-flat-gray,,,png}
+@end ifnottex
+
+@tab
+@iftex
+@image{lily-flat-bw,,4cm}
+@end iftex
+@ifnottex
+@image{lily-flat-bw,,,png}
+@end ifnottex
+@end ifnotinfo
+@ifinfo
+@image{lilypond/henle-flat-bw,,,,png} @image{lilypond/baer-flat-bw,,,,png}
+@image{lilypond/lily-flat-bw,,,,png}
+@end ifinfo
+
+@item @tab
+Henle (2000)
+@tab
+Bärenreiter (1950)
+@tab
+LilyPond Feta font (2003)
+
+@end multitable
+
+
+@cindex musical symbols
+@cindex font
+@cindex blackness
+@cindex balance
+
+@c introduce illustrating aspects of engraving, spacing...
+In spacing, the distribution of space should reflect the durations
+between notes. However, many modern scores adhere to the
+durations with mathematical precision, which leads to poor
+results. In the next example a motive is printed twice: once
+using exact mathematical spacing, and once with corrections. Can
+you spot which fragment is which?
+
+@cindex optical spacing
+@c file spacing-optical.
+@c need to include it here, because we want two images.
+@lilypond
+\paper {
+ ragged-right = ##t
+ indent = #0.0
+}
+
+music = {
+ c'4 e''4 e'4 b'4 |
+ \stemDown
+ b'8[ e'' a' e'']
+ \stemNeutral
+ e'8[ e'8 e'8 e'8]
+}
+
+\score
+{
+ \music
+ \layout {
+ \context {
+ \Staff
+ \override NoteSpacing #'stem-spacing-correction = #0.6
+ }
+ }
+}
+@end lilypond
+
+@lilypond
+\paper {
+ ragged-right = ##t
+ indent = #0.0
+}
+
+music = {
+ c'4 e''4 e'4 b'4 |
+ \stemDown
+ b'8[ e'' a' e'']
+ \stemNeutral
+ e'8[ e'8 e'8 e'8]
+}
+\score
+{
+ \music
+ \layout {
+ \context {
+ \Staff
+ \override NoteSpacing #'stem-spacing-correction = #0.0
+ \override NoteSpacing #'same-direction-correction = #0.0
+ \override StaffSpacing #'stem-spacing-correction = #0.0
+ }
+ }
+}
+@end lilypond
+
+@cindex regular rhythms
+@cindex regular spacing
+@cindex spacing, regular
+
+Each bar in the fragment only uses notes that are played in a
+constant rhythm. The spacing should reflect that. Unfortunately,
+the eye deceives us a little; not only does it notice the distance
+between note heads, it also takes into account the distance
+between consecutive stems. As a result, the notes of an
+up-stem/@/down-stem combination should be put farther apart, and
+the notes of a down-stem/@/up-stem combination should be put
+closer together, all depending on the combined vertical positions
+of the notes. The upper two measures are printed with this
+correction, the lower two measures, however, form
+down-stem/@/up-stem clumps of notes.
+
+
+Musicians are usually more absorbed with performing than with
+studying the looks of a piece of music, so nitpicking
+typographical details may seem academic. But it is not. In
+larger pieces with monotonous rhythms, spacing corrections lead to
+subtle variations in the layout of every line, giving each one a
+distinct visual signature. Without this signature all lines would
+look the same, and they become like a labyrinth. A distinct visual
+signature helps to keep musicians from losing their place on the
+page when they look away or have a lapse in concentration.
+
+Similarly, the strong visual look of bold symbols on heavy staff
+lines stands out better when the music is far away from the
+reader: for example, if it is on a music stand. A careful
+distribution of white space allows music to be set very tightly
+without crowding symbols together. The result minimizes the
+number of page turns, which is a great advantage.
+
+This is a common characteristic of typography. Layout should be
+pretty, not only for its own sake, but especially because it helps
+the reader in her task. For performance material like sheet
+music, this is of double importance: musicians have a limited
+amount of attention. The less attention they need for reading,
+the more they can focus on playing the music. In other words,
+better typography translates to better performances.
+
+These examples demonstrate that music typography is an art that is
+subtle and complex, and that producing it requires considerable
+expertise, which musicians usually do not have. LilyPond is our
+effort to bring the graphical excellence of hand-engraved music to
+the computer age, and make it available to normal musicians. We
+have tuned our algorithms, font-designs, and program settings to
+produce prints that match the quality of the old editions we love
+to see and love to play from.
+
+
+@node Automated engraving
+@unnumberedsec Automated engraving
+
+@cindex engraving, automated
+@cindex automated engraving
+
+How do we go about implementing typography? If craftsmen need
+over ten years to become true masters, how could we simple hackers
+ever write a program to take over their jobs?
+
+The answer is: we cannot. Typography relies on human judgment of
+appearance, so people cannot be replaced completely. However,
+much of the dull work can be automated. If LilyPond solves most
+of the common situations correctly, this will be a huge
+improvement over existing software. The remaining cases can be
+tuned by hand. Over the course of years, the software can be
+refined to do more and more things automatically, so manual
+overrides are less and less necessary.
+
+When we started, we wrote the LilyPond program entirely in the C++
+programming language; the program's functionality was set in stone
+by the developers. That proved to be unsatisfactory for a number
+of reasons:
+
+@itemize
+
+@item When LilyPond makes mistakes, users need to override
+formatting decisions. Therefore, the user must have access to the
+formatting engine. Hence, rules and settings cannot be fixed by
+us at compile-time but must be accessible for users at run-time.
+
+@item Engraving is a matter of visual judgment, and therefore a
+matter of taste. As knowledgeable as we are, users can disagree
+with our personal decisions. Therefore, the definitions of
+typographical style must also be accessible to the user.
+
+@item Finally, we continually refine the formatting algorithms, so
+we need a flexible approach to rules. The C++ language forces a
+certain method of grouping rules that cannot readily be applied to
+formatting music notation.
+
+@end itemize
+
+@cindex Scheme programming language
+
+These problems have been addressed by integrating an interpreter
+for the Scheme programming language and rewriting parts of
+LilyPond in Scheme. The current formatting architecture is built
+around the notion of graphical objects, described by Scheme
+variables and functions. This architecture encompasses formatting
+rules, typographical style and individual formatting decisions.
+The user has direct access to most of these controls.
+
+Scheme variables control layout decisions. For example, many
+graphical objects have a direction variable that encodes the
+choice between up and down (or left and right). Here you see two
+chords, with accents and arpeggios. In the first chord, the
+graphical objects have all directions down (or left). The second
+chord has all directions up (right).
+
+@lilypond[quote,ragged-right]
+\new Score \with {
+ \override SpacingSpanner #'spacing-increment = #3
+ \override TimeSignature #'transparent = ##t
+} \relative c' {
+ \stemDown <e g b>4_>-\arpeggio
+ \override Arpeggio #'direction = #RIGHT
+ \stemUp <e g b>4^>-\arpeggio
+}
+@end lilypond
+
+@cindex score formatting
+@cindex formatting a score
+@cindex formatting rules
+
+@noindent
+The process of formatting a score consists of reading and writing
+the variables of graphical objects. Some variables have a preset
+value. For example, the thickness of many lines -- a
+characteristic of typographical style -- is a variable with a
+preset value. You are free to alter this value, giving your score
+a different typographical impression.
+
+@lilypond[quote,ragged-right]
+fragment = {
+ \clef bass f8 as8
+ c'4-~ c'16 as g f e16 g bes c' des'4
+}
+<<
+ \new Staff \fragment
+ \new Staff \with {
+ \override Beam #'thickness = #0.3
+ \override Stem #'thickness = #0.5
+ \override Bar #'thickness = #3.6
+ \override Tie #'thickness = #2.2
+ \override StaffSymbol #'thickness = #3.0
+ \override Tie #'extra-offset = #'(0 . 0.3)
+ }
+ \fragment
+>>
+@end lilypond
+
+Formatting rules are also preset variables: each object has
+variables containing procedures. These procedures perform the
+actual formatting, and by substituting different ones, we can
+change the appearance of objects. In the following example, the
+rule governing which note head objects are used to produce the
+note head symbol is changed during the music fragment.
+
+@lilypond[quote,ragged-right]
+#(set-global-staff-size 30)
+
+#(define (mc-squared grob orig current)
+ (let* ((interfaces (ly:grob-interfaces grob))
+ (pos (ly:grob-property grob 'staff-position)))
+ (if (memq 'note-head-interface interfaces)
+ (begin
+ (ly:grob-set-property! grob 'stencil
+ (grob-interpret-markup grob
+ (make-lower-markup 0.5
+ (case pos
+ ((-5) "m")
+ ((-3) "c ")
+ ((-2) (make-smaller-markup (make-bold-markup "2")))
+ (else "bla")))))))))
+
+\new Voice \relative c' {
+ \stemUp
+ \set autoBeaming = ##f
+ \time 2/4
+ <d f g>4
+ \once \override NoteHead #'stencil = #ly:note-head::brew-ez-stencil
+ \once \override NoteHead #'font-size = #-7
+ \once \override NoteHead #'font-family = #'sans
+ \once \override NoteHead #'font-series = #'bold
+ <d f g>4
+ \once \override NoteHead #'style = #'cross
+ <d f g>4
+ \applyOutput #'Voice #mc-squared
+ <d f g>4
+ <<
+ { d8[ es-( fis^^ g] fis2-) }
+ \repeat unfold 5 { \applyOutput #'Voice #mc-squared s8 }
+ >>
+}
+@end lilypond
+
+
+@node What symbols to engrave?
+@unnumberedsec What symbols to engrave?
+
+@cindex engraving
+@cindex typography
+@cindex engraver
+@cindex plug-in
+
+The formatting process decides where to place symbols. However,
+this can only be done once it is decided @emph{what} symbols
+should be printed -- in other words, what notation to use.
+
+Common music notation is a system of recording music that has
+evolved over the past 1000 years. The form that is now in common
+use dates from the early Renaissance. Although the basic form
+(i.e., note heads on a 5-line staff) has not changed, the details
+still evolve to express the innovations of contemporary notation.
+Hence, common music notation encompasses some 500 years of music.
+Its applications range from monophonic melodies to monstrous
+counterpoints for a large orchestra.
+
+How can we get a grip on such a seven-headed beast, and force it
+into the confines of a computer program? Our solution is to break
+up the problem of notation (as opposed to engraving, i.e.,
+typography) into digestible and programmable chunks: every type of
+symbol is handled by a separate module, a so-called plug-in. Each
+plug-in is completely modular and independent, so each can be
+developed and improved separately. Such plug-ins are called
+@code{engraver}s, by analogy with craftsmen who translate musical
+ideas to graphic symbols.
+
+In the following example, we start out with a plug-in for note
+heads, the @code{Note_heads_engraver}.
+@lilypond[quote,ragged-right]
+\include "engraver-example.ily"
+
+\score {
+ \topVoice
+ \layout {
+ \context {
+ \Voice
+ \remove "Stem_engraver"
+ \remove "Phrasing_slur_engraver"
+ \remove "Slur_engraver"
+ \remove "Script_engraver"
+ \remove "Beam_engraver"
+ \remove "Auto_beam_engraver"
+ }
+ \context {
+ \Staff
+ \remove "Accidental_engraver"
+ \remove "Key_engraver"
+ \remove "Clef_engraver"
+ \remove "Bar_engraver"
+ \remove "Time_signature_engraver"
+ \remove "Staff_symbol_engraver"
+ \consists "Pitch_squash_engraver"
+ }
+ }
+}
+@end lilypond
+
+@noindent
+Then a @code{Staff_symbol_engraver} adds the staff,
+
+@lilypond[quote,ragged-right]
+\include "engraver-example.ily"
+
+\score {
+ \topVoice
+ \layout {
+ \context {
+ \Voice
+ \remove "Stem_engraver"
+ \remove "Phrasing_slur_engraver"
+ \remove "Slur_engraver"
+ \remove "Script_engraver"
+ \remove "Beam_engraver"
+ \remove "Auto_beam_engraver"
+ }
+ \context {
+ \Staff
+ \remove "Accidental_engraver"
+ \remove "Key_engraver"
+ \remove "Clef_engraver"
+ \remove "Bar_engraver"
+ \consists "Pitch_squash_engraver"
+ \remove "Time_signature_engraver"
+ }
+ }
+}
+@end lilypond
+
+@noindent
+the @code{Clef_engraver} defines a reference point for the staff,
+
+@lilypond[quote,ragged-right]
+\include "engraver-example.ily"
+
+\score {
+ \topVoice
+ \layout {
+ \context {
+ \Voice
+ \remove "Stem_engraver"
+ \remove "Phrasing_slur_engraver"
+ \remove "Slur_engraver"
+ \remove "Script_engraver"
+ \remove "Beam_engraver"
+ \remove "Auto_beam_engraver"
+ }
+ \context {
+ \Staff
+ \remove "Accidental_engraver"
+ \remove "Key_engraver"
+ \remove "Bar_engraver"
+ \remove "Time_signature_engraver"
+ }
+ }
+}
+@end lilypond
+
+@noindent
+and the @code{Stem_engraver} adds stems.
+
+@lilypond[quote,ragged-right]
+\include "engraver-example.ily"
+
+\score {
+ \topVoice
+ \layout {
+ \context {
+ \Voice
+ \remove "Phrasing_slur_engraver"
+ \remove "Slur_engraver"
+ \remove "Script_engraver"
+ \remove "Beam_engraver"
+ \remove "Auto_beam_engraver"
+ }
+ \context {
+ \Staff
+ \remove "Accidental_engraver"
+ \remove "Key_engraver"
+ \remove "Bar_engraver"
+ \remove "Time_signature_engraver"
+ }
+ }
+}
+@end lilypond
+
+@noindent
+The @code{Stem_engraver} is notified of any note head coming
+along. Every time one (or more, for a chord) note head is seen, a
+stem object is created and connected to the note head. By adding
+engravers for beams, slurs, accents, accidentals, bar lines, time
+signature, and key signature, we get a complete piece of notation.
+
+@lilypond[quote,ragged-right]
+\include "engraver-example.ily"
+\score { \topVoice }
+@end lilypond
+
+@cindex polyphony
+@cindex engraving multiple voices
+@cindex contexts
+
+This system works well for monophonic music, but what about
+polyphony? In polyphonic notation, many voices can share a staff.
+
+@lilypond[quote,ragged-right]
+\include "engraver-example.ily"
+\new Staff << \topVoice \\ \botVoice >>
+@end lilypond
+
+In this situation, the accidentals and staff are shared, but the
+stems, slurs, beams, etc., are private to each voice. Hence,
+engravers should be grouped. The engravers for note heads, stems,
+slurs, etc., go into a group called @q{Voice context,} while the
+engravers for key, accidental, bar, etc., go into a group called
+@q{Staff context.} In the case of polyphony, a single Staff
+context contains more than one Voice context. Similarly, multiple
+Staff contexts can be put into a single Score context. The Score
+context is the top level notation context.
+
+
+@seealso
+Internals Reference: @rinternals{Contexts}.
+
+
+@lilypond[quote,ragged-right]
+\include "engraver-example.ily"
+\score {
+ <<
+ \new Staff << \topVoice \\ \botVoice >>
+ \new Staff << \pah \\ \hoom >>
+ >>
+}
+@end lilypond
+
+
+@node Music representation
+@unnumberedsec Music representation
+
+@cindex syntax
+@cindex recursive structures
+
+Ideally, the input format for any high-level formatting system is
+an abstract description of the content. In this case, that would
+be the music itself. This poses a formidable problem: how can we
+define what music really is? Instead of trying to find an answer,
+we have reversed the question. We write a program capable of
+producing sheet music, and adjust the format to be as lean as
+possible. When the format can no longer be trimmed down, by
+definition we are left with content itself. Our program serves as
+a formal definition of a music document.
+
+The syntax is also the user-interface for LilyPond, hence it is
+easy to type:
+
+@example
+@{
+ c'4 d'8
+@}
+@end example
+
+@noindent
+to create a quarter note on middle C (C1) and an eighth note on
+the D above middle C (D1).
+
+@lilypond[quote]
+{
+ c'4 d'8
+}
+@end lilypond
+
+On a microscopic scale, such syntax is easy to use. On a larger
+scale, syntax also needs structure. How else can you enter
+complex pieces like symphonies and operas? The structure is
+formed by the concept of music expressions: by combining small
+fragments of music into larger ones, more complex music can be
+expressed. For example
+
+@lilypond[quote,verbatim,fragment,relative=1]
+f4
+@end lilypond
+
+@noindent
+Simultaneous notes can be constructed by enclosing them with
+@code{<<} and @code{>>}:
+
+@example
+<<c4 d4 e4>>
+@end example
+
+@lilypond[quote,fragment,relative=1]
+\new Voice { <<c4 d4 e>> }
+@end lilypond
+
+@noindent
+This expression is put in sequence by enclosing it in curly braces
+@code{@{@tie{}@dots{}@tie{}@}}:
+
+@example
+@{ f4 <<c4 d4 e4>> @}
+@end example
+
+@lilypond[quote,relative=1,fragment]
+{ f4 <<c d e4>> }
+@end lilypond
+
+@noindent
+The above is also an expression, and so it may be combined again
+with another simultaneous expression (a half note) using
+@code{<<}, @code{\\}, and @code{>>}:
+
+@example
+<< g2 \\ @{ f4 <<c4 d4 e4>> @} >>
+@end example
+
+@lilypond[quote,fragment,relative=2]
+\new Voice { << g2 \\ { f4 <<c d e>> } >> }
+@end lilypond
+
+Such recursive structures can be specified neatly and formally in
+a context-free grammar. The parsing code is also generated from
+this grammar. In other words, the syntax of LilyPond is clearly
+and unambiguously defined.
+
+User-interfaces and syntax are what people see and deal with most.
+They are partly a matter of taste, and also the subject of much
+discussion. Although discussions on taste do have their merit,
+they are not very productive. In the larger picture of LilyPond,
+the importance of input syntax is small: inventing neat syntax is
+easy, while writing decent formatting code is much harder. This
+is also illustrated by the line-counts for the respective
+components: parsing and representation take up less than 10% of
+the source code.
+
+
+@node Example applications
+@unnumberedsec Example applications
+
+@cindex simple examples
+@cindex examples, simple
+
+We have written LilyPond as an experiment of how to condense the
+art of music engraving into a computer program. Thanks to all
+that hard work, the program can now be used to perform useful
+tasks. The simplest application is printing notes.
+
+@lilypond[quote,relative=1]
+{
+ \time 2/4
+ c4 c g'4 g a4 a g2
+}
+@end lilypond
+
+@noindent
+By adding chord names and lyrics we obtain a lead sheet.
+
+@lilypond[quote,ragged-right]
+<<
+ \chords { c2 c f2 c }
+ \new Staff
+ \relative c' {
+ \time 2/4
+ c4 c g' g a a g2
+ }
+ \addlyrics { twin -- kle twin -- kle lit -- tle star }
+>>
+@end lilypond
+
+Polyphonic notation and piano music can also be printed. The
+following example combines some more exotic constructs.
+
+@lilypond[quote]
+\header {
+ title = "Screech and boink"
+ subtitle = "Random complex notation"
+ composer = "Han-Wen Nienhuys"
+}
+
+\score {
+ \context PianoStaff <<
+ \new Staff = "up" {
+ \time 4/8
+ \key c \minor
+ << {
+ \revert Stem #'direction
+ \change Staff = down
+ \set subdivideBeams = ##t
+ g16.[
+ \change Staff = up
+ c'''32
+ \change Staff = down
+ g32
+ \change Staff = up
+ c'''32
+ \change Staff = down
+ g16]
+ \change Staff = up
+ \stemUp
+ \set followVoice = ##t
+ c'''32([ b''16 a''16 gis''16 g''32)]
+ } \\ {
+ s4 \times 2/3 { d'16[ f' g'] } as'32[ b''32 e'' d'']
+ } \\ {
+ s4 \autoBeamOff d''8.. f''32
+ } \\ {
+ s4 es''4
+ } >>
+ }
+
+ \new Staff = "down" {
+ \clef bass
+ \key c \minor
+ \set subdivideBeams = ##f
+ \override Stem #'french-beaming = ##t
+ \override Beam #'thickness = #0.3
+ \override Stem #'thickness = #4.0
+ g'16[ b16 fis16 g16]
+ << \makeClusters {
+ as16 <as b>
+ <g b>
+ <g cis>
+ } \\ {
+ \override Staff.Arpeggio #'arpeggio-direction =#down
+ <cis, e, gis, b, cis>4\arpeggio
+ }
+ >> }
+ >>
+ \midi {
+ \context {
+ \Score
+ tempoWholesPerMinute = #(ly:make-moment 60 8)
+ }
+ }
+ \layout {
+ \context {
+ \Staff
+ \consists Horizontal_bracket_engraver
+ }
+ }
+}
+@end lilypond
+
+The fragments shown above have all been written by hand, but that
+is not a requirement. Since the formatting engine is mostly
+automatic, it can serve as an output means for other programs that
+manipulate music. For example, it can also be used to convert
+databases of musical fragments to images for use on websites and
+multimedia presentations.
+
+This manual also shows an application: the input format is text,
+and can therefore be easily embedded in other text-based formats
+such as @LaTeX{}, HTML, or in the case of this manual, Texinfo.
+By means of a special program, the input fragments can be replaced
+by music images in the resulting PDF or HTML output files. This
+makes it easy to mix music and text in documents.
+
+
+TODO: add extra chapter for computer aesthetics?