LilyPond internals
******************

This documents some aspects of the internals of GNU LilyPond. Some of this stuff comes from e-mail I wrote, some from e-mail others wrote, some are large comments taken away from the headers. This page may be a little incoherent.  Unfortunately, it is also quite outdated.  A more thorough  and understandable document is in the works.

You should use doc++ to take a peek at the sources.

Overview
========

GNU LilyPond is a "multi-pass" system. The different passes have been created so that they do not depend on each other. In a later stage some parts may be moved into libraries, or seperate programs, or they might be integrated in larger systems.

Parsing:

     No difficult algorithms. The .ly file is read, and converted to a list of Scores, which each contain Music and paper/midi-definitions.

Interpreting music

     The music is walked through in time-order. The iterators which do the walking report Music to Translators which use this information to create elements, either MIDI or "visual" elements. The translators form a hierarchy; the ones for paper output are Engravers, for MIDI Performers.

     The translators swallow Music (mostly atomic gobs called Requests), create elements, broadcast them to other translators on higher or same level in the hierarchy:

     The stem of a voice A is broadcast to the staff which contains A, but not to the stems, beams and noteheads of a different voice (say B) or a different staff. The stem and noteheads of A are coupled, because the the Note_heads_engraver broadcasts its heads, and the Stem_engraver catches these.

     The engraver which agrees to handle a request decides whether to to honor the request, ignore it, or merge it with other requests. Merging of requests is preferably done with other requests done by members of the same voicegroups (beams, brackets, stems). In this way you can put the voices of 2 instruments in a conductor's score so they make chords (the Beam requests of both instruments will be merged).

Prebreaking

     Breakable stuff (eg. clefs and bars) are copied into pre and postbreaks.

Preprocessing

     Some dependencies are resolved, such as the direction of stems, beams, and "horizontal" placement issues (the order of clefs,  keys etc, placement of chords in multi-voice music),

Break calculation:

     The lines and horizontal positions of the columns are determined.

Breaking

     Through some magical interactions with Line_of_score and Super_elem (check out the source) the "lines" are produced.

     All other spanners can figure across which lines they are spread. If applicable, they break themselves into pieces. After this, each piece (or, if there are no pieces, the original spanner itself) throws out any dependencies which are in the wrong line.

Postprocesing:

     Some items and all spanners need computation after the Paper_column positions are determined. Examples: slurs, vertical positions of staffs.

Output paper

mudela
======

Most information is stored in the form of a request.  In music typesetting, the user might want to cram a lot more symbols on the paper than actually fits. To reflect this idea (the user asks more than we can do), the container for this data is called Request.

In a lot of other formats this would be called an 'Event'

Barcheck_req
     Checks during music processing if start of this voice element coincides with the start of a measure. Handy to check if you left out some voice elts.

Note_req
     LilyPond has to decide if the ball should be hanging left or right. This influences the horizontal dimensions of a column, and this is why request processing should be done before horizontal spacing. Other voices' frivolities may cause the need for accidentals, so this is also for the to decide. The engraver can decide on positioning based on ottava commands and the appropriate clef.

Rest_req
     Typeset a rest.

Span_req
     This type of request typically results in the creation of a Spanner

Beam_req
     Start/stop a beam. Engraver has to combine this request with the stem_request, since the number of flags that a stem wants to carry will determine the number of beams.

Dynamic
     Each dynamic is bound to one note (a crescendo spanning multiple notes is thought to be made of two "dynamics": a start and a stop). Dynamic changes can occur in a smaller time than the length of its note, therefore fore each  Dynamic request carries a time, measured from the start of its note.

Request_engraver
================

In the previous section the idea of Request has been explained, but this only solves one half of the problem. The other half is deciding which requests should be honored, which should merged with other requests, and which should be ignored. Consider this input

     	\type Staff < % chord
     		{ \meter 2/4; [c8 c8] }
     		{\meter 2/4;  [e8 e8] }
     	>

Both the cs and es are part of a staff (they are in the same Voice_group), so they should share meters, but the two [ ] pairs should be merged.

The judge in this "allocation" problem a set of brokers: the requests are transmitted to so-called engravers which respond if they want to accept a request eg, the Notehead_engraver will accept Note_reqs, and turn down Slur_reqs. If the Music_iterator cannot find a engraver that wants the request, it is junked (with a warning message).

After all requests have been either assigned, or junked, the Engraver will process the requests (which usually means creating an Item or Spanner). If a Request_engraver creates something, it tells the enclosing context. If all items/spanners have been created, then each Engraver is notified of any created Score_element, via a broadcasting system.

example:
--------

     	c4

produces:

     	Note_request (duration 1/4)
     	Stem_request (duration 1/4)

Note_request will be taken by a Notehead_engraver, stem_request will be taken by a Stem_beam_engraver. Notehead_engraver creates a Notehead, Stem_beam_engraver creates a Stem. Both announce this to the Staff_engraver. Staff_engraver will tell Stem_beam_engraver about the Notehead, which will add the Notehead to the Stem it just created.

To decide on merging, several engravers have been grouped. Please check init/engraver.ly.

Items and spanners
==================

The symbols that are printed, are generated by items and spanners (staff-elements). An item has one horizontal position, whereas a spanner spans several columns.

Dependencies
============

In music symbols depend on each other: the stems of a beam should point in the same direction as the beam itself, so the stems of a beam depend on the beam. In the same way do scripts depend on the direction of the stem. To reflect this, LilyPond has the notion of dependency. It works in the same fashion that make uses to build programs: before a stem is calculated, its dependencies (the beam) should be calculated. Before a slur is calculated, its dependencies (stems, noteheads) should be calculated.

Breaking
========

So what is this PREBREAK and POSTBREAK stuff?

Let's take text as an example. In German some compound words change their spelling if they are broken: "backen" becomes "bak-ken".  TeX has a mechanism to deal with this, you would define the spelling of "backen" in TeX in this way

\discretionary{bak-}{ken}{backen}

These 3 arguments are called "prebreak", "postbreak" and "nobreak" text.

The same problem exists when typesetting music. If a line of music is broken, the next line usually gets a clef. So in TeX terms, the clef is a postbreak. The same thing happens with meter signs: Normally the meter follows the bar. If a line is broken at that bar, the bar along with the meter stays on the "last" line, but the next line also gets a meter sign after the clef. Using the previous notation,

\discretionary{bar meter}{clef meter}{ bar meter }

In GNU Lilypond, we have the same concepts (and the same terminology). Each (nonrhythmic) symbol is typeset in  a nonrhythmic column At a breakpoint, multiple symbols are printed; symbols to be printed if the line is not broken, symbols to appear on the previous line, and on the next line if it is broken.

CodingStyle - standards while programming for GNU
*************************************************

LilyPond

We use these standards while doing programming for GNU LilyPond.  If you do some hacking, we appreciate it if you would follow this rules, but if you don't, we still like you.

Functions and methods do not return errorcodes.

     A program should be light and agile, its subroutines connected like a string of pearls.  The spirit and intent of the program should be retained throughout.  There should be neither too little nor too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity.

     A program should follow the 'Law of Least Astonishment'.  What is this law?  It is simply that the program should always respond to the user in the way that astonishes him least.

     A program, no matter how complex, should act as a single unit.  The program should be directed by the logic within rather than by outward appearances.

     If the program fails in these requirements, it will be in a state of disorder and confusion.  The only way to correct this is to rewrite the program.

     -- Geoffrey James, "The Tao of Programming"

Languages
---------

C++ and Python are preferred.  Perl is not.  Python code should use an indent of 8, using TAB characters.

Filenames
---------

Definitions of classes that are only accessed via pointers (*) or references (&) shall not be included as include files.

filenames

     	".hh"	Include files
     	".cc"	Implementation files
     	".icc"	Inline definition files
     	".tcc"	non inline Template defs

in emacs:

     	(setq auto-mode-alist
     	      (append '(("\\.make$" . makefile-mode)
     			("\\.cc$" . c++-mode)
     			("\\.icc$" . c++-mode)
     			("\\.tcc$" . c++-mode)
     			("\\.hh$" . c++-mode)
     			("\\.pod$" . text-mode)
     			)
     		      auto-mode-alist))

The class Class_name_abbreviation is coded in class-name-abbr.*

Indentation
-----------

Standard GNU coding style is used.   In emacs:

     	(add-hook 'c++-mode-hook
     		  '(lambda() (c-set-style "gnu")
     		     )
     		  )

If you like using font-lock, you can also add this to your .emacs:

     	(setq font-lock-maximum-decoration t)
     	(setq c++-font-lock-keywords-3
     	      (append
     	       c++-font-lock-keywords-3
     	       '(("\\b\\([a-zA-Z_]+_\\)\\b" 1 font-lock-variable-name-face)
     	       ("\\b\\([A-Z]+[a-z_]+\\)\\b" 1 font-lock-type-face))
     	       ))

Classes and Types
-----------------

     	This_is_a_class

Members
-------

     	Class::member ()
     	Type Class::member_type_
     	Type Class::member_type ()

the type is a Hungarian notation postfix for Type. See below

Macros
------

Macros should be written completely in uppercase

The code should not be compilable if proper macro declarations are not included.

Don't laugh. It took us a whole evening/night to figure out one of these bugs, because we had a macro that looked like DECLARE_VIRTUAL_FUNCTIONS ().

Broken code
-----------

Broken code (hardwired dependencies, hardwired constants, slow algorithms and obvious limitations) should be marked as such: either with a verbose TODO, or with a short "ugh" comment.

Comments
--------

The source is commented in the DOC++ style.  Check out doc++ at http://www.zib.de/Visual/software/doc++/index.html

     	/*
     		C style comments for multiline comments.
     		They come before the thing to document.
     		[...]
     	*/

     	/**
     		short description.
     		Long class documentation.
     		(Hungarian postfix)

     		TODO Fix boring_member ()
     	*/
     	class Class {
     		/**
     		  short description.
     		  long description
     		*/

     		Data data_member_;

     		/**
     			short memo. long doco of member ()
     			@param description of arguments
     			@return Rettype
     		*/
     		Rettype member (Argtype);

     		/// memo only
     		boring_member () {
     			data_member_ = 121; // ugh
     		}
     	};

Unfortunately most of the code isn't really documented that good.

Members (2)
-----------

Standard methods:

     	///check that *this satisfies its invariants, abort if not.
     	void OK () const

     	/// print *this (and substructures) to debugging log
     	void print () const

     	/**
     	protected member. Usually invoked by non-virtual XXXX ()
     	*/
     	virtual do_XXXX ()

     	/**add some data to *this.
     	Presence of these methods usually imply that it is not feasible to this
     	via  a constructor
     	*/
     	add (..)

     	/// replace some data of *this
     	set (..)

Constructor
-----------

Every class should have a default constructor.

Don't use non-default constructors if this can be avoided:

     	Foo f(1)

is less readable than

     	Foo f;
     	f.x = 1

or

     	Foo f(Foo_convert::int_to_foo (1))

Hungarian notation naming convention
====================================

Proposed is a naming convention derived from the so-called Hungarian Notation.  Macros, enums and consts are all uppercase, with the parts of the names separated by underscores.

Types
-----

byte
     unsigned char. (The postfix _by is ambiguous)

b
     bool

bi
     bit

ch
     char

f
     float

i
     signed integer

str
     string class

sz
     Zero terminated c string

u
     unsigned integer

User defined types
------------------

     	/** Slur blah. blah.
     	(slur)
     	*/
     	class Slur {};
     	Slur* slur_p = new Slur;

Modifiers
---------

The following types modify the meaning of the prefix. These are preceded by the prefixes:

a
     array

array
     user built array.

c
     const. Note that the proper order is Type const i.s.o. const Type

C
     A const pointer. This would be equivalent to _c_l, but since any "const" pointer has to be a link (you can't delete a const pointer), it is superfluous.

l
     temporary pointer to object (link)

p
     pointer to newed object

r
     reference

Adjective
---------

Adjectives such as global and static should be spelled out in full. They come before the noun that they refer to, just as in normal english.

     foo_global_i: a global variable of type int commonly called "foo".

static class members do not need the static_ prefix in the name (the Class::var notation usually makes it clear that it is static)

loop_i
     Variable loop: an integer

u
     Temporary variable: an unsigned integer

test_ch
     Variable test: a character

first_name_str
     Variable first_name: a String class object

last_name_ch_a
     Variable last_name: a char array

foo_i_p
     Variable foo: an Int* that you must delete

bar_i_l
     Variable bar: an Int* that you must not delete

Generally default arguments are taboo, except for nil pointers.

The naming convention can be quite conveniently memorised, by expressing the type in english, and abbreviating it

     	static Array<int*> foo

foo can be described as "the static int-pointer user-array", so you get

     	foo_static_l_arr

Miscellaneous
=============

For some tasks, some scripts are supplied, notably creating patches, a mirror of the website, generating the header to put over cc and hh files, doing a release.

Use them.

name
====

PATCHES - track and distribute your code changes

This page documents how to distribute your changes to GNU lilypond

We would like to have unified context diffs with full pathnames.  A script automating supplied with Lily.

Distributing a change normally goes like this:

  * make your fix/add your code

  * Add changes to NEWS, and add yourself to Documentation/topdocs/AUTHORS.yo

  * generate a patch,

  * e-mail your patch to one of the mailing lists gnu-music-discuss@gnu.org or bug-gnu-music@gnu.org

Please do not send entire files, even if the patch is bigger than the original.  A patch makes it clear what is changed, and it won't overwrite previous not yet released changes.

Generating a patch
==================

In VERSION, set MY_PATCH_LEVEL:

         VERSION:
     	...
     	MY_PATCH_LEVEL=jcn1

In CHANGES, enter a summary of changes:

     	pl 0.1.73.jcn1
     		- added PATCHES.texi

Then, from the top of Lily's source tree, type

         make dist
         make diff

which rolls the tarball ../releases/lilypond-0.1.73.tar.gz and leaves your patch as ./lilypond-0.1.73.jcn1.diff. ('Make diff' generates a patch between two tarballs.  For more info type 'make diff help=='.)  We assume that there is a tarball lilypond-0.1.73.tar.gz in the directory ../releases.

If you didn't configure Lily using --srcdir, you can do:

         make release

         tar-ball: ../patches/lilypond-0.1.73.jcn1.gz
         patch: ../patches/lilypond-0.1.73.jcn1.gz

Prerequisites
=============

For creating a patch you need

  * All items mentioned in INSTALL.  You're not going to send a patch that you haven't even built, right?

  * GNU diff

              make distclean
              cd ..
              diff -urN lilypond-0.1.73 lilypond-0.1.73.jcn1 > lilypond-0.1.73.jcn1

     but there are handy python scripts available.  If you're doing development, you'll need Python for other LilyPond scripts anyway.

  * The Lily directory structure, which looks like:

              doos/                        # gnu/windows32 build and binary releases
              harmonia -> harmonia-x.y.z
              harmonia-x.y.z/
              lilypond -> lilypond-x.y.z   # symlink to development directory
              lilypond-x.y.z/              # current development
              patches/ 		         # patches between different releases
              RedHat/BUILD                 # RedHat build and binary releases
              RedHat/RPMS
              RedHat/SPECS
              releases/                    # .tar.gz releases
              test/                        # tarballs and diffs from current version

     with prefix $HOME/usr/src and (for building rpms only) in $HOME/.rpmrc:

              topdir: /home/fred/usr/src/RedHat

Applying patches
================

If you're following LilyPond development regularly, you probably want to download just the patch for each subsequent release. After downloading the patch (into the patches directory, of course), simply apply it:

         gzip -dc ../patches/lilypond-0.1.74.diff.gz | patch -p1 -E

and don't forget to make automatically generated files:

         autoconf footnote(patches don't include automatically generated files,
         i.e. file(configure) and files generated by file(configure).)

         configure

Synchronise
===========

If you're not very quick with sending your patch, there's a good chance that an new release of LilyPond comes available.  In such a case, the maintainer will probably ask you to make a new patch against the latest release.  Your best bet is to download the latest release, and apply your patch against this new source tree:

         cd lilypond-0.1.74
         gzip -dc ../patches/lilypond-0.1.73.jcn1.diff.gz | patch -p1 -E
         autoconf
         configure

Then, make a patch as shown above.

See also
========

stepmake/INSTALL.txt

maintainer
==========

Han-Wen Nienhuys

Just keep on sending those patches!