Getting involved
****************

Please help us make LilyPond a better program. You can help LilyPond in several ways. Not all tasks requiring programming or understanding the full source code.  You can write to the mailing list (gnu-music-discuss@gnu.org for more information)

Users
-----

Mutopia needs your help. The mutopia project is a collection of public domain sheet music. You can help the project by entering music (either by hand, or by converting from scans or MIDI) and submitting it. Point your browser to the Mutopia webpage (http://sca.uwaterloo.ca/Mutopia).

Font designers
--------------

Our set of glyphs (the Feta font) is far from complete.  We need the following glyphs: segno, coda.  The fonts have been coded in MetaFont, so you will need to know MetaFont if you want to contribute a glyph.

Writers
-------

The documentation of LilyPond and related utilities needs a lot of work. The documentation is written in texinfo (http://www.gnu.org/software/texinfo). The documentation of LilyPond is sorely lacking in terms of completeness, depth and organisation.

Write if you know how to write english documentation in texinfo, and know about music and music notation.  You must also know how to use LilyPond (or be prepared to learn using it).  The task is not especially hard, but it is a lot of work, and you must be familiar with LilyPond.

Translators
-----------

LilyPond is completely ready for internationalized messages, but there are only a few translations so far (dutch, italian, german, japanese, french, russian).  Translation involves writing a .po file, which is relatively easy, and does not even require running LilyPond.

Hackers
-------

There are lots of possibilities of improving the program itself. There are both small projects and big ones. Most of them are listed in our TODO file, listed on the homepage of Jan and Han-Wen (http://www.cs.uu.nl/~hanwen/lily-devel).  Modifying LilyPond almost always requires patches to the C++ part.

There are also numerous other interesting projects that are more or less related  to LilyPond

  * Writing convertors, eg. from NIFF and MIDI (we tried writing one with limited success: midi2ly, included with lilypond.)

     We found that writing them in Python is the easiest.

  * Writing a GUI frontend to LilyPond. At the moment denemo.sourceforge.net (denemo) is the most advanced.

  * Helping write solfege tools (http://solfege.sourceforge.net/)

  * Helping primrose (primrose.sourceforge.net), a tool for scanning sheet music.

LilyPond internals
******************

Overview
========

GNU LilyPond is a "multi-pass" system.

Parsing:

     No difficult algorithms. The .ly file is read, and converted to a list of Scores, which each contain Music and paper/midi-definitions.

Interpreting music

     The music is walked through in time-order. The iterators which do the walking report Music to Translators which use this information to create elements, either MIDI or "visual" elements. The translators form a hierarchy; the ones for paper output are Engravers, for MIDI Performers.

     The translators swallow Music (mostly atomic gobs called Requests), create elements, broadcast them to other translators on higher or same level in the hierarchy:

     The stem of a voice A is broadcast to the staff which contains A, but not to the stems, beams and noteheads of a different voice (say B) or a different staff. The stem and noteheads of A are coupled, because the the Note_heads_engraver broadcasts its heads, and the Stem_engraver catches these.

     The engraver which agrees to handle a request decides whether to to honor the request, ignore it, or merge it with other requests. Merging of requests is preferably done with other requests done by members of the same voicegroups (beams, brackets, stems). In this way you can put the voices of 2 instruments in a conductor's score so they make chords (the Beam requests of both instruments will be merged).

Prebreaking

     Breakable stuff (eg. clefs and bars) are copied into pre and postbreaks.

Preprocessing

     Some dependencies are resolved, such as the direction of stems, beams, and "horizontal" placement issues (the order of clefs,  keys etc, placement of chords in multi-voice music),

Break calculation:

     The lines and horizontal positions of the columns are determined.

Breaking

     Through some magical interactions with Line_of_score and Super_elem (check out the source) the "lines" are produced.

     All other spanners can figure across which lines they are spread. If applicable, they break themselves into pieces. After this, each piece (or, if there are no pieces, the original spanner itself) throws out any dependencies which are in the wrong line.

Postprocesing:

     Some items and all spanners need computation after the Paper_column positions are determined. Examples: slurs, vertical positions of staffs.

Output paper

Request_engraver
================

In the previous section the idea of Request has been explained, but this only solves one half of the problem. The other half is deciding which requests should be honored, which should merged with other requests, and which should be ignored. Consider this input

     	\type Staff < % chord
     		{ \meter 2/4; [c8 c8] }
     		{\meter 2/4;  [e8 e8] }
     	>

Both the cs and es are part of a staff (they are in the same Voice_group), so they should share meters, but the two [ ] pairs should be merged.

The judge in this "allocation" problem a set of brokers: the requests are transmitted to so-called engravers which respond if they want to accept a request eg, the Notehead_engraver will accept Note_reqs, and turn down Slur_reqs. If the Music_iterator cannot find a engraver that wants the request, it is junked (with a warning message).

After all requests have been either assigned, or junked, the Engraver will process the requests (which usually means creating an Item or Spanner). If a Request_engraver creates something, it tells the enclosing context. If all items/spanners have been created, then each Engraver is notified of any created Score_element, via a broadcasting system.

example:
--------

     	c4

produces:

     	Note_request (duration 1/4)
     	Stem_request (duration 1/4)

Note_request will be taken by a Notehead_engraver, stem_request will be taken by a Stem_beam_engraver. Notehead_engraver creates a Notehead, Stem_beam_engraver creates a Stem. Both announce this to the Staff_engraver. Staff_engraver will tell Stem_beam_engraver about the Notehead, which will add the Notehead to the Stem it just created.

To decide on merging, several engravers have been grouped. Please check init/engraver.ly.

The backend of LilyPond
=======================

blah blah blah

Music notation is composed of a sets of interrelated glyphs.  In Lilypond every glyph usually is represented by one object, a so-called Graphic Object.  The primary relations between graphic objects involve positions:

  consecutive notes are printed left to right, grouped  in a staff

  simultaneous notes are horizontally aligned (internally grouped in a paper column).

  the staccato mark is horizontally centered on the note it applies to.

The abstract encoding of such relations is done with the concept reference point.  The reference point (in X direction) of the staccato mark is the note it applies to.  The (X) position of the staccato mark is stored relative to the position of the note head.  This means that the staccato will always maintain a fixed offset wrt to the note head, whereever the head is moved to.

In the same vein, all notes on a staff have their Y positions stored relative to an object that groups the staff.  If that object is moved, the absolute Y positions of all objects in that spanner change along, in effect causing the staff and all its contents to move as a whole.

Each graphic object stores a pointer and an relative offset for each direction: one for the X-axis, one for the Y-axis.  For example, the X parent of a Note_head usually is a Note_column.  The X parent of a Note_column usually is either a Collision or a Paper_column. The X parent of a Collision usually is a Paper_column.  If the Collision moves, all Note_heads that have that Collision as parent also move, but the absolute position of the Paper_column does not change.

To build a graphical score with Graphic_elements, conceptually, one needs to have one Root object (in Lilypond: Line_of_score), and recursively attach objects to the Root.   However, due to the nature of the context selection mechanisms, it turns out to be more advantageous to construct the tree the other way around: first small trees (note heads, barlines etc.) are created, and these are subsequently composed into larger trees, which are finally hung on a Paper_column (in X direction) or Line_of_score (in Y direction).

The structure of the X,Y parent relations are determined by the engravers and notation contexts:

The most important X-axis parent relation depends on the timing: notes that come earlier are attached to Paper_column that will be positioned more to the left.

The most important Y-axis relation depends on containment of contexts: notes (created in a Thread or Voice context) are put in the staff where the originating context lives in.

Graphical_axis_groups are special graphic objects, that are designed to function as a parent.  The size of a Graphical_axis_groups group is the union of its children.

The positions are, as explained relative to a parent reference point. Most positions are not known when an object is created, so these are calculated as needed. This is done by adding a callback for a specific direction, eg

             Real
             my_translate (Score_element * ptr, Axis a)
             {
                     return 5.0 PT;
             }

             [..]
             my_element->add_offset_callback (my_translate, Y_AXIS)

When a call is made to my_element->relative_position (obj, Y_AXIS), my_translate will be called. The result is that my_element will be translated up by 5 pt.  There are numerous callbacks, for example

  * to offset element by staff-spaces (See class Staff_symbol_referencer).

  * to align elements next to other groups of elements (See class Side_position_interface)

  * to

Offset callbacks can be stacked. The callbacks will be executed in the order that they were added.

Width and height are similarly implemented using extent callbacks. There can be only one callback for each axis. No callback (the 0 ptr) means: "empty in this direction".

Score elements can have other properties besides positioning, for example, text strings (for text objects) style settings, glyphs, padding settings (for scripts). These settings are stored in element properties.

Properties are stored as GUILE association lists, with symbols as keys. Element properties can be accessed using the C++ functions

       SCM  get_elt_property (SCM) const;
       void set_elt_property (const char * , SCM val);
       void set_immutable_elt_property (const char * , SCM val);
       void set_immutable_elt_property (SCM key, SCM val);
       void set_elt_property (SCM , SCM val);
       void set_elt_pointer (const char*, SCM val);
       SCM  remove_elt_property (const char* nm);

All lookup functions identify undefined properties with GUILE end-of-list (ie. '() in Scheme or SCM_EOL in C)

Implementation wise, there are two kinds of properties:

  * mutable properties: element properties that change from object to object. The storage of these are private to a Score element. Typically this is used to store lists of pointers to other objects

  * immutable properties: element properties that are shared across objects. The storage is shared, and hence is read-only. Typically, this is used to store function callbacks, and values for shared element properties are read from ly/engraver.ly.

     The following is from lily 1.3.80, and it shows the settings for the bar numbers: Bar numbers are  breakable, and visible at the beginning of the line. The setting for molecule-callback indicates that Bar_number is implemented as a text.

          	basicBarNumberProperties = #`(
          		(molecule-callback . ,Text_item::brew_molecule)
          		(breakable . #t)
          		(visibility-lambda . ,begin-of-line-visible)
          	)

In 1.3.81 an music expression was added to add to the immutable property list, eg. like this:

             \pushproperty #'(basicBarNumberProperties)
     		#'visibility-lambda #end-of-line-visible

This will add the entry `(visibility-lambda . ,end-of-line-visible) to the immutable property list for bar numbers, in effect overriding the setting from ly/engraver.ly. This can be undone as follows

             \popproperty #'(basicBarNumberProperties)
     		#'visibility-lambda

Note that you must accompany these statements with a proper context selection in most cases.

[FIXME: we want to get rid of dependencies in the implementation.]

Besides relative positions there are lots of other relations between elements. Lilypond does not contain other specialized relation management (Like the relative positioning code).  Instead, objects can be connected through dependencies, which sets the order in which objects are to be processed.

Example: the direction of a beamed stem should equal the direction of the beam.  When the stem is a added to the beam, a dependency on the beam is set in the stem: this means that Beam::do_pre_processing () (which does various direction related things) will be called before Stem::do_pre_processing ().

The code that manages dependencies resides in the class Score_element, a derived class of Graphical_element.  The bulk of the code handles line breaking related issues.

To sketch the problems with line breaking: suppose a slur spans a line break,

     c4(  c'''' c | \break d d )d

In this case, the slur will appear as two parts, the first part spanning the first three notes (the cs), the second spanning the last three (the ds).  Within Lilypond this is modeled as breaking the slur in parts: from the Original slur, two new clones of the old slur are made. Initially both clones depend on the six notes.  After the hairy code in Score_element, Spanner and Item which does substitutions in sets of dependencies, the first clone depends on the first three notes, the second on the last three.

The major derived classes of Score_element are Item and  Spanner. An item has one horizontal position.  A spanner hangs on two  items.

Items
=====

An item is a score element that is associated with only one Paper_column. Examples are note heads, clefs, super and superscripts, etc. Item is a derived class of Score_element.

The shape of an item is known before the break calculations, and line spacing depends on the size of items: very wide items need more space than very small ones.

An additional complication is the behavior of items at linebreaks.  For example, when you do a time signature change, you get only one symbol. If it occurs at a linebreak, the new time signature must be printed both before and after the linebreak.  Other `breakable symbols' such as clefs, and bar lines also exhibit this behavior.

if a line of music is broken, the next line usually gets a clef. So in TeX terms, the clef is a postbreak. The same thing happens with meter signs: Normally the meter follows the bar. If a line is broken at that bar, the bar along with the meter stays on the "last" line, but the next line also gets a meter sign after the clef.   To support this, breakable items are generated in the three versions: original (unbroken), left (before line break) and right (after line break). During the line spacing, these versions are used to try how the spacing of a  line works out.

Once the definitive spacing is determined, dependencies (and various other pointers) are substituted such that all dependencies point at the active items: either they point at the original, or they point at left and right.

Spanners
========

Spanners are symbols that are of variable shape, eg. Slurs, beams, etc. Spanners is a derived class of Score_element.

The final shape can only be determined after the line breaking process. All spanners are spanned on two items, called the left and right boundary item.  The X reference point is the left boundary item.

Future work
===========

There are plans to unify Spanner and Item, so there will no longer be such a clear distinction between the two.  Right now, Score_elements are always either Item or either Spanner.

CodingStyle - standards while programming for GNU LilyPond
**********************************************************

Functions and methods do not return errorcodes: they never crash, but report a programming_error and try to carry on.q

Languages
---------

C++ and Python are preferred.  Perl is forbidden.  Python code should use an indent of 8, using TAB characters.

Filenames
---------

Definitions of classes that are only accessed via pointers (*) or references (&) shall not be included as include files.

filenames

     	".hh"	Include files
     	".cc"	Implementation files
     	".icc"	Inline definition files
     	".tcc"	non inline Template defs

in emacs:

     	(setq auto-mode-alist
     	      (append '(("\\.make$" . makefile-mode)
     			("\\.cc$" . c++-mode)
     			("\\.icc$" . c++-mode)
     			("\\.tcc$" . c++-mode)
     			("\\.hh$" . c++-mode)
     			("\\.pod$" . text-mode)
     			)
     		      auto-mode-alist))

The class Class_name_abbreviation is coded in class-name-abbr.*

Indentation
-----------

Standard GNU coding style is used.   In emacs:

     	(add-hook 'c++-mode-hook
     		  '(lambda() (c-set-style "gnu")
     		     )
     		  )

If you like using font-lock, you can also add this to your .emacs:

     	(setq font-lock-maximum-decoration t)
     	(setq c++-font-lock-keywords-3
     	      (append
     	       c++-font-lock-keywords-3
     	       '(("\\b\\([a-zA-Z_]+_\\)\\b" 1 font-lock-variable-name-face)
     	       ("\\b\\([A-Z]+[a-z_]+\\)\\b" 1 font-lock-type-face))
     	       ))

Classes and Types
-----------------

     	This_is_a_class

Members
-------

     	Class::member ()
     	Type Class::member_type_
     	Type Class::member_type ()

the type is a Hungarian notation postfix for Type. See below

Macros
------

Macros should be written completely in uppercase

The code should not be compilable if proper macro declarations are not included.

Don't laugh. It took us a whole evening/night to figure out one of these bugs, because we had a macro that looked like DECLARE_VIRTUAL_FUNCTIONS ().

Broken code
-----------

Broken code (hardwired dependencies, hardwired constants, slow algorithms and obvious limitations) should be marked as such: either with a verbose TODO, or with a short "ugh" comment.

Comments
--------

The source is commented in the DOC++ style.  Check out doc++ at http://www.zib.de/Visual/software/doc++/index.html

     	/*
     		C style comments for multiline comments.
     		They come before the thing to document.
     		[...]
     	*/

     	/**
     		short description.
     		Long class documentation.
     		(Hungarian postfix)

     		TODO Fix boring_member ()
     	*/
     	class Class {
     		/**
     		  short description.
     		  long description
     		*/

     		Data data_member_;

     		/**
     			short memo. long doco of member ()
     			@param description of arguments
     			@return Rettype
     		*/
     		Rettype member (Argtype);

     		/// memo only
     		boring_member () {
     			data_member_ = 121; // ugh
     		}
     	};

Unfortunately most of the code isn't really documented that good.

Hungarian notation naming convention
====================================

Proposed is a naming convention derived from the so-called Hungarian Notation.  Macros, enums and consts are all uppercase, with the parts of the names separated by underscores.

The hungarian notation  is to be used when variables are not declared near usage (mostly in member variables and functions).

Types
-----

byte
     unsigned char. (The postfix _by is ambiguous)

b
     bool

bi
     bit

ch
     char

f
     float

i
     signed integer

str
     string class

sz
     Zero terminated c string

u
     unsigned integer

User defined types
------------------

     	/** Slur blah. blah.
     	(slur)
     	*/
     	class Slur {};
     	Slur* slur_p = new Slur;

Modifiers
---------

The following types modify the meaning of the prefix. These are preceded by the prefixes:

a
     array

array
     user built array.

c
     const. Note that the proper order is Type const i.s.o. const Type

C
     A const pointer. This would be equivalent to _c_l, but since any "const" pointer has to be a link (you can't delete a const pointer), it is superfluous.

l
     temporary pointer to object (link)

p
     pointer to newed object

r
     reference

Adjective
---------

Adjectives such as global and static should be spelled out in full. They come before the noun that they refer to, just as in normal english.

     foo_global_i: a global variable of type int commonly called "foo".

static class members do not need the static_ prefix in the name (the Class::var notation usually makes it clear that it is static)

loop_i
     Variable loop: an integer

u
     Temporary variable: an unsigned integer

test_ch
     Variable test: a character

first_name_str
     Variable first_name: a String class object

last_name_ch_a
     Variable last_name: a char array

foo_i_p
     Variable foo: an Int* that you must delete

bar_i_l
     Variable bar: an Int* that you must not delete

Generally default arguments are taboo, except for nil pointers.

The naming convention can be quite conveniently memorised, by expressing the type in english, and abbreviating it

     	static Array<int*> foo

foo can be described as "the static int-pointer user-array", so you get

     	foo_static_l_arr

Miscellaneous
=============

For some tasks, some scripts are supplied, notably creating patches, a mirror of the website, generating the header to put over cc and hh files, doing a release.

Use them.

Track and distribute your code changes
======================================

This page documents how to distribute your changes to GNU lilypond

We would like to have unified context diffs with full pathnames.  A script automating supplied with Lily.

Distributing a change normally goes like this:

  * make your fix/add your code

  * Add changes to CHANGES, and add yourself to Documentation/topdocs/AUTHORS.texi

  * generate a patch,

  * e-mail your patch to one of the mailing lists gnu-music-discuss@gnu.org or bug-gnu-music@gnu.org

Please do not send entire files, even if the patch is bigger than the original.  A patch makes it clear what is changed, and it won't overwrite previous (not yet released) changes.

Generating a patch
==================

Simple version: run

             make -C lilypond-x.y.z/ distclean
             make -C lilypond-x.y.z.NEW/ distclean
             diff -urN lilypond-x.y.z/ lilypond-x.y.z.NEW/

Complicated (but automated) version:

In VERSION, set MY_PATCH_LEVEL:

         VERSION:
     	...
     	MY_PATCH_LEVEL=jcn1

In CHANGES, enter a summary of changes:

     	pl 0.1.73.jcn1
     		- added PATCHES.texi

Then, from the top of Lily's source tree, type

         make release

These handy python scripts assume a directory structure which looks like:

         lilypond -> lilypond-x.y.z   # symlink to development directory
         lilypond-x.y.z/              # current development
         patches/ 		         # patches between different releases
         releases/                    # .tar.gz releases

(Some scripts also assume this lives in  $HOME/usr/src).

Applying patches
================

If you're following LilyPond development regularly, you probably want to download just the patch for each subsequent release. After downloading the patch (into the patches directory, of course), simply apply it:

         gzip -dc ../patches/lilypond-0.1.74.diff.gz | patch -p1 -E

and don't forget to make automatically generated files:

         autoconf footnote(patches don't include automatically generated files,
         i.e. file(configure) and files generated by file(configure).)

         configure

Localisation - User messages in LilyPond
****************************************

Introduction
============

This document provides some guidelines for uniformising user messages. In the absence of other standards, we'll be using these rules when coding for for LilyPond (In addition to the C++ coding standards that come with Lily).  Hopefully, this can be replaced by general GNU guidelines in the future.

Not-preferred messages are marked with +. By convention, agrammatical examples are marked with *.

Guidelines
==========

  * Every message to the user should be localised (and thus be marked for localisation).  This includes warning and error messages.

  * Don't localise/gettextify:

       - programming_error ()s

       - programming_warning ()s

       - debug strings

       - output strings (PostScript, TeX) (This may seem ridiculously obvious, however, makeinfo-3.12s localises output strings.  Sending bug report now ---jcn)

  * Messages to be localised must be encapsulated in _ (STRING) or _f (FORMAT, ...).  Eg:

          warning (_ ("Need music in a score"));
          error (_f ("Can't open file: `%s'", file_name));

     In some rare cases you may need to call gettext () by hand. This happens when you pre-define (a list of) string constants for later use.  In that case, you'll probably also need to mark these string constants for translation, using _i (STRING).  The _i macro is a no-op, it only serves as a marker for xgettext.

          char const* messages[] = {
            _i ("enable debugging output"),
            _i ("ignore lilypond version"),
            0
          };

          void
          foo (int i)
          {
            puts (gettext (messages [i]));
          }

     See also flower/getopt-long.cc and lily/main.cc.

  * Don't use leading or trailing whitespace in messages.

  * Messages containing a final verb, or a gerund (-ing-form) always start with a capital.  Other (simpler) messages start with a lowercase letter:

          The word `foo' is not declared.
          `foo': not declared.
          Not declaring: `foo'.

  * To avoid having a number of different messages for the same situation, we'll use quoting like this "message: `%s'" for all strings. Numbers are not quoted:

          _f ("Can't open file: `%s'", name_str)
          _f ("Can't find charater number: %d", i)

  * Think about translation issues. In a lot of cases,it's better to translate a whole message. The english grammar mustn't be imposed on the translator. So, iso

          _ ("Stem at ") + moment.str () + _(" doen't fit in beam")

     have

          _f ("Stem at %s doen't fit in beam", moment.str ())

  * Split up multi-sentence messages, whenever possible.  Instead of

          warning (_f ("out of tune!  Can't find: `%s', "Key_engraver"));

          warning (_f ("Can't find font `%s', loading default",
                       font_name));

     rather say:

          warning (_ ("out of tune:");
          warning (_f ("Can't find: `%s', "Key_engraver"));

          warning (_f ("Can't find font: `%s', font_name));
          warning (_f ("Loading default font"));

  * If you must have multiple-sentence messages, use full punctuation. Use two spaces after end of sentence punctuation. No punctuation (esp. period) is used at the end of simple messages.

             _f ("Non-matching braces in text `%s', adding braces", text)
             _ ("Debug output disabled.  Compiled with NPRINT.")
             _f ("Huh?  Not a Request: `%s'.  Ignoring.", request)

  * Don't modularise too much; a lot of words cannot be translated without context. It's probably safe to treat most occurences of words like stem, beam, crescendo as separately translatable words.

  * When translating, it is preferrable to put interesting information at the end of the message, rather than embedded in the middle. This especially applies to frequently used messages, even if this would mean sacrificing a bit of eloquency.  This holds for original messages too, of course.

              en: can't open: `foo.ly'
          +   nl: kan `foo.ly' niet openen (1)
              kan niet openen: `foo.ly'*   (2)
              niet te openen: `foo.ly'*    (3)

     The first nl message, although gramatically and stylishly correct, is not friendly for parsing by humans (even if they speak dutch). I guess we'd prefer something like (2) or (3).

  * Please don't run make po/po-update with GNU gettext < 0.10.35