release: 1.3.116

[lilypond.git] / Documentation / hacking.texi

\input texinfo @c -*-texinfo-*-
@setfilename internals.info
@settitle LilyPond internals


@node Top, LilyPond internals, (dir), (dir)
@top


@menu
* LilyPond internals::
* Overview::
* Request_engraver::            
* Coding standards::
* Making patches::
* Localisation::
@end menu

@node LilyPond internals,  , Top, Top

@menu
* Overview::                      Overview
* Request_engraver::              Request_engraver
@end menu


@chapter Getting involved

Please help us make LilyPond a better program. You can help LilyPond in
several ways. Not all tasks requiring programming or understanding the
full source code.  You can write to the mailing list
(@email{gnu-music-discuss@@gnu.org} for more information)

@unnumberedsubsec Users

Mutopia needs your help. The mutopia project is a collection of public
domain sheet music. You can help the project by entering music (either
by hand, or by converting from scans or MIDI) and submitting it. Point
your browser to the @uref{http://sca.uwaterloo.ca/Mutopia, Mutopia
webpage}.

@unnumberedsubsec Font designers

Our set of glyphs (the Feta font) is far from complete.  We need the
following glyphs: segno, coda.  The fonts have been coded in MetaFont,
so you will need to know MetaFont if you want to contribute a glyph.

@unnumberedsubsec Writers

The documentation of LilyPond and related utilities needs a lot of
work. The documentation is written in
@uref{http://www.gnu.org/software/texinfo,texinfo}. The documentation of
LilyPond is sorely lacking in terms of completeness, depth and
organisation.

Write if you know how to write english documentation in texinfo, and
know about music and music notation.  You must also know how to use
LilyPond (or be prepared to learn using it).  The task is not especially
hard, but it is a lot of work, and you must be familiar with LilyPond.

@unnumberedsubsec Translators

LilyPond is completely ready for internationalized messages, but there
are only a few translations so far (dutch, italian, german, japanese,
french, russian).  Translation involves writing a .po file, which is
relatively easy, and does not even require running LilyPond.

@unnumberedsubsec Hackers

There are lots of possibilities of improving the program itself. There
are both small projects and big ones. Most of them are listed in our
TODO file, listed on the homepage of Jan and
@uref{http://www.cs.uu.nl/~hanwen/lily-devel,Han-Wen}.  Modifying
LilyPond almost always requires patches to the C++ part.

There are also numerous other interesting projects that are more or less
related  to LilyPond

@itemize @bullet
@item Writing convertors, eg. from NIFF and MIDI (we tried writing one with
limited success: midi2ly, included with lilypond.)

We found that writing them in Python is the easiest.

@item Writing a GUI frontend to
LilyPond. At the moment @uref{denemo,denemo.sourceforge.net} is the most
advanced.

@item Helping write @uref{ http://solfege.sourceforge.net/,solfege
tools}

@item Helping @uref{primrose.sourceforge.net,primrose}, a tool for
scanning sheet music.
@end itemize


@chapter LilyPond internals


@node Overview, , , Top
@section Overview

GNU LilyPond is a "multi-pass" system.

@table @samp

@item Parsing:

No difficult algorithms. The .ly file is read, and converted to a list
of @code{Scores}, which each contain @code{Music} and paper/midi-definitions.

@item Interpreting music

The music is walked through in time-order. The iterators which do the
walking report Music to Translators which use this information to
create elements, either MIDI or "visual" elements. The translators
form a hierarchy; the ones for paper output are Engravers, for MIDI
Performers.

The translators swallow Music (mostly atomic gobs called Requests),
create elements, broadcast them to other translators on higher or same
level in the hierarchy:

The stem of a voice A is broadcast to the staff which contains A, but
not to the stems, beams and noteheads of a different voice (say B) or
a different staff. The stem and noteheads of A are coupled, because
the the Note_heads_engraver broadcasts its heads, and the Stem_engraver catches
these.

The engraver which agrees to handle a request decides whether to to
honor the request, ignore it, or merge it with other requests. Merging
of requests is preferably done with other requests done by members of
the same voicegroups (beams, brackets, stems). In this way you can put
the voices of 2 instruments in a conductor's score so they make chords
(the Beam requests of both instruments will be merged).

@item Prebreaking

Breakable stuff (eg. clefs and bars) are copied into pre and
postbreaks.

@item Preprocessing

Some dependencies are resolved, such as the direction of stems, beams,
and "horizontal" placement issues (the order of clefs,  keys etc,
placement of chords in multi-voice music), 

@item Break calculation:

The lines and horizontal positions of the columns are determined.

@item Breaking

Through some magical interactions with Line_of_score and Super_elem
(check out the source) the "lines" are produced.

All other spanners can figure across which lines they are spread. If
applicable, they break themselves into pieces. After this, each piece
(or, if there are no pieces, the original spanner itself) throws out
any dependencies which are in the wrong line.

@item Postprocesing:

Some items and all spanners need computation after the Paper_column
positions are determined. Examples: slurs, vertical positions of
staffs.

@item Output paper

@end table


@node Request_engraver, , , Top
@section Request_engraver

In the previous section the idea of Request has been explained, but
this only solves one half of the problem. The other half is deciding
which requests should be honored, which should merged with other
requests, and which should be ignored. Consider this input

@example 

        \type Staff < % chord
                @{ \meter 2/4; [c8 c8] @}
                @{\meter 2/4;  [e8 e8] @}
        >
 
@end example 

Both the cs and es are part of a staff (they are in the same
Voice_group), so they should share meters, but the two [ ] pairs
should be merged.

The judge in this "allocation" problem a set of brokers: the requests
are transmitted to so-called engravers which respond if they want to
accept a request eg, the @code{Notehead_engraver} will accept
@code{Note_req}s, and turn down @code{Slur_req}s. If the Music_iterator
cannot find a engraver that wants the request, it is junked (with a
warning message).

After all requests have been either assigned, or junked, the Engraver
will process the requests (which usually means creating an @code{Item}
or @code{Spanner}). If a @code{Request_engraver} creates something, it
tells the enclosing context. If all items/spanners have been created,
then each Engraver is notified of any created Score_element, via a
broadcasting system.

@unnumberedsubsec example:

@example 

        c4
 
@end example 

produces:

@example 

        Note_request (duration 1/4)
        Stem_request (duration 1/4)
 
@end example 

Note_request will be taken by a @code{Notehead_engraver}, stem_request
will be taken by a @code{Stem_beam_engraver}. @code{Notehead_engraver}
creates a @code{Notehead}, @code{Stem_beam_engraver} creates a
@code{Stem}. Both announce this to the Staff_engraver. Staff_engraver
will tell @code{Stem_beam_engraver} about the @code{Notehead}, which
will add the @code{Notehead} to the @code{Stem} it just created.

To decide on merging, several engravers have been grouped. Please
check @file{init/engraver.ly}.

@node Coding standards, , , Top

@chapter CodingStyle - standards while programming for GNU LilyPond

Functions and methods do not return errorcodes: they never crash, but
report a programming_error and try to carry on.q 


@unnumberedsubsec Languages

C++ and Python are preferred.  Perl is forbidden.  Python code should
use an indent of 8, using TAB characters.

@unnumberedsubsec Filenames

Definitions of classes that are only accessed via pointers
(*) or references (&) shall not be included as include files.

filenames

@example 
        ".hh"   Include files
        ".cc"   Implementation files
        ".icc"  Inline definition files
        ".tcc"  non inline Template defs
@end example 

in emacs:

@example 
        (setq auto-mode-alist
              (append '(("\\.make$" . makefile-mode)
                        ("\\.cc$" . c++-mode)
                        ("\\.icc$" . c++-mode)
                        ("\\.tcc$" . c++-mode)
                        ("\\.hh$" . c++-mode)
                        ("\\.pod$" . text-mode)         
                        )
                      auto-mode-alist))
@end example 


The class Class_name_abbreviation is coded in @file{class-name-abbr.*}

@unnumberedsubsec Indentation

Standard GNU coding style is used.   In emacs:

@example 
        (add-hook 'c++-mode-hook
                  '(lambda() (c-set-style "gnu")
                     )
                  )
@end example 

If you like using font-lock, you can also add this to your @file{.emacs}:

@example 
        (setq font-lock-maximum-decoration t)
        (setq c++-font-lock-keywords-3 
              (append
               c++-font-lock-keywords-3
               '(("\\b\\([a-zA-Z_]+_\\)\\b" 1 font-lock-variable-name-face)
               ("\\b\\([A-Z]+[a-z_]+\\)\\b" 1 font-lock-type-face))
               ))
@end example 

@unnumberedsubsec Classes and Types

@example 
        This_is_a_class
@end example 

@unnumberedsubsec Members

@example 
        Class::member ()
        Type Class::member_type_
        Type Class::member_type ()
@end example 

the @code{type} is a Hungarian notation postfix for @code{Type}. See below

@unnumberedsubsec Macros

Macros should be written completely in uppercase

@unnumberedsubsec Broken code

Broken code (hardwired dependencies, hardwired constants, slow
algorithms and obvious limitations) should be marked as such: either
with a verbose TODO, or with a short "ugh" comment.

@unnumberedsec Hungarian notation naming convention

Proposed is a naming convention derived from the so-called
@emph{Hungarian Notation}.  Macros, @code{enum}s and @code{const}s are all
uppercase, with the parts of the names separated by underscores.

The hungarian notation  is to be used when variables are not declared
near usage (mostly in member variables and functions).

@unnumberedsubsec Types

@table @samp
@item @code{byte}
    unsigned char. (The postfix _by is ambiguous)
@item @code{b}
    bool
@item @code{bi}
    bit
@item @code{ch}
    char
@item @code{f}
    float
@item @code{i}
    signed integer
@item @code{str}
    string class
@item @code{sz}
    Zero terminated c string
@item @code{u}
    unsigned integer
@end table

@unnumberedsubsec User defined types

@example 

        /** Slur blah. blah.
        (slur)
        */
        class Slur @{@};
        Slur* slur_p = new Slur;
 
@end example 

@unnumberedsubsec Modifiers

The following types modify the meaning of the prefix. 
These are preceded by the prefixes:

@table @samp
@item @code{a}
    array
@item @code{array}
    user built array.
@item @code{c}
    const. Note that the proper order is @code{Type const}
    i.s.o. @code{const Type}
@item @code{C}
    A const pointer. This would be equivalent to @code{_c_l}, but since any
    "const" pointer has to be a link (you can't delete a const pointer),
    it is superfluous.
@item @code{l}
    temporary pointer to object (link)
@item @code{p}
    pointer to newed object
@item @code{r}
    reference
@end table

@unnumberedsubsec Adjective

Adjectives such as global and static should be spelled out in full.
They come before the noun that they refer to, just as in normal english.

@example 

foo_global_i: a global variable of type int commonly called "foo".
 
@end example 

static class members do not need the static_ prefix in the name (the
Class::var notation usually makes it clear that it is static)

@table @samp
@item @code{loop_i}
    Variable loop: an integer
@item @code{u}
    Temporary variable: an unsigned integer
@item @code{test_ch}
    Variable test: a character
@item @code{first_name_str}
    Variable first_name: a String class object
@item @code{last_name_ch_a}
    Variable last_name: a @code{char} array
@item @code{foo_i_p}
    Variable foo: an @code{Int*} that you must delete
@item @code{bar_i_l}
    Variable bar: an @code{Int*} that you must not delete
@end table

Generally default arguments are taboo, except for nil pointers.

The naming convention can be quite conveniently memorised, by
expressing the type in english, and abbreviating it

@example 

        static Array<int*> foo
 
@end example 

@code{foo} can be described as "the static int-pointer user-array", so you get

@example 

        foo_static_l_arr
 
@end example 


@unnumberedsec Miscellaneous
    
For some tasks, some scripts are supplied, notably creating patches, a
mirror of the website, generating the header to put over cc and hh
files, doing a release.

Use them.

@node Making patches, , , Top


@unnumberedsec  Track and distribute your code changes

This page documents how to distribute your changes to GNU lilypond
    
We would like to have unified context diffs with full pathnames.  A
script automating supplied with Lily.

Distributing a change normally goes like this:

@itemize @bullet
@item make your fix/add your code 
@item Add changes to CHANGES, and add yourself to Documentation/topdocs/AUTHORS.texi
@item generate a patch, 
@item e-mail your patch to one of the mailing lists
    gnu-music-discuss@@gnu.org or bug-gnu-music@@gnu.org
@end itemize

Please do not send entire files, even if the patch is bigger than the
original.  A patch makes it clear what is changed, and it won't
overwrite previous (not yet released) changes.

@unnumberedsec Generating a patch

Simple version: run

@example
        make -C lilypond-x.y.z/ distclean
        make -C lilypond-x.y.z.NEW/ distclean
        diff -urN lilypond-x.y.z/ lilypond-x.y.z.NEW/
@end example

Complicated (but automated) version:

In @file{VERSION}, set MY_PATCH_LEVEL:

@example 

    VERSION:
        ...
        MY_PATCH_LEVEL=jcn1
 
@end example 

In @file{CHANGES}, enter a summary of changes:

@example 
        pl 0.1.73.jcn1
                - added PATCHES.texi
@end example 

Then, from the top of Lily's source tree, type

@example 
    make release
@end example 

These handy python scripts assume a directory structure which looks
like:

@example 

    lilypond -> lilypond-x.y.z   # symlink to development directory
    lilypond-x.y.z/              # current development
    patches/                     # patches between different releases
    releases/                    # .tar.gz releases

@end example 

(Some scripts also assume this lives in  @file{$HOME/usr/src}).

        
@unnumberedsec Applying patches
    

If you're following LilyPond development regularly, you probably want to
download just the patch for each subsequent release.
After downloading the patch (into the patches directory, of course), simply 
apply it:

@example 

    gzip -dc ../patches/lilypond-0.1.74.diff.gz | patch -p1 -E
 
@end example 

and don't forget to make automatically generated files:

@example 

    autoconf footnote(patches don't include automatically generated files, 
    i.e. file(configure) and files generated by file(configure).)

    configure
 
@end example 

@node Localisation, , , Top

@chapter Localisation - User messages in LilyPond

@section Introduction

This document provides some guidelines for uniformising user messages.
In the absence of other standards, we'll be using these rules when coding
for for LilyPond@footnote{
In addition to the C++ coding standards that come with Lily
}.  Hopefully, this can be replaced by general GNU
guidelines in the future.  

Not-preferred messages are marked with @code{+}.
By convention, agrammatical examples are marked with @code{*}.


@section Guidelines

@itemize @bullet

@item
Every message to the user should be localised (and thus be marked
for localisation).  This includes warning and error messages.

@item
Don't localise/gettextify:

@itemize @minus
@item @code{programming_error ()}s
@item @code{programming_warning ()}s
@item debug strings
@item output strings (PostScript, TeX)@footnote{
This may seem ridiculously obvious, however, makeinfo-3.12s localises
output strings.  Sending bug report now ---jcn
}
@end itemize



@item
Messages to be localised must be encapsulated in @code{_ (STRING)}
or @code{_f (FORMAT, ...)}.  Eg:

@example
warning (_ ("Need music in a score"));
error (_f ("Can't open file: `%s'", file_name));
@end example

In some rare cases you may need to call @code{gettext ()} by hand.
This happens when you pre-define (a list of) string constants for later
use.  In that case, you'll probably also need to mark these string
constants for translation, using @code{_i (STRING)}.  The @code{_i}
macro is a no-op, it only serves as a marker for @file{xgettext}.

@example
char const* messages[] = @{
  _i ("enable debugging output"),
  _i ("ignore lilypond version"),
  0
@};

void
foo (int i)
@{
  puts (gettext (messages [i]));
@}
@end example

See also
@file{flower/getopt-long.cc} and @file{lily/main.cc}.

@item
Don't use leading or trailing whitespace in messages.

@item
Messages containing a final verb, or a gerund (@code{-ing}-form)
always start with a capital.  Other (simpler) messages start with
a lowercase letter:

@example
The word `foo' is not declared.
`foo': not declared.
Not declaring: `foo'.
@end example

@item
To avoid having a number of different messages for the same situation,
we'll use quoting like this @code{"message: `%s'"} for all strings.
Numbers are not quoted:

@example
_f ("Can't open file: `%s'", name_str)
_f ("Can't find charater number: %d", i)
@end example

@item
Think about translation issues.  
In a lot of cases,it's better to translate a whole message.
The english grammar mustn't be imposed on the translator.
So, iso

@example
_ ("Stem at ") + moment.str () + _(" doen't fit in beam")
@end example

@noindent
have

@example
_f ("Stem at %s doen't fit in beam", moment.str ())
@end example

@item
Split up multi-sentence messages, whenever possible.  Instead of

@example
warning (_f ("out of tune!  Can't find: `%s', "Key_engraver"));

warning (_f ("Can't find font `%s', loading default", 
             font_name));
@end example

@noindent
rather say:

@example
warning (_ ("out of tune:");
warning (_f ("Can't find: `%s', "Key_engraver"));

warning (_f ("Can't find font: `%s', font_name));
warning (_f ("Loading default font"));
@end example

@item
If you must have multiple-sentence messages, use full punctuation.
Use two spaces after end of sentence punctuation.
No punctuation (esp. period) is used at the end of simple messages.

@example
   _f ("Non-matching braces in text `%s', adding braces", text)
   _ ("Debug output disabled.  Compiled with NPRINT.")
   _f ("Huh?  Not a Request: `%s'.  Ignoring.", request)
@end example

@item
Don't modularise too much; a lot of words cannot be translated
without context.
It's probably safe to treat most occurences of words like
stem, beam, crescendo as separately translatable words.

@item
When translating, it is preferrable to put interesting information 
at the end of the message, rather than embedded in the middle.
This especially applies to frequently used messages, even if this
would mean sacrificing a bit of eloquency.  This holds for original
messages too, of course.

@example
    en: can't open: `foo.ly'
+   nl: kan `foo.ly' niet openen (1)
    kan niet openen: `foo.ly'*   (2)
    niet te openen: `foo.ly'*    (3)
@end example

The first nl message, although gramatically and stylishly correct,
is not friendly for parsing by humans (even if they speak dutch).
I guess we'd prefer something like (2) or (3).

@item
Please don't run make po/po-update with GNU gettext < 0.10.35

@end itemize

@bye

    

@bye