release: 1.2.11

[lilypond.git] / Documentation / hacking.texi

\input texinfo @c -*-texinfo-*-
@setfilename internals.info
@settitle LilyPond internals

@node Top, LilyPond internals, (dir), (dir)
@top


@menu
* LilyPond internals::
* Overview::
* mudela::                      
* Request_engraver::            
* Items and spanners::          
* Dependencies::                
* Breaking::                    
* Coding standards::
* Making patches::
@end menu

@node LilyPond internals,  , Top, Top
@menu
* Overview::                      Overview
* mudela::                        mudela
* Request_engraver::              Request_engraver
* Items and spanners::            Items and spanners
* Dependencies::                  Dependencies
* Breaking::                      Breaking
@end menu

@chapter LilyPond internals


This documents some aspects of the internals of GNU LilyPond. Some of
this stuff comes from e-mail I wrote, some from e-mail others wrote,
some are large comments taken away from the headers. This page may be
a little incoherent.  Unfortunately, it is also quite outdated.  A
more thorough  and understandable document is in the works.

You should use @code{doc++} to take a peek at the sources.

@node Overview, mudela, Top, Top
@section Overview

GNU LilyPond is a "multi-pass" system. The different passes have been
created so that they do not depend on each other. In a later stage
some parts may be moved into libraries, or seperate programs, or they
might be integrated in larger systems.

@table @samp

@item Parsing:

No difficult algorithms. The .ly file is read, and converted to a list
of @code{Scores}, which each contain @code{Music} and paper/midi-definitions.

@item Interpreting music

The music is walked through in time-order. The iterators which do the
walking report Music to Translators which use this information to
create elements, either MIDI or "visual" elements. The translators
form a hierarchy; the ones for paper output are Engravers, for MIDI
Performers.

The translators swallow Music (mostly atomic gobs called Requests),
create elements, broadcast them to other translators on higher or same
level in the hierarchy:

The stem of a voice A is broadcast to the staff which contains A, but
not to the stems, beams and noteheads of a different voice (say B) or
a different staff. The stem and noteheads of A are coupled, because
the the Note_heads_engraver broadcasts its heads, and the Stem_engraver catches
these.

The engraver which agrees to handle a request decides whether to to
honor the request, ignore it, or merge it with other requests. Merging
of requests is preferably done with other requests done by members of
the same voicegroups (beams, brackets, stems). In this way you can put
the voices of 2 instruments in a conductor's score so they make chords
(the Beam requests of both instruments will be merged).

@item Prebreaking

Breakable stuff (eg. clefs and bars) are copied into pre and
postbreaks.

@item Preprocessing

Some dependencies are resolved, such as the direction of stems, beams,
and "horizontal" placement issues (the order of clefs,  keys etc,
placement of chords in multi-voice music), 

@item Break calculation:

The lines and horizontal positions of the columns are determined.

@item Breaking

Through some magical interactions with Line_of_score and Super_elem
(check out the source) the "lines" are produced.

All other spanners can figure across which lines they are spread. If
applicable, they break themselves into pieces. After this, each piece
(or, if there are no pieces, the original spanner itself) throws out
any dependencies which are in the wrong line.

@item Postprocesing:

Some items and all spanners need computation after the Paper_column
positions are determined. Examples: slurs, vertical positions of
staffs.

@item Output paper

@end table

@node mudela, Request_engraver, Overview, Top
@section mudela

Most information is stored in the form of a request.  In music
typesetting, the user might want to cram a lot more symbols on the
paper than actually fits. To reflect this idea (the user asks more
than we can do), the container for this data is called Request.

In a lot of other formats this would be called an 'Event'

@table @samp
@item @code{Barcheck_req}
    Checks during music processing if start of this voice element
    coincides with the start of a measure. Handy to check if you left out
    some voice elts.
@item @code{Note_req}
    LilyPond has to decide if the ball should be hanging left or
    right. This influences the horizontal dimensions of a column, and this
    is why request processing should be done before horizontal spacing.
    Other voices' frivolities may cause the need for accidentals, so this
    is also for the to decide. The engraver can decide on positioning based on
    ottava commands and the appropriate clef.
@item @code{Rest_req}
    Typeset a rest.
@item @code{Span_req}
    This type of request typically results in the creation of a @code{Spanner}
@item @code{Beam_req}
    Start/stop a beam.
    Engraver has to combine this request with the stem_request, since the
    number of flags that a stem wants to carry will determine the
    number of beams.
@item @code{Dynamic}
    Each dynamic is bound to one note (a crescendo spanning multiple
    notes is thought to be made of two "dynamics": a start and a stop).
    Dynamic changes can occur in a smaller time than the length of its
    note, therefore fore each  @code{Dynamic} request carries a time, measured
    from the start of its note.
@end table

@node Request_engraver, Items and spanners, mudela, Top
@section Request_engraver

In the previous section the idea of Request has been explained, but
this only solves one half of the problem. The other half is deciding
which requests should be honored, which should merged with other
requests, and which should be ignored. Consider this input

@example 

        \type Staff < % chord
                @{ \meter 2/4; [c8 c8] @}
                @{\meter 2/4;  [e8 e8] @}
        >
 
@end example 

Both the cs and es are part of a staff (they are in the same
Voice_group), so they should share meters, but the two [ ] pairs
should be merged.

The judge in this "allocation" problem a set of brokers: the requests
are transmitted to so-called engravers which respond if they want to
accept a request eg, the @code{Notehead_engraver} will accept
@code{Note_req}s, and turn down @code{Slur_req}s. If the Music_iterator
cannot find a engraver that wants the request, it is junked (with a
warning message).

After all requests have been either assigned, or junked, the Engraver
will process the requests (which usually means creating an @code{Item}
or @code{Spanner}). If a @code{Request_engraver} creates something, it
tells the enclosing context. If all items/spanners have been created,
then each Engraver is notified of any created Score_element, via a
broadcasting system.

@unnumberedsubsec example:

@example 

        c4
 
@end example 

produces:

@example 

        Note_request (duration 1/4)
        Stem_request (duration 1/4)
 
@end example 

Note_request will be taken by a @code{Notehead_engraver}, stem_request
will be taken by a @code{Stem_beam_engraver}. @code{Notehead_engraver}
creates a @code{Notehead}, @code{Stem_beam_engraver} creates a
@code{Stem}. Both announce this to the Staff_engraver. Staff_engraver
will tell @code{Stem_beam_engraver} about the @code{Notehead}, which
will add the @code{Notehead} to the @code{Stem} it just created.

To decide on merging, several engravers have been grouped. Please
check @file{init/engraver.ly}.

@node Items and spanners, Dependencies, Request_engraver, Top
@section Items and spanners

The symbols that are printed, are generated by items and spanners
(staff-elements). An item has one horizontal position, whereas a
spanner spans several columns.

@node Dependencies, Breaking, Items and spanners, Top
@section Dependencies

In music symbols depend on each other: the stems of a beam should
point in the same direction as the beam itself, so the stems of a beam
depend on the beam. In the same way do scripts depend on the direction
of the stem. To reflect this, LilyPond has the notion of dependency.
It works in the same fashion that @code{make} uses to build programs:
before a stem is calculated, its dependencies (the beam) should be
calculated. Before a slur is calculated, its dependencies (stems,
noteheads) should be calculated.

@node Breaking, Making patches, Dependencies, Top
@section Breaking

So what is this PREBREAK and POSTBREAK stuff?

Let's take text as an example. In German some compound
words change their spelling if they are broken: "backen" becomes
"bak-ken".  TeX has a mechanism to deal with this, you would define
the spelling of "backen" in TeX in this way

        \discretionary@{bak-@}@{ken@}@{backen@}

These 3 arguments are called "prebreak", "postbreak" and "nobreak"
text.

The same problem exists when typesetting music. If a line of music is
broken, the next line usually gets a clef. So in TeX terms, the clef
is a postbreak. The same thing happens with meter signs: Normally the
meter follows the bar. If a line is broken at that bar, the bar along
with the meter stays on the "last" line, but the next line also gets a
meter sign after the clef. Using the previous notation,

        \discretionary@{bar meter@}@{clef meter@}@{ bar meter @}

In GNU Lilypond, we have the same concepts (and the same
terminology). Each (nonrhythmic) symbol is typeset in  a nonrhythmic column
At a breakpoint, multiple symbols are printed; symbols to be printed
if the line is not broken, symbols to appear on the previous line, and
on the next line if it is broken.

@node Coding standards,  , Top, Top

@chapter CodingStyle - standards while programming for GNU
LilyPond

We use these standards while doing programming for GNU LilyPond.  If
you do some hacking, we appreciate it if you would follow this rules,
but if you don't, we still like you.

Functions and methods do not return errorcodes.

@quotation

A program should be light and agile, its subroutines
connected like a string of pearls.  The spirit and intent of
the program should be retained throughout.  There should be
neither too little nor too much, neither needless loops nor
useless variables, neither lack of structure nor overwhelming
rigidity.

A program should follow the 'Law of Least
Astonishment'.  What is this law?  It is simply that the
program should always respond to the user in the way that
astonishes him least.

A program, no matter how complex, should act as a
single unit.  The program should be directed by the logic
within rather than by outward appearances.

If the program fails in these requirements, it will be
in a state of disorder and confusion.  The only way to correct
this is to rewrite the program.

-- Geoffrey James, "The Tao of Programming"
@end quotation


@unnumberedsubsec Languages

C++ and Python are preferred.  Perl is not.  Python code should use an
indent of 8, using TAB characters.

@unnumberedsubsec Filenames

Definitions of classes that are only accessed via pointers
(*) or references (&) shall not be included as include files.

filenames

@example 

        ".hh"   Include files
        ".cc"   Implementation files
        ".icc"  Inline definition files
        ".tcc"  non inline Template defs
 
@end example 

in emacs:

@example 

        (setq auto-mode-alist
              (append '(("\\.make$" . makefile-mode)
                        ("\\.cc$" . c++-mode)
                        ("\\.icc$" . c++-mode)
                        ("\\.tcc$" . c++-mode)
                        ("\\.hh$" . c++-mode)
                        ("\\.pod$" . text-mode)         
                        )
                      auto-mode-alist))
 
@end example 


The class Class_name_abbreviation is coded in @file{class-name-abbr.*}

@unnumberedsubsec Indentation

Standard GNU coding style is used.   In emacs:

@example 

        (add-hook 'c++-mode-hook
                  '(lambda() (c-set-style "gnu")
                     )
                  )
 
@end example 

If you like using font-lock, you can also add this to your @file{.emacs}:

@example 

        (setq font-lock-maximum-decoration t)
        (setq c++-font-lock-keywords-3 
              (append
               c++-font-lock-keywords-3
               '(("\\b\\([a-zA-Z_]+_\\)\\b" 1 font-lock-variable-name-face)
               ("\\b\\([A-Z]+[a-z_]+\\)\\b" 1 font-lock-type-face))
               ))
 
@end example 

@unnumberedsubsec Classes and Types

@example 

        This_is_a_class
 
@end example 

@unnumberedsubsec Members

@example 

        Class::member ()
        Type Class::member_type_
        Type Class::member_type ()
 
@end example 

the @code{type} is a Hungarian notation postfix for @code{Type}. See below

@unnumberedsubsec Macros

Macros should be written completely in uppercase

The code should not be compilable if proper macro declarations are not
included. 

Don't laugh. It took us a whole evening/night to figure out one of
these bugs, because we had a macro that looked like
@code{DECLARE_VIRTUAL_FUNCTIONS ()}. 

@unnumberedsubsec Broken code

Broken code (hardwired dependencies, hardwired constants, slow
algorithms and obvious limitations) should be marked as such: either
with a verbose TODO, or with a short "ugh" comment.

@unnumberedsubsec Comments

The source is commented in the DOC++ style.  Check out doc++ at
@uref{http://www.zib.de/Visual/software/doc++/index.html}

@example 

        /*
                C style comments for multiline comments.
                They come before the thing to document.
                [...]
        */

        /**
                short description.
                Long class documentation.
                (Hungarian postfix)

                TODO Fix boring_member ()
        */
        class Class @{
                /**
                  short description.
                  long description
                */

                Data data_member_;

                /**
                        short memo. long doco of member ()
                        @@param description of arguments
                        @@return Rettype
                */
                Rettype member (Argtype);

                /// memo only
                boring_member () @{
                        data_member_ = 121; // ugh
                @}
        @};
 
@end example 

        
Unfortunately most of the code isn't really documented that good.

@unnumberedsubsec Members (2)

Standard methods:

@example 

        ///check that *this satisfies its invariants, abort if not.
        void OK () const

        /// print *this (and substructures) to debugging log
        void print () const

        /**
        protected member. Usually invoked by non-virtual XXXX ()
        */
        virtual do_XXXX ()

        /**add some data to *this.
        Presence of these methods usually imply that it is not feasible to this
        via  a constructor
        */
        add (..)

        /// replace some data of *this
        set (..)
 
@end example 

@unnumberedsubsec Constructor

Every class should have a default constructor.  

Don't use non-default constructors if this can be avoided:

@example 

        Foo f(1)
 
@end example 

is less readable than

@example 

        Foo f;
        f.x = 1
 
@end example 

or 

@example 

        Foo f(Foo_convert::int_to_foo (1))
 
@end example 

@unnumberedsec Hungarian notation naming convention

Proposed is a naming convention derived from the so-called
@emph{Hungarian Notation}.  Macros, @code{enum}s and @code{const}s are all
uppercase, with the parts of the names separated by underscores.

@unnumberedsubsec Types

@table @samp
@item @code{byte}
    unsigned char. (The postfix _by is ambiguous)
@item @code{b}
    bool
@item @code{bi}
    bit
@item @code{ch}
    char
@item @code{f}
    float
@item @code{i}
    signed integer
@item @code{str}
    string class
@item @code{sz}
    Zero terminated c string
@item @code{u}
    unsigned integer
@end table

@unnumberedsubsec User defined types

@example 

        /** Slur blah. blah.
        (slur)
        */
        class Slur @{@};
        Slur* slur_p = new Slur;
 
@end example 

@unnumberedsubsec Modifiers

The following types modify the meaning of the prefix. 
These are preceded by the prefixes:

@table @samp
@item @code{a}
    array
@item @code{array}
    user built array.
@item @code{c}
    const. Note that the proper order is @code{Type const}
    i.s.o. @code{const Type}
@item @code{C}
    A const pointer. This would be equivalent to @code{_c_l}, but since any
    "const" pointer has to be a link (you can't delete a const pointer),
    it is superfluous.
@item @code{l}
    temporary pointer to object (link)
@item @code{p}
    pointer to newed object
@item @code{r}
    reference
@end table

@unnumberedsubsec Adjective

Adjectives such as global and static should be spelled out in full.
They come before the noun that they refer to, just as in normal english.

@example 

foo_global_i: a global variable of type int commonly called "foo".
 
@end example 

static class members do not need the static_ prefix in the name (the
Class::var notation usually makes it clear that it is static)

@table @samp
@item @code{loop_i}
    Variable loop: an integer
@item @code{u}
    Temporary variable: an unsigned integer
@item @code{test_ch}
    Variable test: a character
@item @code{first_name_str}
    Variable first_name: a String class object
@item @code{last_name_ch_a}
    Variable last_name: a @code{char} array
@item @code{foo_i_p}
    Variable foo: an @code{Int*} that you must delete
@item @code{bar_i_l}
    Variable bar: an @code{Int*} that you must not delete
@end table

Generally default arguments are taboo, except for nil pointers.

The naming convention can be quite conveniently memorised, by
expressing the type in english, and abbreviating it

@example 

        static Array<int*> foo
 
@end example 

@code{foo} can be described as "the static int-pointer user-array", so you get

@example 

        foo_static_l_arr
 
@end example 


@unnumberedsec Miscellaneous
    

For some tasks, some scripts are supplied, notably creating patches, a
mirror of the website, generating the header to put over cc and hh
files, doing a release.

Use them.

@node Making patches,  , Breaking, Top


@unnumberedsec name
    

PATCHES - track and distribute your code changes

This page documents how to distribute your changes to GNU lilypond
    
We would like to have unified context diffs with full pathnames.  A
script automating supplied with Lily.

 Distributing a change normally goes like this:

@itemize @bullet
@item make your fix/add your code 
@item Add changes to NEWS, and add yourself to Documentation/topdocs/AUTHORS.yo
@item generate a patch, 
@item e-mail your patch to one of the mailing lists
    gnu-music-discuss@@gnu.org or bug-gnu-music@@gnu.org
@end itemize

Please do not send entire files, even if the patch is bigger than the
original.  A patch makes it clear what is changed, and it won't
overwrite previous not yet released changes.

@unnumberedsec Generating a patch
    

In @file{VERSION}, set MY_PATCH_LEVEL:

@example 

    VERSION:
        ...
        MY_PATCH_LEVEL=jcn1
 
@end example 

In @file{CHANGES}, enter a summary of changes:

@example 
        pl 0.1.73.jcn1
                - added PATCHES.texi
@end example 

Then, from the top of Lily's source tree, type

@example 

    make dist
    make diff
 
@end example 

which rolls the tarball @file{../releases/lilypond-0.1.73.tar.gz}
and leaves your patch as @file{./lilypond-0.1.73.jcn1.diff}.
@footnote{'Make diff' generates a patch between two tarballs.  For 
more info type 'make diff help=='.}  We assume that there is a tarball 
@file{lilypond-0.1.73.tar.gz} in the directory @file{../releases}.

If you didn't configure Lily using --srcdir, you can do:

@example 

    make release

    tar-ball: ../patches/lilypond-0.1.73.jcn1.gz
    patch: ../patches/lilypond-0.1.73.jcn1.gz
@end example 

@unnumberedsec Prerequisites
    

For creating a patch you need

@itemize @bullet
@item All items mentioned in @file{INSTALL}.  You're not going to send a patch
    that you haven't even built, right?
@item GNU diff
@example 
    make distclean
    cd ..
    diff -urN lilypond-0.1.73 lilypond-0.1.73.jcn1 > lilypond-0.1.73.jcn1
@end example 

but there are handy python scripts available.  If you're doing development,
you'll need Python for other LilyPond scripts anyway.

@item The Lily directory structure, which looks like:

    @example 

    doos/                        # gnu/windows32 build and binary releases
    harmonia -> harmonia-x.y.z 
    harmonia-x.y.z/
    lilypond -> lilypond-x.y.z   # symlink to development directory
    lilypond-x.y.z/              # current development
    patches/                     # patches between different releases
    RedHat/BUILD                 # RedHat build and binary releases
    RedHat/RPMS
    RedHat/SPECS
    releases/                    # .tar.gz releases
    test/                        # tarballs and diffs from current version

 
@end example 

with prefix @file{$HOME/usr/src}
and (for building rpms only) in @file{$HOME/.rpmrc}:
@example 

    topdir: /home/fred/usr/src/RedHat
 
@end example 

@end itemize
        
@unnumberedsec Applying patches
    

If you're following LilyPond development regularly, you probably want to
download just the patch for each subsequent release.
After downloading the patch (into the patches directory, of course), simply 
apply it:

@example 

    gzip -dc ../patches/lilypond-0.1.74.diff.gz | patch -p1 -E
 
@end example 

and don't forget to make automatically generated files:

@example 

    autoconf footnote(patches don't include automatically generated files, 
    i.e. file(configure) and files generated by file(configure).)

    configure
 
@end example 

@unnumberedsec Synchronise
    

If you're not very quick with sending your patch, there's a good
chance that an new release of LilyPond comes available.  In such a
case, the maintainer will probably ask you to make a new patch against
the latest release.  Your best bet is to download the latest release,
and apply your patch against this new source tree:

@example 

    cd lilypond-0.1.74
    gzip -dc ../patches/lilypond-0.1.73.jcn1.diff.gz | patch -p1 -E
    autoconf
    configure
 
@end example 

Then, make a patch as shown above.

@unnumberedsec See also
    

@file{stepmake/INSTALL.txt}

@unnumberedsec maintainer
    

@email{hanwen@@cs.uu.nl, Han-Wen Nienhuys}

Just keep on sending those patches!


@bye