[lilypond.git] / TODO

-*-outline-layout:(2 (-1 -1 0 :) 0);outline-stylish-prefixes:nil -*-

* GNU LilyPond TODO
Features you cannot find in the doco as working, should be
mentioned here.  This is an assorted collection of stuff that will be
done, might be done, or is an idea that I want to think about

Most of the items are marked in the code as well
Grep for TODO and ugh/ugr/urg.  

.* BUGS
. * rewrite context defs using Scheme
. * PostScript
.  * header for PS enteredby = "bla <bla@bar.com>"
.  * ps/lily.ps see comments.
. * devise standard for functioning of Vertical_align_element.
. * staff size for post/prebreaks
. * .ly files
.  * input/star-spangled-banner
. * midi key.
. * P.P.S. It can be cool in mudela-book to distinguish in
pre,postMudelaExample, whether MudelaExample is epsed or not: ( if
this fragment is floating eps, than 1, otherwise 2). say
preMudelaExample[eps]{}, and change it in document body sometimes.
. * fix singleStaffBracket
. * declare performers in \midi
. * fix MIDI
. * \shape 1st dim skipped? 
. * turn slope-damping on/off
. * tremolo stuff
. * fix height of / collisions with lyrics (chords), 
see input/test/vertical-text.ly; input/test/repeat.ly
. * We need feta-din*.mf files for more sizes than 10.
. * fix dynamics decently, ie. use kerning & ligatures.  
. * dots & rest collisions?
. * mudela-book.py
.  * fix \mudelafile{} to have decent functionality
.  * use less RE's
. * support_l_arr_ empty in staff-margin's G_staff_side_item
. * slurs colllide with lyrics.
. * Is there any reasonable way to do this: I would like to have one file
that can generate the same tune in two different modes without having
to go through and rewrite all the music in the other mode.

Long lyrics don't get a space at the end.  They just run right up
against the next lyric.  (I don't think this used to be the case.)

Lyrics and scripts collide.

\key b; \time 4/4; looks bad.  The clef and time signature are too close.

restStyle=mensural doesn't seem to work (lots of characters not found
for rests.fly)
. * Phrasing:  Indicating the textual underlay of a piece, or
otherwise dividing a piece into phrases.

There are distinct ways of specifying the first two (although at
present, they're not properly distinguished) but no distinct way to
specify the third.  Should there be?
. * rest collision minimum dist.
. * Matanya's tips: 
.  * spacing for prefatory matter
.  * The line length is 7.5 inch (19.05
> cm) and the height of normal size is equivalent to the traditional raster 1
> size. (The Schott company sells Letraset sheets of musical symbols which are
> measured in raster sizes). It is agreed among most engravers I know that the
> standard size for adult readers in solo music should 70% of raster 1.  Jeff's
> example was 75%, so I had to modify my file to match. 

So that makes for a staff space of 4 2/3 point. I'll add the
dimensions to the TODO file.
.  * accidentals closer to note head
.  * to space the staffs/staffgroups contained in the score.  

In the orchestscore.ly file, I tried:
\paper{
    \translator {
        \OrchestralScoreContext
        minVerticalAlign = 3.0*\staffheight;
    }
    \translator { 
        \StaffGroupContext
        minVerticalAlign = 3.0*\staffheight;
    }
    \translator { 
        \StaffContext
        minVerticalAlign = 3.0*\staffheight;
    }
}
.  * heavier beams?

.* Cleanups needed
. * Beam
. * Stem + Notehead
. * Slur
. * Repeats
. * Rhythmic_grouping
. * Duration
. * Spring generation
. * Collision
. * Rest_collision
. * clef grav prop's 

.* TODO before 1.2
. * \selectmusic to cut pieces from music. 
. * Morally pure LilyPond.
.  * Remove mutopia ramblings.
.  * Remove non-free software links.
.  * Remove meta article LilyPond.
. * TODO: merge atom & molecule; make  tree from molecule.
. * break priority setting from SCM.
. * Gade score
.  * remove [] in favour of auto-beamer
. * hangOnClef with non-breakable clefs.
. * Aha, the problem is the new TeX code generation for changing fonts.
> (The explanation below is mostly intended for Han-Wen and Jan)
> Before, the font was changed with macros declared separately for 
> pure TeX and LaTeX, but now all the fonts are declared directly in
> the generated code using the low-level TeX commands. This circumvents
> the fontencoding mechanism of LaTeX and is a potential source of
> problems also for us users of the Latin-1 character set. 
> I think it still works since the inputencoding latin1 is 
> declared by ly2dvi, but it still means that e.g. an o with
> umlaut is typeset as a combination of the two symbols 'o' and
> 'umlaut' instead of as a single character, as would be the case
> if fontencoding OT1 had been used. This shouldn't give any 
> difference in layout, though. One of the main argument for using
> fontencoding OT1 is for the hyphenation mechanism to work, an 
> argument that's clearly irrelevant in our application. Yet we
> see the problem for August.
. * Abstraction for engravers:
.  * make "in-between" engraver (Tie, Extender)
.  * make wide_spanner_engraver (line_group_spanne,r staff_symbol)
. * remove Interval dim_ from Dimension_cache and rename the struct.
. * do scaled fonts generally 
. * The Lilypond accidental symbols are available through the TeX macros, 
\textflat, \textsharp and \textnatural defined in the init file
lilyponddefs.tex. All the symbols are defined in the files fetaXX.tex
where XX corresponds to the size. I don't know where (or even if) they
are installed but they are generated in the mf/out/ directory during the
compilation.
. * make dependencies () a virtual function.
. * fix Repeated_music design
. * one big ly2dvi input that includes *all* tests.
. * store req -> element, element -> element causes for warnings
. * include examples in RPM.
. * fix partial measures in meaningful way.
. * add scripts to bars eg.  |^"bla" 
. * relative mode for mi2mu
. * uniformise recent feta contributions.
.  * use SCM and Molecules for {accordeon stuff, }
.  * bigger and fatter 4/4 C
. * sort out directory stuff.
.  * --prefix option?
.  * -I option per filetype?
.  * kpathsea?
. * ly2dvi/lilypond logfile.
. * fractional chord durs.
. * hang Item on Spanner
. * do --safe for PS output?
. * convert-mudela --output
. * Staff_margin (with a partial measure.)
. * fix pletvisibility properties:
.  * bracket on/off/no-bracket-when-beam
.  * num on/off/no-num-when-beam
. * fix: standchen.
. * junk text{sharp,flat,etc}
. * fix convert-mudela manpage
. * decimal point in \paper {}
. * {  \voiceone <a'4.*2/3  d''4.*2/3 fis''4.*2/3>  [g''8 fis'' e''] d''4 |}
         and
         { \voicetwo  d'4 d'2.*2/3 } spacing
. * tied notes for MIDI
. * sharp / flat on trills (in MIDI ??)
. * scm-ify \property values.
. * move class Lookup {} into scm
. * msgfmt -o check?
. * \breathmark TeX macro 
. * add new glyphs to font.ly
. * formatting of input stuff. 
. * \notes{ a \< b \cr } vs \notes{ a \< b \! }
. * space after bars?
. * 'hinterfleisch' before bar (e.g. wtk1-fugue2)?
. * Summary of minor spelling irregularities:
.  *  capitalization/use of underscores in property names
.  * fix SkipBars  -> skipBars
. * broken scripts:
        lbheel = \script { "bheel" 0 0 -1  0 0 }
        rbheel = \script { "bheel" 0 0 1 0 0 }
        lbtoe = \script { "btoe" 0 0 -1 0 0 }
        rbtoe = \script { "btoe" 0 0 1 0 0 }
        lfheel = \script { "fheel" 0 0 -1  0 0 }
        rfheel = \script { "fheel" 0 0 1 0 0 }
        lftoe = \script { "ftoe" 0 0 -1 0 0 }
        rftoe = \script { "ftoe" 0 0 1 0 0 }
        and also
        portato= \script { "portato" 0 -1 0 1 0 }
. * ly2dvi
.  * bottomnote for ly2dvi
.  * deps for ly2dvi
.* STUFF
. * compulsory hshift
. * Give # lines in linebreaking.  
. * arrows on slurs.
. * Align_element::padding  ?
. * uniformise property names...
.  * ydirection <-> yDirection
.  * rather allow '_' in identifiers first (i.e. junk ^ and _),
and do y_direction?
.  * typo checks on property names?
. * use streambufs and iostream
        to provide IO handling for TeX stream, mudela stream, data-file.
. * seriously buffer TeX output (do profile of writing .5 mb TeX file.)
. * strip EXEs before installing
. * zip target for binary windows dist (JBR)
. * junking \skip req in lyrics
. * mi2mu empty staffs.
. * use Fourier transformation for rhythmic quantisation.
. * horizontal centering of dynamics 
. * $DEPENDENCIES_OUTPUT support
. * Xdvi zooming ?! Try to understand GS garbage collection.
gs: Error: /undefined in draw_beam
gs: Operand stack:
. * fix vertical alignment and stafflines
. * declaring Performers
. * *.yo: fix pod manpage layout legacy
. * text-items clash with stems/beams
. * --include, -I option for ly2dvi (pass on to lily)
. * fix placement of beam-less abbrev
. * \meter 3/4;
        <{\voiceone cis4. cis8 cis4 | cis4 cis cis | r1 }
         {\voicetwo cis,4 r r | r2. | r1 }> |
        }
. * Musical_pitch (analogous to Duration and Rhythmic_req)
          think about, analogous to pitch: 
           * { a4 a16 } c <-- duration of c?
           * < a4 b8 > c <-- duration of c?
. * \tempo 4. = 90;
. * fix audio-items (don't use (input) requests, but copy data)
. * make sure all ex's have a mudela-version
. * do rest-collisions for Multi_measure_rests as well.
. * split error/warning in error/warning/non_fatal_error
. * add a Duration_convert member to Duration_iter to set parameters.
        Junk global duration settings. 
. * en-,discouraged linebreaking: 
.  * handle DISALLOW < penalty > FORCE
.  * discourage breaking of slurs
. * fix variable define/lookup parser global<->paper
. * fix title/instrument placements; see input/test/title.ly
. * <\voiceone c4. \voicetwo c4>
. * add full cello suites to web site (kom op, Maarten!)
. * Rethink Time_description
\cadenza , \meter, \grouping should all be \properties
. * rename
.  * measure -> bar
.  * abbrev -> tremolo
.  * abbreviation-beam -> (measured/unmeasured) tremolo
.  * partial -> anacrouse
. * robustify Beam (in case of no stems, 32nd notes)
. * move paper vars into engraver properties
. * check for groff / troff/ nroff et
. * more intelligent file searching
. * disable spaces in TeX stuff
. * handle ^C for tmp/file creation.
. * make LilyPond RPM fully relocatable
. * dots & rest collisions.
. * the warning about the negative slur/tie length appears twice
which is irritating.
. * better hshift (dots, distance, head dependent)
. * clefs (AG): The "8" should appear closer to the actual clef,
touching it.
. * put errorlevel in Input class
. * junk nesting slurs
. * integrate midi-elts from mi2mu and lily?
. * distinguish between unset and empty properties. 

.* FONT
. * the bracket is ugly (wings are too parabolic, should be more circular)
. * versioning for Feta
. * rewrite dynamic signs.  They're kind of spaghetti now. 
. * The `3' in the meter key is a one or two pixels too thin (at 600dpi)
        in the middle:
. * design macro for penstroke with rounded endings
. * timesig C: fat yet less curved.  Check out relations. 
. * design decent macro for bulbed endings
. * printer dependencies.
. * y-refpoint for rests should be the 4th staffline (gclef: b')
. * clean-up eight+ rests
. * versioning  
. * lo-res fixes.
. * work out sizes of character fonts.
. * more symbols
.  * piano pedals: Ped/*
.  * maxima notehead
.  * coda signs: Segno, O+
.  * glissando, arpeggio
.  * lengthened trill
. * include important spacing dims in fetalog 
. * scalability: use blot_diameter everywhere, make font chubbier for
    small sizes.

.* 3RD PARTY BUGS
. * GNU diff 2.7: diff -rN does not see a new directory with empty file
. * check out GCC signatures?
. * glibc 2.0:
          f = fopen ("/dev/null", "r")
          assert (feof (f))

. * tetex: mfplain.mem
.* 3RD PARTY PROJECTS:
. * make GCC warn about ctor that leaves member vars  uninitialised.
. * GNU patch 
.  * 'double-fix' mode: ignore identical fix, rather than suggest 
            to revert patch direction when (more or less? exactly) the same 
            thing is fixed
.* PROJECTS
. * Scripts:
.  * stack scripts in chords:
          % fingering:        footing:
            < a-4 c-2 g-1 >   < c-\ltoe e-\lheel >
.  * script-spacing
.  * slur parts as a script
.  * stack parameterised scripts for fingering and footing:

. * convert-mudela:
.  * automatically add  a \version if not present.
.  * better parsing
.  * make smarter
.  * convert-mudela -e empty.ly

. * emacs mudela-mode
.  *  should handle block comments too.
.  *  handle lexer modes (\header, \melodic, \lyric) etc.
.  *  indentation
.  *  notenames?
.  *  fontlock: \melodic \melodic

. * use properties for:
.  * default tempo.
.  * cadenza mode? 

. * page handling:
.  * PS output
.  * optimal pagebreaking.

. * Unicode support? -> man 7 unicode
.  * 16 bit Strings
.  * -u switch
.  * detect of 16/8 bit files
.  * examples to go with it.

. * Spring_spacer:
.  * write a faster Spring_spacer (without matrices if possible)
.  * relate energybound to linelen unitspace fontsize etc.
.  * used fixed point fp?

. * \header
.  * output header info to MIDI too.

. * a musical dictionary. See Documentation/vocabulary-*, other
languages:
.  * explanations
.  * italian
.  * german
.  * dutch
.  * swedish
         .. ?

. * chords
.  * guitar chords (fret diagrams)
.  * other chord name styles (american(?) style)
.  * basso continuo
.  * MIDI output?

. * Debug logs:
.  * derive lily_stream, Dstream, texstream from ostream?
.  * indentable stream as baseclass for TeX stream, lily stream, Dstream.
.  * use dstream feature in mi2mu
.  * integrate IO checking into stream class

. * input converters
.  * NIFF?
.  * ABC? 
.  * SMDL?

. * add to MIDI output:
.  * tempo change
.  * ornaments (and trills?)
.  * repeat/volta
.  * slurs
.  * accents
.  * dynamics
> Would it be hard to add support for proper dynamics in MIDI output?  I
> would really like to have this feature...

To answer this, I have to give a little background.  Please say so if
I am being obtuse.

Both audio and notation elements are created decentrally, when
LilyPond says "Interpreting music ... [8][16] etc."  After all
elements are created, they are processed again to calculate all
parameters.  After that is done, they are dumped onto the output (the
MIDI stream or the TeX file).

The audio and notation elements are created by objects called
performer and engraver respectively.  They work by taking a chunk of
input (a so-called request), and using the request to create an audio
or notation element.

After some processing, the engraver sends the created elements to
its enclosing notation/performance context.

In the case of notation, engravers can also modify existing elements
created by sibling engravers.  Eg. the beam-engraver will modify stems
when it creates a beam over a couple of stems.  This is done in
Engraver::do_acknowledge_element () and
Engraver::do_process_acknowledged ().

In the case of audio, there are no mechanisms to modify existing audio
elements. After the audio elements are created, they are sent directly
to the enclosing context (an instance of Performer_group_performer),
through the method Performer::play ()

Concretely spoken, for dynamics you have to write a performer
(Dynamics_performer), that will take the input requests. If I
understand correctly, every MIDI note played must have its dynamic
strength set separately.  That means the the Dynamics_performer must
set the strength of every Audio_note it finds.  This means that one
has to modify existing Audio items.  The best way of doing this is
with a broadcast/acknowledge process.

So the best way of handling this, is

  1 supporting dynamic settings in Audio_note

  2 mimicking the broadcast/acknowledge mechanism of the Engravers in
  the Performers

  3 using that mechanism to write a Dynamics_performer that will modify
  any notes it finds to set appropriate strengths.

You could also kludge this by deriving from Performer_group_performer
a performer that also sets dynamics within notes, but that is not the
elegant way to do it.

I'd say that the work involved is not hard, but you have to be fluent
with C++ and need some insight into the working of Notation Contexts
and friends.  I guess I could do it in a day or so, but if you are not
so fluent with the inner workings of LilyPond, it could take you some
more time (A few more days?).

Please note, that I am *not* going to code this myself: my days are
valuable, and I think I have more interesting things to do than fixing
the MIDI output (I don't even have a soundcard).  But I *do* want to help
you or other people with implementing this, so if you need help, don't
hesitate to ask.

.  * account for rhythmic position in measure
.  * etc.

. * grace notes
.  * make separate class for Grace_*_engravers 
        (which is at Voice_gravs level)
.  * make encapsulated spacing problems.

. * accidentals per Voice_group

. * Output class, handles : (smallish)
.  * help text /(c) notice?
.  * version line
.  * warning /errors/progress
.  * abort on error/warning; exit status
.  * quiet/ignore-version options
.  * logfile output

. * PS 
.  * speed up PS code.
.  * PS lines to .eps files
.  * write custom prolog

. * SGML input

. * Documentation
.  * remove links to non-free SW.
.  * internal documentation
.  * a general paper. 
.  * \properties
.   * left/rightbeamcount
.   * various other new properties.
.  * introduction?
.  * info?
.  * LaTeX?
.  * more manpages?
.  * Doc (C) stuff of sheet music
.  * a better format?  SGML?  TeX?  Info?
.  * a better tutorial

. * more spanners (smallish)
.  * Glissando
.  * trill

. * Rewrite Stem, Beam and Rhythmic_grouping (hairy)
.  * [c8. c32 c32]
.  * doc on Rhythmic_grouping
.  * general shaving on Beam and Stem
.  * use plet grouping
.  * separate visual from rhythmic info even more
.  * beams over bars

. * lines:
.  * Ledger lines, should be separate item: Ledger_lines
.  * set different line sizes / thicknesses
.  * beam stem showthrough, inter beam sizes (french vs german style)

. * Collisions
.  * left/right note balls should be handled by Collision:
        < \multi 2; { \stem 1; <b c> } { <f as b> } >

. * Keys:
.  * merge key{item} & localkey{item}?
.  * spacing, national styles?

. * Support for TAB
.  * redo Notehead to allow digits as heads
.  * add a \tablature {} lexer mode for easy entering

. * should adjust stemlength for flag number.

. * breaking:
.  * caching breakpoints
.  * saving them
.  * specify number of lines

.* INPUTLANGUAGE
. * \rhythms 4 16 16 16 16; c c c c c -> c4 c16 etc. 
. * Language:
.  * \translator -> ?
.  * fix \partial 
.  * \bla {} vs \bla ; 
.  * mix engraver hacking with music ?
.  * \once\property  KEY = VAL
.  * \addtranslator, \removetranslator
.  * junk ^ and _ for scripts
.  * junk _ for lyrics.
.  * abstract grammar.
. * Figure out semicolons.
. * c4 4 4 4 for c4 c4 c4 c4?
. * <c f a>4 ?
. * Viola mode?

                @c c g e g 
        for
                c g es g, 

                @A c g e g
        for
                cis gis e gis
. * configure pitch_byte
. * rest name configurable


.* SMALLISH PROJECTS

. * Faulenzer sign (% repeat)

. * fix confusing naming in Engraver vs Engraver_group_engraver (context)

. * progress when creating MIDI elts.

. * A range for context errors (eg. mark both { and }. )

. * lyric in staff  (sharpsharp in staff, text below)

. * half sharps/flats

. * write Dynamic_line (to group dynamics horizontally)

. * use Real for all y positions.

. * half-sharps, half-flats

. * adaptive accidental spacing.

. * handle EOF graciously in error messages.

. * midi esp.: use I32 iso int where 32 bits are needed (or assumed...)

. * stafftypes: voice names/ instrument names.

. * lily \tempo

. *   % toe to bheel   four to five
            %   ^_u            4_5  
            %     -                           
            %  --|x--         --|x--
            %  --|---         --|---
            %  --|---         --|---
          (where "to" is a tiny bow)

. * auxilliary file for caching info.

. * Text_crescendo

. * revise calcideal

.* IDEAS
. * Output data structures of Mudela in XML/SGML.
. * create libmudela, or liblily_frontend
. * move MIDI stuff (including Quantization) to a ANSI C libmidi library.
. * Spacing_request for manually adjusting spacing
. * caching breakpoints
. * used fixedpoint arithmetic for min. energy.
. * move towards incremental algorithms.
. * versioning stuff (cvt mudela, mudela, etc.)
. * dynamic loading of engravers? 
.* SMOBS
Han-Wen Nienhuys <hanwen@cs.uu.nl> writes:

>       class Foo_class {
>               Foo_smob *smob_ptr_;
>       }
> 
>       struct Foo_smob {
>               Foo_class *class_ptr_
>       }

Usually you can get away without having this interlinked structure.
One piece of information you need when exporting objects to scheme is
the smob tag for your class.  You might store this value (a long) into
a class static variable, or a file static variable.  I'll use a class
static variable in this example.

I typically use code that works like this (untested code ahead):

class Foo_class {
    static long *Foo_smob_tag;
    SCM obj; // representation as a scheme object, description comes later
}

// call this once on startup
void init_Foo_class {
    static scm_smobfuns type_rec;

    type_rec.mark = foo_mark;
    type_rec.free = foo_free;
    type_rec.print = foo_display;
    type_rec.equalp = 0;
    Foo_class::Foo_smob_tag = new int(scm_new_smob(type_rec));
}

When you want to export an object to scheme, you put a pointer to the
object itself in the cdr of the cell, and the tag in the car of the
cell.  The code looks like this:

// Is this a Foo?
static int
foo_p(SCM obj)
{
    return(SCM_NIMP(obj) && SCM_CAR(obj) == Foo_class::Foo_smob_tag);
}

// given a Scheme representation, return, a C++ representation
static Foo_class *
foo_unbox(SCM obj)
{
    return((Foo_class*)SCM_CDR(obj));
}

// the SCM representation of the object is stored inside itself
// this will become useful when destructor gets called
static SCM
foo_box(Foo_class *foo)
{
    SCM_DEFER_INTS;
    SCM_NEWCELL(foo->obj);
    SCM_SETCAR(foo->obj, Foo_class::Foo_smob_tag);
    SCM_SETCDR(foo->obj, (SCM)foo);
    SCM_ALLOW_INTS;
    return foo->obj;
}

> C++ determines life time, so what is the strategy?

What happens now when the destructor gets called?  Lets set the cdr to
NULL, and then check for that in all of the primitive functions that
use your smob.  We'll call this notion 'live'; a scheme object is
'live' if the C++ representation still exists, it's dead if the C++
object is gone.  You can still have references to dead objects in
scheme code; it's just an error to use them.  This idea is stolen from
Emacs's handling of buffers ('buffer-live-p' and friends).

Add another function, foo_live_p:

static int
foo_live_p(SCM obj)
{
    return(foo_p(obj) && SCM_CDR(obj) != NULL);
}

In you destructor, you need to do:

~Foo_class()
{
    SCM_CDR(this->obj, NULL);
}

When writing new primitives, now just check foo_live_p().

I hope the above helps,


* OUTLINE SETUP
.* .EMACS
;;; from outline.el
(defvar outline-font-lock-keywords
  '(;; Highlight headings according to the level.
    ;; must change this to allout node layout
    ;;("^\\(\\*+\\)[ \t]*\\(.+\\)?[ \t]*$"
    ("^\\([.]*[ ]*[*]\\)[ \t]*\\(.*\\)$"
     (1 font-lock-string-face)
     (2 (let ((len (- (match-end 1) (match-beginning 1))))
          (or (cdr (assq len '((1 . font-lock-function-name-face)
                               (2 . font-lock-keyword-face)
                               (3 . font-lock-comment-face))))
              font-lock-variable-name-face))
        nil t))
    ;; Highlight citations of the form [1] and [Mar94].
    ("\\[\\([A-Z][A-Za-z]+\\)*[0-9]+\\]" . font-lock-type-face))
  "Additional expressions to highlight in Outline mode.")

(add-hook 'outline-mode-hook
          '(lambda ()
            (setq font-lock-maximum-decoration t)
            (setq font-lock-maximum-decoration t)
            (make-local-variable 'font-lock-defaults)
            (setq font-lock-defaults '(outline-font-lock-keywords t))
            (font-lock-mode global-font-lock-mode)
        ))
 
(require 'allout)
(outline-init 't)