Issue 2607: Allow immediate Scheme expressions with splicing and multiple values

$xxx has been able to interpret 0-valued expressions (namely
SCM_UNSPECIFIED) so far by, well, not interpolating anything.

This change allows it to interpret multi-valued expressions by
returning multiple tokens, allowing things like

{ < $(apply values (map (lambda (i) (ly:make-pitch 0 i 0)) (iota 8))) > }

to work as intended, by creating a chord enclosing all 8 pitches.

In addition, in analogy to the unquote-splicing operator ,@ the form

{ < $@(map (lambda (i) (ly:make-pitch 0 i 0)) (iota 8)) > }

is provided for interpolating a list of Scheme expressions into the
token stream.

diff --git a/Documentation/changes.tely b/Documentation/changes.tely
index fb9fdb6c14d1ae38ed4ec1ca8a95eced80b4df1a..e1c05e52c8befbc4c9cf4c9068c71af828995596 100644 (file)
--- a/Documentation/changes.tely
+++ b/Documentation/changes.tely
@@ -211,12 +211,13 @@ the tweaked event, like accidentals, stems, and flags:
 Scheme expressions inside of embedded Lilypond (@code{#@{@dots{}#@}})
 are now executed in lexical closure of the surrounding Scheme code.
 @code{$} is no longer special in embedded Lilypond.  It can be used
-unconditionally in Lilypond code for immediate evaluation, similar to
-how @code{ly:export} could previously be used.  @code{ly:export} has
-been removed.  As a consequence, @code{#} is now free to delay
-evaluation of its argument until the parser actually reduces the
-containing expression, greatly reducing the potential for premature
-evaluation.
+unconditionally in Lilypond code for immediate evaluation of Scheme
+expressions, similar to how @code{ly:export} could previously be used.
+@code{ly:export} has been removed.  As a consequence, @code{#} is now
+free to delay evaluation of its argument until the parser actually
+reduces the containing expression, greatly reducing the potential for
+premature evaluation.  There are also @q{splicing} operators @code{$@@}
+and @code{#@@} for interpreting the members of a list individually.
 
 @item
 Support for jazz-like chords has been improved: Lydian and altered

diff --git a/Documentation/extending/scheme-tutorial.itely b/Documentation/extending/scheme-tutorial.itely
index 6f8622e8775bef2868af5d4f605c29bd2816c9e0..9b055dcba6475654ce457bc1fcaaf3860d04de01 100644 (file)
--- a/Documentation/extending/scheme-tutorial.itely
+++ b/Documentation/extending/scheme-tutorial.itely
@@ -611,6 +611,7 @@ guile> (cond ((< a b) "a is less than b")
 * LilyPond Scheme syntax::
 * LilyPond variables::
 * Input variables and Scheme::
+* Importing Scheme in LilyPond::
 * Object properties::
 * LilyPond compound variables::
 * Internal music representation::
@@ -661,7 +662,7 @@ at all is passed to the parser.
 
 This is, in fact, exactly the same mechanism that Lilypond employs when
 you call any variable or music function by name, as @code{\name}, with
-the only difference that its end is determined by the Lilypond lexer
+the only difference that the name is determined by the Lilypond lexer
 without consulting the Scheme reader, and thus only variable names
 consistent with the current Lilypond mode are accepted.
 
@@ -669,6 +670,11 @@ The immediate action of @code{$} can lead to surprises, @ref{Input
 variables and Scheme}.  Using @code{#} where the parser supports it is
 usually preferable.
 
+@funindex $@@
+@funindex #@@
+There are also @q{list splicing} operators @code{$@@} and @code{#@@}
+that insert all elements of a list in the surrounding context.
+
 Now let's take a look at some actual Scheme code.  Scheme procedures can
 be defined in LilyPond input files:
 
@@ -800,7 +806,7 @@ traLaLa = { c'4 d'4 }
 #(define twice
   (make-sequential-music newLa))
 
-{ \twice }
+\twice
 @end lilypond
 
 @c Due to parser lookahead
@@ -812,6 +818,11 @@ reads @code{#} and the following Scheme expression @emph{without}
 evaluating it, so it can go ahead with the assignment, and
 @emph{afterwards} execute the Scheme code without problem.
 
+@node Importing Scheme in LilyPond
+@subsection Importing Scheme in LilyPond
+@funindex $
+@funindex #
+
 The above example shows how to @q{export} music expressions from the
 input to the Scheme interpreter.  The opposite is also possible.  By
 placing it after @code{$}, a Scheme
@@ -821,7 +832,7 @@ been written as
 
 @example
 ...
-@{ $(make-sequential-music (list newLa)) @}
+$(make-sequential-music newLa)
 @end example
 
 You can use @code{$} with a Scheme expression anywhere you could use
@@ -835,9 +846,29 @@ following Scheme definition would have failed because @code{traLaLa}
 would not yet have been defined.  For an explanation of this timing
 problem, @ref{LilyPond Scheme syntax}.
 
-In any case, evaluation of Scheme code happens in the parser at latest.
-If you need it to be executed at a later point of time, @ref{Void scheme
-functions}, or store it in a macro:
+@funindex $@@
+@funindex #@@
+A further convenience can be the @q{list splicing} operators @code{$@@}
+and @code{#@@} for inserting the elements of a list in the surrounding
+context.  Using those, the last part of the example could have been
+written as
+
+@example
+...
+@{ $@@newLa @}
+@end example
+
+Here, every element of the list stored in @code{newLa} is taken in
+sequence and inserted into the list, as if we had written
+
+@example
+@{ $(first newLa) $(second newLa) @}
+@end example
+
+Now in all of these forms, the Scheme code is evaluated while the
+input is still being consumed, either in the lexer or in the parser.
+If you need it to be executed at a later point of time, check out
+@ref{Void scheme functions}, or store it in a procedure:
 
 @example
 #(define (nopc)

diff --git a/lily/include/lily-lexer.hh b/lily/include/lily-lexer.hh
index 637580056589ed73c00c596733923ac0932b7e1b..9a48e3b8e90196b931f7776580a03df37db69c7f 100644 (file)
--- a/lily/include/lily-lexer.hh
+++ b/lily/include/lily-lexer.hh
@@ -50,8 +50,9 @@ private:
   SCM scopes_;
   SCM start_module_;
   int hidden_state_;
+  SCM eval_scm (SCM, char extra_token = 0);
 public:
-  SCM eval_scm (SCM);
+  SCM eval_scm_token (SCM sval)  { return eval_scm (sval, '#'); }
   SCM extra_tokens_;
   YYSTYPE *lexval_;
   Input *lexloc_;

diff --git a/lily/lexer.ll b/lily/lexer.ll
index a56c182eab1d1d4067d590f898593e63c23e9ddf..afb881f2b8f370e3d2afbc87f744517be2c44281 100644 (file)
--- a/lily/lexer.ll
+++ b/lily/lexer.ll
@@ -276,7 +276,7 @@ BOM_UTF8    \357\273\277
 
 
 }
-<sourcefilename>\"[^"]*\"     {
+<sourcefilename>\"[^""]*\"     {
        string s (YYText_utf8 () + 1);
        s = s.substr (0, s.rfind ('\"'));
 
@@ -438,11 +438,11 @@ BOM_UTF8  \357\273\277
        }
        char_count_stack_.back () += n;
 
-       sval = eval_scm (sval);
-               
+       sval = eval_scm (sval, '$');
+
        int token = scan_scm_id (sval);
        if (!scm_is_eq (yylval.scm, SCM_UNSPECIFIED))
-         return token;
+               return token;
 }
 
 <INITIAL,notes,lyrics>{ 
@@ -1005,8 +1005,15 @@ Lily_lexer::is_figure_state () const
        return get_state () == figures;
 }
 
+// The extra_token parameter specifies how to convert multiple values
+// into additional tokens.  For '#', additional values get pushed as
+// SCM_IDENTIFIER.  For '$', they get checked for their type and get
+// pushed as a corresponding *_IDENTIFIER token.  Since the latter
+// tampers with yylval, it can only be done from the lexer itself, so
+// this function is private.
+
 SCM
-Lily_lexer::eval_scm (SCM readerdata)
+Lily_lexer::eval_scm (SCM readerdata, char extra_token)
 {
        SCM sval = SCM_UNDEFINED;
 
@@ -1023,6 +1030,33 @@ Lily_lexer::eval_scm (SCM readerdata)
                error_level_ = 1;
                return SCM_UNSPECIFIED;
        }
+
+       if (extra_token && SCM_VALUESP (sval))
+       {
+               sval = scm_struct_ref (sval, SCM_INUM0);
+
+               if (scm_is_pair (sval)) {
+                       for (SCM v = scm_reverse (scm_cdr (sval));
+                            scm_is_pair (v);
+                            v = scm_cdr (v))
+                       {
+                               int token;
+                               switch (extra_token) {
+                               case '$':
+                                       token = scan_scm_id (scm_car (v));
+                                       if (!scm_is_eq (yylval.scm, SCM_UNSPECIFIED))
+                                               push_extra_token (token, yylval.scm);
+                                       break;
+                               case '#':
+                                       push_extra_token (SCM_IDENTIFIER, scm_car (v));
+                                       break;
+                               }
+                       }
+                       sval = scm_car (sval);
+               } else
+                       sval = SCM_UNSPECIFIED;
+       }
+
        return sval;
 }
 

diff --git a/lily/parse-scm.cc b/lily/parse-scm.cc
index fbd60a941a4bdff3b1536fd5ce11c97ca4c6aff4..3dc07fa3d5c68e5295ebe3c6b1353c383088fa58 100644 (file)
--- a/lily/parse-scm.cc
+++ b/lily/parse-scm.cc
@@ -47,9 +47,14 @@ internal_ly_parse_scm (Parse_start *ps)
   scm_set_port_line_x (port, scm_from_int (ps->start_location_.line_number () - 1));
   scm_set_port_column_x (port, scm_from_int (ps->start_location_.column_number () - 1));
 
-  SCM form = scm_read (port);
+  bool multiple = ly_is_equal (scm_peek_char (port), SCM_MAKE_CHAR ('@'));
+
+  if (multiple)
+    (void) scm_read_char (port);
 
+  SCM form = scm_read (port);
   SCM to = scm_ftell (port);
+
   ps->nchars = scm_to_int (to) - scm_to_int (from);
 
   if (!SCM_EOF_OBJECT_P (form))
@@ -62,6 +67,10 @@ internal_ly_parse_scm (Parse_start *ps)
             // Replace form with a call to previously compiled closure
             form = scm_list_1 (c);
         }
+      if (multiple)
+       form = scm_list_3 (ly_symbol2scm ("apply"),
+                          ly_symbol2scm ("values"),
+                          form);
       return scm_cons (form, make_input (ps->start_location_));
     }
 
@@ -171,4 +180,3 @@ ly_eval_scm (SCM form, Input i, bool safe, Lily_parser *parser)
   scm_remember_upto_here_1 (form);
   return ans;
 }
-

diff --git a/lily/parser.yy b/lily/parser.yy
index 08e3ed7adf25dd424bd4ae5edb1b6e09ccf6930b..a6b3b99d585ae4c395ae5f0e7b04e63cc987018b 100644 (file)
--- a/lily/parser.yy
+++ b/lily/parser.yy
@@ -641,7 +641,7 @@ toplevel_expression:
 embedded_scm_bare:
        SCM_TOKEN
        {
-               $$ = parser->lexer_->eval_scm ($1);
+               $$ = parser->lexer_->eval_scm_token ($1);
        }
        | SCM_IDENTIFIER
        ;

diff --git a/scm/parser-ly-from-scheme.scm b/scm/parser-ly-from-scheme.scm
index 19e6559448e23c95d5648b79c8127ddd87ba9d89..ad95a16d24d2f74ebc70e793b6d485f09a67d2e4 100644 (file)
--- a/scm/parser-ly-from-scheme.scm
+++ b/scm/parser-ly-from-scheme.scm
@@ -49,6 +49,8 @@ from @var{port} and return the corresponding Scheme music expression.
                                                           (port-line port))
                                           (set-port-column! copycat
                                                             (port-column port))
+                                          (if (char=? (peek-char port) #\@)
+                                              (read-char copycat))
                                           (read copycat))))
                                   ;; kill unused lookahead, it has been
                                   ;; written out already