1 @c -*- coding: utf-8; mode: texinfo; -*-
4 @node Build system notes
5 @chapter Build system notes
7 @warning{This chapter is in high flux, and is being run in a
8 @qq{wiki-like} fashion. Do not trust anything you read in this
12 * Build system overview::
13 * Tips for working on the build system::
19 @node Build system overview
20 @section Build system overview
22 Build system is currently GNU make, with an extra "stepmake" layer
23 on top. Look at files in @file{make/} and @file{stepmake/} and
24 all @file{GNUmakefile}s.
26 There is wide-spread dissatisfaction with this system, and we are
27 considering changing. This would be a huge undertaking (estimated
28 200+ hours). This change will probably involve not using GNU make
29 any more -- but a discussion about the precise build system will
30 have to wait. Before we reach that point, we need to figure out
31 (at least approximately) what the current build system does.
33 Fundamentally, a build system does two things:
37 Constructs command-line commands, for example:
41 --tons --of --options \
44 --more --imperial --and --metric --tons --of --options \
49 If there was a previous build, it decides which parts of the
50 system need to be rebuilt.
54 When I try to do anything in the build system, it helps to remind
55 myself of this. The "end result" is just a series of command-line
56 commands. All the black magick is just an attempt to construct
59 @node Tips for working on the build system
60 @section Tips for working on the build system
72 to the build system files in various places. This will let you
73 track where the program is, in various points of the build.
76 First task: understand how @code{make website} works,
77 @emph{without} the translations. Looking at the english-only
78 website is the best introduction to the build system... it only
79 covers about 5% of the whole thing, but even that will likely take
90 * Building a bibliography::
93 @node Building a bibliography
94 @subsection Building a bibliography
96 Bibliography files contain a list of citations, like this:
100 author = @{Vinci, Albert C.@},
101 title = @{Fundamentals of Traditional Music Notation@},
102 publisher = @{Kent State University Press@},
107 There are a variety of types of citation (e.g. Book (as above),
108 article, publication). Each cited publication has a list of
109 entries that can be used to identify the publication.
110 Bibliograpies are normally stored as files with a .bib
111 extension. One part of the doc-build process is transforming the
112 bibliography information into @code{texinfo} files. The commands
113 to do this are in the @file{GNUmakefile} in the
114 @file{Documentation} directory.
116 A typical line of the makefile to translate a single bibliography
120 $(outdir)/colorado.itexi:
121 BSTINPUTS=$(src-dir)/essay $(buildscript-dir)/bib2texi \
122 -s $(top-src-dir)/Documentation/lily-bib \
123 -o $(outdir)/colorado.itexi \
124 $(src-dir)/essay/colorado.bib
130 $(outdir)/colorado.itexi:
133 We're making the file @file{colorado.itexi} and so this is the
137 BSTINPUTS=$(src-dir)/essay $(buildscript-dir)/bib2texi \
140 It's in the @file{essay} directory and we want to run the
141 bib2texi.py script against it.
144 -s $(top-src-dir)/Documentation/lily-bib \
147 The style template is @file{lily-bib.bst} and is found in the
148 @file{Documentation} directory.
151 -o $(outdir)/colorado.itexi \
154 The output file in @file{colorado.itexi}.
157 $(src-dir)/essay/colorado.bib
160 The input file is @file{colorado.bib} in the @file{essay}
163 The @code{bib2texi} Python script used to be used with a variety
164 of options, but now is always called using the same options, as
165 above. Its job is to create the file containing the options for
166 @code{bibtex} (the program that actually does the translation),
167 run bibtex, and then clean up some temporary files. Its main
168 "value add" is the creation of the options file, using this code:
171 open (tmpfile + '.aux', 'w').write (r'''
174 \bibstyle@{%(style)s@}
175 \bibdata@{%(files)s@}''' % vars ())
178 The key items are the style file (now always lily-bib for us) and
181 The style file is written in its own specialised language,
182 described to some extent at
185 @uref{http://amath.colorado.edu/documentation/LaTeX/reference/faq/bibtex.pdf}
188 The file @file{lily-bib.bst} also has fairly extensive commenting.
191 @section Website build
193 Start here: @file{make/website.make}
195 Typing make website runs the file @file{GNUmakefile} from the
196 build directory. This only contains 3 lines:
200 include config$(if $(conf),-$(conf),).make
201 include $(configure-srcdir)/GNUmakefile.in
204 The variable @code{depth} is used throughout the make system to
205 track how far down the directory structure the make is. The first
206 include sets lots of variables but doesn't "do" anything. The
207 second runs the file @file{GNUmakefile.in} from the top level
210 This sets another load of variables, and then includes (i.e.
211 immediately runs) @file{stepmake.make} from the @file{make}
212 subdirectory. This sets a load of other variables, does some
213 testing to see if SCONS (another build tool?) is being used, and
214 then runs @file{make/config.make} - which doesn't seem to exist...
216 Next, it runs @file{make/toplevel-version.make}, which sets the
217 version variables for major, minor, patch, stable, development and
218 mypatchlevel (which seems to be used for patch numbers for
219 non-stable versions only?).
221 Next - @file{make/local.make}, which doesn't exist.
223 Then a few more variable and the interesting comment:
226 # Don't try to outsmart us, you puny computer!
227 # Well, UGH. This only removes builtin rules from
230 and then tests to see whether BUILTINS_REMOVED is defined. It
231 appears to be when I run make, and so
232 @file{stepmake/stepmake/no-builtin-rules.make} is run. The
233 comment at the head of this file says:
236 # UGH. GNU make comes with implicit rules.
237 # We don't want any of them, and can't force users to run
241 I've not studied that file at length, but assume it removes all
242 make's build-in rules (e.g. @file{*.c} files are run through the
243 GNU C compiler) - there's a lot of them in here, and a lot of
244 comments, and I'd guess most of it isn't needed.
246 We return to @file{stepmake.make}, where we hit the make rule all:
247 The first line of this is:
250 -include $(addprefix $(depth)/make/,$(addsuffix -inclusions.make, $(LOCALSTEPMAKE_TEMPLATES)))
253 which, when the variables are substituted, gives:
256 ./make/generic-inclusions.make
257 ./make/lilypond-inclusions.make.
260 (Note - according to the make documentation, -include is only
261 different from include in that it doesn't produce any kind of
262 error message when the included file doesn't exist).
264 And the first file doesn't exist. Nor the second. Next:
267 -include $(addprefix $(stepdir)/,$(addsuffix -inclusions.make, $(STEPMAKE_TEMPLATES)))
270 which expands to the following files:
273 /home/phil/lilypond-git/stepmake/stepmake/generic-inclusions.make
274 /home/phil/lilypond-git/stepmake/stepmake/toplevel-inclusions.make
275 /home/phil/lilypond-git/stepmake/stepmake/po-inclusions.make
276 /home/phil/lilypond-git/stepmake/stepmake/install-inclusions.make.
279 One little feature to notice here - these are all absolute file
280 locations - the line prior to this used relative locations. And
281 none of these files exist, either. (Further note - I'm assuming
282 all these lines of make I'm following are autogenerated, but
283 that'll be something else to discover.)
288 include $(addprefix $(stepdir)/,$(addsuffix -vars.make, $(STEPMAKE_TEMPLATES)))
294 /home/phil/lilypond-git/stepmake/stepmake/generic-vars.make
295 /home/phil/lilypond-git/stepmake/stepmake/toplevel-vars.make
296 /home/phil/lilypond-git/stepmake/stepmake/po-vars.make
297 /home/phil/lilypond-git/stepmake/stepmake/install-vars.make.
300 Woo. They all exist (they should - no - in front of the include).
302 Next step will be to work out what these do.
304 That's my 30 minutes for Monday.
306 Website build includes @ref{Building a bibliography}.