X-Git-Url: https://git.donarmstrong.com/?a=blobdiff_plain;f=Documentation%2Fcontributor%2Fbuild-notes.itexi;h=a5f8c2ccc0b00acc39df6983675bc03c57db2f54;hb=2ffb2f01c37e98372767af3cdbb3d40ec0830408;hp=b84bdbef6067dbae72fd48c29e0ba2ab54a88c99;hpb=f875ef39c544bd3499dae5360e9e24f69933575f;p=lilypond.git diff --git a/Documentation/contributor/build-notes.itexi b/Documentation/contributor/build-notes.itexi index b84bdbef60..a5f8c2ccc0 100644 --- a/Documentation/contributor/build-notes.itexi +++ b/Documentation/contributor/build-notes.itexi @@ -14,7 +14,6 @@ chapter.} * General build system notes:: * Doc build:: * Website build:: -* Building an Ubuntu distro:: @end menu @@ -117,9 +116,11 @@ include $(configure-srcdir)/GNUmakefile.in The variable @code{depth} is used throughout the make system to track how far down the directory structure the make is. The first -include sets lots of variables but doesn't "do" anything. The -second runs the file @file{GNUmakefile.in} from the top level -source directory. +include sets lots of variables but doesn't "do" anything. Default +values for these variables are automatically detected at the +./configure step, which creates the file @file{config.make}. +The second include runs the file @file{GNUmakefile.in} from +the top level source directory. This sets another load of variables, and then includes (i.e. immediately runs) @file{stepmake.make} from the @file{make} @@ -196,9 +197,15 @@ which expands to the following files: One little feature to notice here - these are all absolute file locations - the line prior to this used relative locations. And -none of these files exist, either. (Further note - I'm assuming -all these lines of make I'm following are autogenerated, but -that'll be something else to discover.) +none of these files exist, either. + +(Further note - I'm assuming all these lines of make I'm following are +autogenerated, but that'll be something else to discover.) + +JM: @emph{``No, these lines are not useful in LilyPond (this is why +you think they are autogenerated), but they are part of StepMake, +which was meant to be a package to be installed as a build system over +autoconf/make in software project source trees.''} Next in @file{stepmake.make}: @@ -286,11 +293,11 @@ are: @example $(outdir)/%.ly: %.lym4 - $(M4) $< | sed "s/\`/,/g" > $@ + $(M4) $< | sed "s/\`/,/g" > $@@ $(outdir)/%: %.in - rm -f $@ - cat $< | sed $(sed-atfiles) | sed $(sed-atvariables) > $@ + rm -f $@@ + cat $< | sed $(sed-atfiles) | sed $(sed-atvariables) > $@@ @end example I believe the first rule is for *.ly files, and has a prerequisite @@ -311,7 +318,7 @@ can comment on exactly what the rules mean/do. @example ## TODO: fail dist or web if no \version present. check-version: - grep -L version $(LY_FILES) + grep -L version $(LY_FILES) @end example @file{stepmake/generic-targets.make} contains lots of rules - too @@ -341,9 +348,9 @@ ifeq ($(out),www) # All web targets, except info image symlinks and info docs are # installed in non-recursing target from TOP-SRC-DIR install-WWW: - -$(INSTALL) -m 755 -d $(DESTDIR)$(webdir) - rsync -rl --exclude='*.signature' $(outdir)/offline-root $(DESTDIR)$(webdir) - $(MAKE) -C Documentation omf-local-install + -$(INSTALL) -m 755 -d $(DESTDIR)$(webdir) + rsync -rl --exclude='*.signature' $(outdir)/offline-root $(DESTDIR)$(webdir) + $(MAKE) -C Documentation omf-local-install @end smallexample I don't currently understand the @code{ifeq}, since @code{$(out)} @@ -442,10 +449,10 @@ From git grep: stepmake/stepmake/generic-vars.make has this: @smallexample -LOOP=+$(foreach i, $(SUBDIRS), $(MAKE) PACKAGE=$(PACKAGE) package=$(package) -C $(i) $@ &&) true +LOOP=+$(foreach i, $(SUBDIRS), $(MAKE) PACKAGE=$(PACKAGE) package=$(package) -C $(i) $@@ &&) true @end smallexample -$@ is the name of the target - WWW-1 in this case. +$@@ is the name of the target - WWW-1 in this case. In GNUmakefile.in we find: @@ -554,13 +561,16 @@ they're up to date. (From the make manual: -To this end, after reading in all makefiles, make will consider each as a goal target and -attempt to update it. If a makefile has a rule which says how to update it (found either -in that very makefile or in another one) or if an implicit rule applies to it (see Chapter 10 -[Using Implicit Rules], page 103), it will be updated if necessary. After all makefiles have -been checked, if any have actually been changed, make starts with a clean slate and reads -all the makefiles over again. (It will also attempt to update each of them over again, but -normally this will not change them again, since they are already up to date.) +To this end, after reading in all makefiles, make will consider each +as a goal target and attempt to update it. If a makefile has a rule +which says how to update it (found either in that very makefile or in +another one) or if an implicit rule applies to it (see Chapter 10 +[Using Implicit Rules], page 103), it will be updated if +necessary. After all makefiles have been checked, if any have actually +been changed, make starts with a clean slate and reads all the +makefiles over again. (It will also attempt to update each of them +over again, but normally this will not change them again, since they +are already up to date.) So my assumption seems correct) @@ -676,7 +686,7 @@ Must remake target `out-www/collated-files.tely' @smallexample $(outdir)/collated-files.tely: $(COLLATED_FILES) - $(LYS_TO_TELY) --name=$(outdir)/collated-files.tely --title="$(TITLE)" --author="$(AUTHOR)" $^ + $(LYS_TO_TELY) --name=$(outdir)/collated-files.tely --title="$(TITLE)" --author="$(AUTHOR)" $^ @end smallexample @file{lysdoc-vars.make} has: @@ -710,7 +720,7 @@ depends on @file{git/VERSION}: @smallexample $(outdir)/version.%: $(top-src-dir)/VERSION - $(PYTHON) $(top-src-dir)/scripts/build/create-version-itexi.py > $@ + $(PYTHON) $(top-src-dir)/scripts/build/create-version-itexi.py > $@ @end smallexample This causes create-version-itexi.py to run and create @@ -888,10 +898,10 @@ is: @example $(outdir)/colorado.itexi: - BSTINPUTS=$(src-dir)/essay $(buildscript-dir)/bib2texi \ - -s $(top-src-dir)/Documentation/lily-bib \ - -o $(outdir)/colorado.itexi \ - $(src-dir)/essay/colorado.bib + BSTINPUTS=$(src-dir)/essay $(buildscript-dir)/bib2texi \ + -s $(top-src-dir)/Documentation/lily-bib \ + -o $(outdir)/colorado.itexi \ + $(src-dir)/essay/colorado.bib @end example Line by line: @@ -904,27 +914,27 @@ We're making the file @file{colorado.itexi} and so this is the make instruction. @example - BSTINPUTS=$(src-dir)/essay $(buildscript-dir)/bib2texi \ + BSTINPUTS=$(src-dir)/essay $(buildscript-dir)/bib2texi \ @end example It's in the @file{essay} directory and we want to run the bib2texi.py script against it. @example - -s $(top-src-dir)/Documentation/lily-bib \ + -s $(top-src-dir)/Documentation/lily-bib \ @end example The style template is @file{lily-bib.bst} and is found in the @file{Documentation} directory. @example - -o $(outdir)/colorado.itexi \ + -o $(outdir)/colorado.itexi \ @end example The output file in @file{colorado.itexi}. @example - $(src-dir)/essay/colorado.bib + $(src-dir)/essay/colorado.bib @end example The input file is @file{colorado.bib} in the @file{essay} @@ -957,80 +967,343 @@ described to some extent at The file @file{lily-bib.bst} also has fairly extensive commenting. + @node Website build @section Website build -Start here: @file{make/website.make} +@warning{This information applies only to the standard @code{make +website} from the normal build directory. The process is +different for @code{dev/website-build}.} -The overall build system begins with @ref{How stepmake works}. -Summary: when you type @code{make website} this ends up running -@file{GNUmakefile.in} in the @file{git} directory. Right at the -bottom, this has the lines: +The rule for make website is found in GNUmakefile.in: @example -# we want this separate for security; see CG 4.2. -gp website: - $(MAKE) config_make=$(config_make) \ - top-src-dir=$(top-src-dir) \ - -f $(top-src-dir)/make/website.make \ - website +$(MAKE) config_make=$(config_make) \ + top-src-dir=$(top-src-dir) \ + -f $(top-src-dir)/make/website.make \ + website @end example -On my system this expands to: +This translates as: @example make --no-builtin-rules config_make=./config.make \ - top-src-dir=/home/phil/lilypond-git \ - -f /home/phil/lilypond-git/make/website.make \ - website + top-src-dir=/home/phil/lilypond-git \ + -f /home/phil/lilypond-git/make/website.make \ + website +@end example + +which has the effect of setting the variables @code{config_make} +and @code{top-src-dir} and then processing the file +@code{git/make/website.make} with the target of website. + +@code{website.make} starts with the following: + +@example +ifeq ($(WEBSITE_ONLY_BUILD),1) +@end example + +which checks to see whether the variable @code{WEBSITE_ONLY_BUILD} +was set to one on the command line. This is only done for +standalone website builds, not in the normal case. The result of +the test determines the value of some variables that are set. A +number of other variables are set, in order to establish locations +of various files. An example is: + +@example +CREATE_VERSION=python $(script-dir)/create-version-itexi.py +@end example + +The rule for website is: + +@smallexample +website: website-texinfo website-css website-pictures website-examples web-post + cp $(SERVER_FILES)/favicon.ico $(OUT)/website + cp $(SERVER_FILES)/robots.txt $(OUT)/website + cp $(top-htaccess) $(OUT)/.htaccess + cp $(dir-htaccess) $(OUT)/website/.htaccess +@end smallexample + +so we see that this starts by running the rules for 5 other +targets, then finishes by copying some files. We'll cover that +later - first @code{website-texinfo}. That rule is: + +@example +website-texinfo: website-version website-xrefs website-bibs + for l in '' $(WEB_LANGS); do \ + if test -n "$$l"; then \ + langopt=--lang="$$l"; \ + langsuf=.$$l; \ + fi; \ + $(TEXI2HTML) --prefix=index \ + --split=section \ + --I=$(top-src-dir)/Documentation/"$$l" \ + --I=$(top-src-dir)/Documentation \ + --I=$(OUT) \ + $$langopt \ + --init-file=$(texi2html-init-file) \ + -D web_version \ + --output=$(OUT)/"$$l" \ + $(top-src-dir)/Documentation/"$$l"/web.texi ; \ + ls $(OUT)/$$l/*.html | xargs grep -L \ + 'UNTRANSLATED NODE: IGNORE ME' | \ + sed 's!$(OUT)/'$$l'/!!g' | xargs \ + $(MASS_LINK) --prepend-suffix="$$langsuf" \ + hard $(OUT)/$$l/ $(OUT)/website/ ; \ + done +@end example + +which therefore depends on @code{website-version}, +@code{website-xrefs} and @code{website-bibs}. + +@example +website-version: + mkdir -p $(OUT) + $(CREATE_VERSION) $(top-src-dir) > $(OUT)/version.itexi + $(CREATE_WEBLINKS) $(top-src-dir) > $(OUT)/weblinks.itexi +@end example + +which translates as: + +@example +mkdir -p out-website +python /home/phil/lilypond-git/scripts/build/create-version-itexi.py + /home/phil/lilypond-git > out-website/version.itexi +python /home/phil/lilypond-git/scripts/build/create-weblinks-itexi.py + /home/phil/lilypond-git > out-website/weblinks.itexi +@end example + +So, we make out-website then send the output of +@code{create-version-itexi.py} to @code{out-website/version.itexi} +and @code{create-weblinks-itexi.py} to +@code{out-website/weblinks.itexi}. + +@code{create-version-itexi.py} parses the file @code{VERSION} in +the top source dir. It contains: + +@example +PACKAGE_NAME=LilyPond +MAJOR_VERSION=2 +MINOR_VERSION=15 +PATCH_LEVEL=13 +MY_PATCH_LEVEL= +VERSION_STABLE=2.14.2 +VERSION_DEVEL=2.15.12 +@end example + +currently. @code{c-v-i.py} parses this to: + +@example +@@c ************************ Version numbers ************ +@@macro version +2.15.13 +@@end macro + +@@macro versionStable +2.14.2 +@@end macro + +@@macro versionDevel +2.15.12 +@@end macro +@end example + +@code{create-weblinks-itexi.py} creates a load of texi macros (of +the order of 1000) similar to: + +@example +@@macro manualStableGlossaryPdf +@@uref@{../doc/v2.14/Documentation/music-glossary.pdf,Music glossary.pdf@} +@@end macro. +@end example + +It loads its languages from langdefs.py, and therefore outputs the following unhelpful warning: + +@code{langdefs.py: warning: lilypond-doc gettext domain not found.} + +Next: + +@example +website-xrefs: website-version + for l in '' $(WEB_LANGS); do \ @end example -We see that the @code{$(MAKE)} expands to -@code{make --no-builtin-rules} which is how @code{MAKE} is -defined higher up the makefile. The -f switch defines the -makefile to be used - in this case -@file{git/make/website.make}. That's where all the action -happens. +is the start of the rule, truncated for brevity. This loops +through the languages to be used on the website, processing some +variables which I don't fully understand, to run this command: -We believe that note that *none* of the variables that -are loaded (from depth to version numbers to whatever) are used in -@file{website.make}. Instead, @file{website.make} sets up its own -variables at the top of the file. If you're wondering if there's -some smart reason for this, then the answer is "no". It's because -I (GP) didn't know/trust the original variables when I was writing -that file. +@smallexample +python /home/phil/lilypond-git/scripts/build/extract_texi_filenames.py \ + -I /home/phil/lilypond-git/Documentation \ + -I /home/phil/lilypond-git/Documentation/"$l" \ + -I out-website -o out-website --split=node \ + --known-missing-files= \ + /home/phil/lilypond-git/scripts/build/website-known-missing-files.txt \ + -q \ + /home/phil/lilypond-git/Documentation/"$l"/web.texi ;\ +@end smallexample -Website build includes @ref{Building a bibliography}. +There's a good description of what +@code{extract_texi_filenames.py} does at the top of the script, +but a shortened version is: -@subsubheading Output from @code{make -n website} +@code{If this script is run on a file texifile.texi, it produces +a file texifile[.LANG].xref-map with tab-separated entries +of the form NODE\tFILENAME\tANCHOR.} -Sorry, including this output directly produces problems in the -build system. Please run: +An example from +@code{web.nl.xref-map} is: @example -make -n website &> my-file.txt +Inleiding Introduction Introduction @end example -to see the full output from the make. +@code{e-t-f.py} follows the includes from document to document. +We know some have not been created yet, and +@code{known-missing-files} option tells @code{e-t-f.py} which +these are. -@subsubheading website.make variables +It then does this: -The file begins by setting up some variables. These -may/might/probably mirror existing variables, but lacking any docs -about those variables, I thought it would be simpler to keep -everything in the same file. +@example +for m in $(MANUALS); do \ +@end example -Note that for security reasons, we @strong{don't} call scripts in -the git dir when building on the web server. See @ref{Uploading -and security}. So we definitely want to keep those definitions -for the WEBSITE_ONLY_BUILD. +to run @code{e-t-f.py} against all of the manuals, in each +language. Next: -After some split WEBSITE_ONLY_BUILD vs. normal build definitions, -there's another bunch of lines setting up generic variables. +@example +website-bibs: website-version + BSTINPUTS=$(top-src-dir)/Documentation/web \ + $(WEB_BIBS) -s web \ + -s $(top-src-dir)/Documentation/lily-bib \ + -o $(OUT)/others-did.itexi \ + $(quiet-flag) \ + $(top-src-dir)/Documentation/web/others-did.bib +@end example -@subsubheading website.make building parts +This is half the command. It runs @code{bib2texi.py} on 2 +@code{.bib} files - @code{others-did.bib} and @code{we-wrote.bib}. +This converts bibliography files into texi files with +@code{bibtex}. -Parts of @file{website.make}: +Next the commands in the @code{website-texinfo} rule are run: + +@example +for l in '' $(WEB_LANGS); do \ +@end example + +run @code{texi2html}. This is the program that outputs the +progress message (found in +@code{Documentation/lilypond-texi2html.init}): + +@code{Processing web site: []} + +It also outputs warning messages like: + +@code{WARNING: Unable to find node 'Řešení potíží' in book usage.} + +@example +website-css: + cp $(top-src-dir)/Documentation/css/*.css $(OUT)/website +@end example + +Copies 3 css files to out-website/website. Then: + +@example +website-pictures: + mkdir -p $(OUT)/website/pictures + if [ -d $(PICTURES) ]; \ + then \ + cp $(PICTURES)/* $(OUT)/website/pictures ; \ + ln -sf website/pictures $(OUT)/pictures ;\ + fi +@end example + +which translates as: + +@smallexample +if [ -d Documentation/pictures/out-www ]; \ + then \ + cp Documentation/pictures/out-www/* out-website/website/pictures ; \ + ln -sf website/pictures out-website/pictures ;\ + fi +@end smallexample + +i.e. it copies the contents of +@code{build/Documentation/pictures/out-www/*} to +@code{out-website/website/pictures}. Unfortunately, the pictures +are only created once @code{make doc} has been run, so an initial +run of @code{make website} copies nothing, and the pictures on the +website (e.g. the logo) do not exist. Next: + +@example +website-examples: + mkdir -p $(OUT)/website/ly-examples + if [ -d $(EXAMPLES) ]; \ + then \ + cp $(EXAMPLES)/* $(OUT)/website/ly-examples ; \ + fi +@end example + +translates to: + +@smallexample +mkdir -p out-website/website/ly-examples +if [ -d Documentation/web/ly-examples/out-www ]; \ + then \ + cp Documentation/web/ly-examples/out-www/* out-website/website/ly-examples ; \ + fi +@end smallexample + +This does the same with the LilyPond examples (found at +@uref{http://lilypond.org/examples.html}). Again, these are +actually only created by @code{make doc} (and since they are +generated from LilyPond source files, require a working LilyPond +@code{exe} made with @code{make}). So this does nothing +initially. Then: + +@example +web-post: + $(WEB_POST) $(OUT)/website +@end example + +which is: + +@smallexample +python /home/phil/lilypond-git/scripts/build/website_post.py out-website/website +@end smallexample + +which describes itself as: + +@code{This is web_post.py. This script deals with translations +in the "make website" target.} + +It also does a number of other things, including adding the Google +tracker code and the language selection footer. We're now at +the end of our story. The final 4 lines of the recipe for website +are: + +@example +cp $(SERVER_FILES)/favicon.ico $(OUT)/website +cp $(SERVER_FILES)/robots.txt $(OUT)/website +cp $(top-htaccess) $(OUT)/.htaccess +cp $(dir-htaccess) $(OUT)/website/.htaccess +@end example + +The first translates as: + +@smallexample +cp /home/phil/lilypond-git/Documentation/web/server/favicon.ico out-website/website +@end smallexample + +so we see these are just copying the support files for the web +server. + +@subsubheading website.make summary + +Recipes in @file{website.make}: @itemize @@ -1043,7 +1316,6 @@ of the process it produces. @item @code{website-version}: this calls the python scripts below: - @itemize @item @example @@ -1068,7 +1340,6 @@ stable/devel, source/docs, lang/lang/lang*10, etc. @end itemize - @item @code{website-xrefs:} creates files used for complicated "out-of-build" references to @@ -1175,66 +1446,3 @@ website: runs website_post.py Then some file copying @end example - -@node Building an Ubuntu distro -@section Building an Ubuntu distro - - -Here's the short instruction on how to create lilybuntu iso image -(Jonathan Kulp did this on a spare drive, -but he supposes it can be done in a VM too): - -@enumerate - -@item -Install ubuntu, reboot. -@item -Run all updates, reboot if asked. -@item -Enable src repos, refresh package lists. -@item -Install LilyPond build deps: -@example -sudo apt-get build-dep lilypond -@end example -@item -Install git and autoconf: -@example -sudo apt-get install git-core gitk autoconf -@end example - -@item -Test to see whether everything works fine now: -@enumerate -@item -use @command{lily-git.tcl} to grab source files -@item -go to source dir and do -@example -"./autogen.sh" ; make ; make doc -@end example -@item -if all compiles, move on to iso creation... -@end enumerate - -@item -Download & install "remastersys": -@uref{http://sourceforge.net/projects/remastersys/, http://sourceforge.net/projects/remastersys/} -@item -Copy @command{lily-git.tcl} script file into @file{/etc/skel/}. -@item -Modify @file{/etc/remastersys.conf} as desired (change @code{.iso} name, -default live session username, etc). -@item -Remove non-essential desktop software as desired. -@item -Create iso: -@example -sudo remastersys dist -@end example -New iso is in @file{/home/remastersys/remastersys/}. -@item -Test iso by installing in VM and repeating steps above for -getting source files and building lp and docs. - -@end enumerate