* feat(build): Make build tasks invoke their prerequisites
- Divide gulp targets into three kinds: main sequence,
manually invokable, and script-only. The first two categories
automatically invoke their prerequisites.
- Give (most of) the affected gulp targets shorter and more memorable
names that could become their npm script names in future.
* feat(build): Make package tasks invoke their prerequisites
Have the package task invoke the cleanBuildDir (as well as
cleanPackageDir) and build tasks. Remove the checkBuildDir
task as it is now redundant since a fresh build is done every
time.
* feat(build): Make git tasks invoke their prerequisites
* feat(build): Make cleanup, license [sic] tasks invoke their prerequisites
Turns out they don't have any, so this commit just classifies
their gulp targets according to the established scheme.
* feat(build): Make appengine tasks invoke their prerequisites
In this case prepareDeployDir will eventually depend on package
but does not for now.
* feat(build): Have npm scripts run npm ci first where applicable
Have any npm script that have external effects (e.g. publishing an
npm package, pushing a new version to appengine, or updating GitHub
Pages) start by running npm ci to ensure that all dependencies are
up-to-date with respect to package-lock.json.
(This is done by npm and not a gulp script because gulp itself
might need updating. So might npm, but that is less likely to
make any difference to what gets published/pushed.)
* chore(build): have tests use package target
Have the tests just run the package target (with debug flags)
since that runs the the build target automatically.
* feat(tests): Write Closure Compiler output directly to dist/
Since they are already UMD-wrapped, have Closure Compiler write
output chunks directly to RELEASE_DIR, i.e. dist/.
* chore(tests): Use freshly-build files in compressed mode.
Use the freshly-built build/*_compresssed.js files when bootstrapping
in compressed mode, rather than using the checked-in files in the
repository root.
This helps ensure that compressed and uncompressed mode will be
testing (as closely as possible) the same code.
Obsoletes #6218 (though the issues discussed there have not actually
yet been addressed in this branch).
* chore(build): Write intermediate langfiles to build/msg
Write the results of create_messages.py to build/msg instead of
build/msg/js.
* fix(build): Use build/msg/en.js instead of msg/messages.js in tests
This has no direct effect but fixes a long-standing misdesign
where we are testing against the input to, rather than the output
of, the language file processing pipeline.
* feat(demos): Use freshly-built files
Use the freshly-built dist/*_compresssed.js and build/msg/* files
rather than using the checked-in files in the repository root.
This helps ensure that these demos are using the most recent
version of Blockly (even in the develop branch).
* fix(build): Update appengine deployment to include built files
Modify the prepareDemos task as follows:
- Use the git index instead of HEAD, so that most local changes
will be applied (without copying whatever .gitignored cruft
might be in the local directory).
- Run clean and build and then copy build/msg and
dist/*_compressed.js* to the deploy directory.
This fixes the problem created by the previous commit, wherein the
demos relied on built files that were not being deployed to
appengine.
* fix(build): Update GitHub Pages deployment to include built files
Modify the updateGithubPages task to run clean and build and
then git add build/msg dist/*_compressed.js*, so that they will
be included in the deployed pages.
This fixes the problem created by the previous^2 commit,
wherein the demos relied on built files that were not being
deployed to GitHub Pages.
* chore(build): Remove build products from repository
Remove *_compressed.js* and msg/js/* from the blockly repository.
Also remove the now-obsolete checkinBuilt gulp task.
* chore(build): Apply relevant changes to test_tasks.js
Apply changes made to run_all_tests.sh and check_metadata.sh to
the corresponding parts of their JS replacements in
test_tasks.js.
* chore(build): Make updates suggested in PR #6475
- Remove `clean:builddir` and `clean:releasedir` - `clean`
is sufficient.
- Remove duplicate `require` from `appengine_tasks.js`.
* feat(build): Use shorter npm script names
Since scripts that run build tasks now automatically run their
prerequisite tasks, the previous naming scheme of task `build`
running all the `build:subtask`s no longe really makes very
much sense.
Additionally, following a chat discussion, there seems to be a
rough consensus to use "messages" to refer to the .json input
files, and "langfiles" to the generated .js output files.
Consequently, simplify npm script names by renaming as follows:
- "generate:langfiles" -> "messages"
- "build:langfiles" -> "langfiles"
- "build:js" -> "tsc"
- "build:deps" -> "deps"
- "build:compiled" -> "minify"
- "build:compressed": delete this synonym for "build:compiled",
("minify" was chosen as agnostic to Closure Compiler vs. WebPack.)
* chores(build): Add deprecation notice for old scripts
To reduce potential confusion/frustration, restore the previous
npm scripts but have them display a deprecation notice instead
(note that npm prints the script contents before running it, so
echo is not needed).
* docs(build): Add comments distinguishing 'messages' from 'langfiles'
* chore(build): Add "all" modules for blocks & generators
These modules (Blockly.blocks.all and Blockly.<Generator>.all) will
be the entry points for the corresponding chunks.
They also make it easier to pull in all the modules in each package
(e.g. for playground and tests).
It is necessary to set the Closure Compiler dependency_mode to
SORT_ONLY as otherwise it tries to compile the "all" modules before
their dependencies, which fails.
The only impact on the _compressed.js files is the addition of a short
string to the very end of each file, e.g.:
var module$exports$Blockly$JavaScript$all={};
* chore(deps): Add devDependency on closure-calculate-chunks
* feat(build): First pass at chunked complation
Add a new buildCompiled gulp target (npm run build:compiled) that
uses closure-calculate-chunks to do chunked compliation of core/,
blocks/ and generators/ all in a single pass.
This work is incomplete: the resulting *_compressed.js files don't
(yet) have UMD wrappers.
* chore(build): Generate chunk wrappers
A first pass; this does not have support for a namespace object yet.
* refactor(build): Use chunked compilation by default
Remove old "compressed" gulp tasks in favour of new "compiled" task.
* chore(build): Remove cruft from buildCompiled
Remove unneeded `done` parameter and commented-out options that had
been cargo-culted from the old build pipeline.
* fix(build): Fix test failures caused by new build pipeline
- Exclude closure/goog/base.js from compiler input; use
externs/goog-externs.js instead.
- Have the build:debug and build:strict targets only build the first
chunk (blockly_compressed.js).
- Fix namespace entries for blocks and generators.
* fix(build): Fix build failures on node v12
closure-calculate-chunks requires node.js v14 or later.
When running on node.js v14 or later have getChunkOptions save
the output of closure-calculate-chunks to
scripts/gulpfiles/chunks.json. When running on older versions of
node.js have it use this checked-in, cached output instead of
attempting to run closure-calculate-chunks.
* chore(build): enable --rename_prefix_namespace
This will allow modules in blocks/ and generators/ to use
goog.require to obtain the exports object of goog.modules from
core/.
* fix(build): Always build all chunks
The previous commit enabled --rename_prefix_namespace option to
Closure Compiler, and this causes the buildCompressed target to
work fine when run without --debug or --strict, but adding either
of those flags (as for example when `npm test` runs
`npm run build:debug`) causes an issue:
- Because of many compiler errors in blocks/ and generators/,
a previous commit added a hack to only build the first chunk
when doing debug/strict builds.
- When asked to build only one chunk, Closure Compiler ignores the
--rename_prefix_namespace flag, because it 'correctly' infers
that there are no later chunks that will need to access global
variables from the first chunk.
- This causes a test failure, because `npm test` first runs
`npm run build`, which generates a valid blockly_compressed.js,
but this is then overrwritten by an invalid one when it next runs
`npm run build:debug`.
(The invalid one is missing all `$.` prefixes on 'global' variables,
including on Blockly, so the wrapper's last two lines -
"$.Blockly.internal_ = $;" and "return $.Blockly" - fail.)
The fix is to add appropriate @suppress annotations to blocks/*.js and
generators/**/*.js and then remove the first-chunk-only hack.
* refactor(build): Just build once
Since the previous commit caused `npm run build:debug` to do
everything that `... build:compressed` does - and to produce
byte-for-byte identical output - it doesn't make sense to run
both when testing. To that end:
- Replace the build:debug and build:strict package scripts that
did `gulp buildCompressed --...` with new scripts build-debug
and build-strict that do `gulp build --...` instead.
(The target names are changed so as to extend our existing naming
convention as follows: a target named "foo:bar" does some sub-part
of the job done by target "foo", but a target named "foo-bar" does
all the work of the target "foo" with some extra options.)
- build:debug:log and build:strict:log are similarly replaced with
build-debug-log and build-strict-log.
- Modify run_all_tests.js to just do `npm run build-debug` instead of
doing both `npm run build` and `npm run build:debug`.
- Also remove the 'build:blocks' script that should have been removed
when the buildBlocks gulp task was deleted previously.
* refactor(build): Compile with base_minimal.js instead of base.js
Introduce a (very!) cut-down version of closure/goog/base.js named
base_minimal.js that is used as input to the compiler as an
alternative to using externs/goog-externs.js (which will be deleted
once the buildAdvancedCompilationTest target has been updated).
This will allow use of goog.setTestOnly since it will now exist in
compiled mode, and allows the changes made in 5b112db to filter
base.js out of the files for the first chunk to be reverted.
(It also obliges a change to the compiled-mode check in blockly.js.)
* fix(build): Fix buildAdvanceCompilationTest
- In build_tasks.js:
- Replace the old compile() function with a new one factored out of
buildCompiled().
- Update buildAdvancedCompilationTest to use the new compile()
and other helpers created in the meantime.
- Remove no-longer-used maybeAddClosureLibrary().
- Remove externs/{block,generator,goog}-externs.js, which are no longer
used by any compile pipeline.
- Update core/blockly.js to fix issue with detection of compiled mode
when using ADVANCED_OPTIMISATIONS.
- Update only other use of globalThis, in core/utils/xml.js, to
consistently treat it as a dictionary object.
- Update instructions in tests/compile/index.html.
This commit is sort-of-a-prerequisite to #5602; test:compile:advanced
was previously working but the generated `main_compresed.js` would
throw errors upon loading.
