Commit Graph

79 Commits

Author SHA1 Message Date
Christopher Allen
985af10f6e chore(build): Use chunked compilation (#5721)
* chore(build): Add "all" modules for blocks & generators

These modules (Blockly.blocks.all and Blockly.<Generator>.all) will
be the entry points for the corresponding chunks.

They also make it easier to pull in all the modules in each package
(e.g. for playground and tests).

It is necessary to set the Closure Compiler dependency_mode to
SORT_ONLY as otherwise it tries to compile the "all" modules before
their dependencies, which fails.

The only impact on the _compressed.js files is the addition of a short
string to the very end of each file, e.g.:

    var module$exports$Blockly$JavaScript$all={};

* chore(deps): Add devDependency on closure-calculate-chunks

* feat(build): First pass at chunked complation

Add a new buildCompiled gulp target (npm run build:compiled) that
uses closure-calculate-chunks to do chunked compliation of core/,
blocks/ and generators/ all in a single pass.

This work is incomplete: the resulting *_compressed.js files don't
(yet) have UMD wrappers.

* chore(build): Generate chunk wrappers

A first pass; this does not have support for a namespace object yet.

* refactor(build): Use chunked compilation by default

Remove old "compressed" gulp tasks in favour of new "compiled" task.

* chore(build): Remove cruft from buildCompiled

Remove unneeded `done` parameter and commented-out options that had
been cargo-culted from the old build pipeline.

* fix(build): Fix test failures caused by new build pipeline

- Exclude closure/goog/base.js from compiler input; use
  externs/goog-externs.js instead.

- Have the build:debug and build:strict targets only build the first
  chunk (blockly_compressed.js).

- Fix namespace entries for blocks and generators.

* fix(build): Fix build failures on node v12

closure-calculate-chunks requires node.js v14 or later.

When running on node.js v14 or later have getChunkOptions save
the output of closure-calculate-chunks to
scripts/gulpfiles/chunks.json.  When running on older versions of
node.js have it use this checked-in, cached output instead of
attempting to run closure-calculate-chunks.

* chore(build): enable --rename_prefix_namespace

This will allow modules in blocks/ and generators/ to use
goog.require to obtain the exports object of goog.modules from
core/.

* fix(build): Always build all chunks

The previous commit enabled --rename_prefix_namespace option to
Closure Compiler, and this causes the buildCompressed target to
work fine when run without --debug or --strict, but adding either
of those flags (as for example when `npm test` runs
`npm run build:debug`) causes an issue:

- Because of many compiler errors in blocks/ and generators/,
  a previous commit added a hack to only build the first chunk
  when doing debug/strict builds.

- When asked to build only one chunk, Closure Compiler ignores the
  --rename_prefix_namespace flag, because it 'correctly' infers
  that there are no later chunks that will need to access global
  variables from the first chunk.

- This causes a test failure, because `npm test` first runs
  `npm run build`, which generates a valid blockly_compressed.js,
  but this is then overrwritten by an invalid one when it next runs
  `npm run build:debug`.

  (The invalid one is missing all `$.` prefixes on 'global' variables,
  including on Blockly, so the wrapper's last two lines -
  "$.Blockly.internal_ = $;" and "return $.Blockly" - fail.)

The fix is to add appropriate @suppress annotations to blocks/*.js and
generators/**/*.js and then remove the first-chunk-only hack.

* refactor(build): Just build once

Since the previous commit caused `npm run build:debug` to do
everything that `... build:compressed` does - and to produce
byte-for-byte identical output - it doesn't make sense to run
both when testing.  To that end:

- Replace the build:debug and build:strict package scripts that
  did `gulp buildCompressed --...` with new scripts build-debug
  and build-strict that do `gulp build --...` instead.

  (The target names are changed so as to extend our existing naming
  convention as follows: a target named "foo:bar" does some sub-part
  of the job done by target "foo", but a target named "foo-bar" does
  all the work of the target "foo" with some extra options.)

- build:debug:log and build:strict:log are similarly replaced with
  build-debug-log and build-strict-log.

- Modify run_all_tests.js to just do `npm run build-debug` instead of
  doing both `npm run build` and `npm run build:debug`.

- Also remove the 'build:blocks' script that should have been removed
  when the buildBlocks gulp task was deleted previously.

* refactor(build): Compile with base_minimal.js instead of base.js

Introduce a (very!) cut-down version of closure/goog/base.js named
base_minimal.js that is used as input to the compiler as an
alternative to using externs/goog-externs.js (which will be deleted
once the buildAdvancedCompilationTest target has been updated).

This will allow use of goog.setTestOnly since it will now exist in
compiled mode, and allows the changes made in 5b112db to filter
base.js out of the files for the first chunk to be reverted.
(It also obliges a change to the compiled-mode check in blockly.js.)

* fix(build): Fix buildAdvanceCompilationTest

- In build_tasks.js:
  - Replace the old compile() function with a new one factored out of
    buildCompiled().
  - Update buildAdvancedCompilationTest to use the new compile()
    and other helpers created in the meantime.
  - Remove no-longer-used maybeAddClosureLibrary().

- Remove externs/{block,generator,goog}-externs.js, which are no longer
  used by any compile pipeline.

- Update core/blockly.js to fix issue with detection of compiled mode
  when using ADVANCED_OPTIMISATIONS.

- Update only other use of globalThis, in core/utils/xml.js, to
  consistently treat it as a dictionary object.

- Update instructions in tests/compile/index.html.

This commit is sort-of-a-prerequisite to #5602; test:compile:advanced
was previously working but the generated `main_compresed.js` would
throw errors upon loading.
2021-11-29 17:50:17 +00:00
alschmiedt
8a89e080fd chore: adds a check for properly formatted files (#5624)
* chore: add check for clang format

* chore: updates clang format script
2021-10-25 11:43:59 -07:00
kozbial
6fc90213a6 Merge branch 'develop' into merge-develop-to-goog_module 2021-09-21 16:04:33 -07:00
Monica Kozbial
96420f0daf Add command to deploy beta version of demos (#4958) 2021-07-21 11:53:52 -07:00
Christopher Allen
f5a9f2cf2b Load dependency graph from tests/deps.js in uncompiled mode
* New gulp task buildDeps (npm run build:deps) to create tests/deps.js.
* Old gulp task buildUncompressed is deleted.
* blockly_uncompressed.js is now handwritten.
  * And simplified; in particular, BLOCKLY_BOOT is gone.
  * And linted!
* For consistency with the Closure documentation and base.js,
  consistently refer to what blockly_uncompressed.js is used for as
  "uncompiled mode" rather than "uncompressed mode" (but don't yet
  rename it to blockly_uncompiled.js)
2021-07-12 02:28:09 +01:00
Christopher Allen
ffd0252947 Merge branch 'develop' into build-elsewhere
Fix conflict in package.json.
2021-07-09 14:13:23 +01:00
Rachel Fenichel
23602bb177 Initial commit for appengine deploy action 2021-07-07 17:57:48 -07:00
Christopher Allen
5410ba6ab3 Do trivial gulp tasks in parallel where possible
N.B. can't run typings.typings and typings.msgTypings in parallel
yet because the latter depends on the existence of an output directory
created by the former.
2021-06-30 16:12:10 +01:00
Christopher Allen
37ecce8d80 Have npm run typings use built directory; fix typescript tests
* Modify scripts/gulpfiles/typings.js to write typings to BUILD_DIR.
* Modify tests/scripts/compile_typings.sh to check compilability of
  resulting output from BUILD_DIR.
* Rename checkin script to checkin:built, add a checkin:typings script
  to do the same for .d.ts files, and a new checkin script to do both.
* Have recompile run checkin:typings.
2021-06-30 15:34:54 +01:00
Christopher Allen
e8eb30fa8b Script to clean release directory
You can now do npm run clean:buildDir, ... clean:releaseDir, or just
... clean, which does both.

The release directory is automatically cleaned before packaging
commences.
2021-06-23 13:07:20 +01:00
Christopher Allen
961319b3eb Script to clean build directory 2021-06-22 18:10:02 +01:00
Christopher Allen
e2c4a9b7f9 Script to copy built files from BUILD_DIR to repository
This is to allow built files to be checked in.
2021-06-22 18:08:23 +01:00
Christopher Allen
a03cd29b68 Separate script to rebuild msg/json/en.json, qqq.json etc.
There are some files in msg/json/ (currently en.json, qqq.json,
constants.json and synonyms.json) that are generated by
scripts/i18n/js_to_json.py as part of the language file build process
- but this only needs to be done when messages.js is updated and
and usually requires some manual cleanup, so remove this step from the
existing buildLangfiles gulp script and create a separate command
('npm run generate:langfiles') to do this when required.
2021-06-22 10:59:24 +01:00
Sam El-Husseini
f97730e83c Sort requires (#4658)
* Cleanup gulp method to sort requires
2021-02-25 17:17:42 -08:00
alschmiedt
1208987dbb Update release scripts (#4558) 2021-01-08 11:52:02 -08:00
Monica Kozbial
77884550a5 Add script for deploying demos (#4551) 2021-01-05 11:08:29 -08:00
Sam El-Husseini
f3136e61a1 Advanced compilation gulp script to replace the shell script. (#3996) 2020-07-01 10:55:17 -07:00
alschmiedt
afba82222e Fix recompile script (#3926) 2020-05-29 11:17:16 -07:00
alschmiedt
8ec3a49487 Update build scripts (#3923)
* Update build scripts
2020-05-28 17:10:52 -07:00
Sam El-Husseini
7e4e6c7286 Add typings modules to match UMD modules. (#3821)
* Add typings modules to match UMD modules.
2020-04-15 09:44:25 -07:00
alschmiedt
40d39fd0c5 Update gh pages task (#3762)
* Add task for updating github pages from develop
2020-03-23 16:49:03 -07:00
Sam El-Husseini
d838778a4a Script to check licenses (#3757)
* Add script task to check licenses of all dependenies
2020-03-19 10:53:29 -07:00
Sam El-Husseini
a6e3f78425 Cleanup build (#3756) 2020-03-18 12:08:24 -07:00
Rachel Fenichel
e78a87a1e5 Break gulpfile into multiple subfiles 2020-02-14 13:42:38 -08:00
Rachel Fenichel
9f356fa27d Add package to exports 2020-02-13 16:29:03 -08:00
Rachel Fenichel
633fa9a5e2 Update gulpfile to use exports instead of gulp.task syntax 2020-02-13 16:11:34 -08:00
Neil Fraser
4e2f8e6e02 Use SPDX licences.
This is a followup to #3127.
At the time, SPDX licenses were pending approval by Google.
2020-02-11 13:27:20 -08:00
Sam El-Husseini
7ef5fe8cec Update gulp task for package-json to remove scripts. (#3650) 2020-01-23 16:23:52 -08:00
Sam El-Husseini
cf90154b23 Support for closure library option in uncompressed local build (#3633)
* Support for closure library option in uncompressed local build

* Refactor into maybeAddClosureLibrary method
2020-01-22 11:10:35 -08:00
alschmiedt
de6a7f9d88 Add push to github pages after build (#3615) 2020-01-21 14:13:21 -08:00
Sam El-Husseini
30aa75506c Release tasks renames (#3620)
* Revert package.json version

* Rename some tasks

* Added prerelease task
2020-01-21 14:08:27 -08:00
Sam El-Husseini
b87d23b976 Rebuild for release with options parsing fix. (#3618) 2020-01-21 13:40:17 -08:00
alschmiedt
847385715a Rebuilds typings for release script (#3612) 2020-01-20 20:53:36 -08:00
alschmiedt
1a2fb6ddf0 Add new build scripts (#3608)
* Add new build scripts

* Fix pr comments
2020-01-16 16:52:20 -08:00
Sam El-Husseini
6bd6bdefbc Upgrade closure + support for closure-library dependency (#3554)
* Upgrade closure and include hints on how to include the closure library in the build.

* Revert build-uncomprssed.

* Sort uncompressed.

* nit: comment.
2020-01-14 12:59:43 -08:00
Sam El-Husseini
034ae878c6 Fix gulp blocks compile build (#3519)
* Fix gulp build by adding Comment, Mutator and Warning to the remove regex.
2019-12-16 14:05:31 -08:00
Sam El-Husseini
e8a294b138 Use google-closure-deps in local build (#3432)
* Use google-closure-deps instead of python script for building in gulpfile (JS)
2019-11-12 11:26:20 -08:00
Neil Fraser
4d25193ead Add extern for goog. (#3417)
As of closure-compiler-v20191027.jar the compile now breaks due to a lack of ‘goog’ being defined.

../../temp_core/constants.js:24: ERROR - [JSC_UNDEFINED_VARIABLE] variable goog is undeclared
goog.provide('Blockly.constants');
^^^^

1 error(s), 0 warning(s)
Compiler exit code: 1
Compilation FAIL.

Upgrade closure compiler to v20191027
2019-11-09 08:10:30 -08:00
Neil Fraser
22c1eba546 Make warnings an optional module. 2019-11-01 11:52:12 -07:00
Sam El-Husseini
93755fef53 Treat compiler warnings as errors and run on travis (#3378)
* Enable compiler warnings as errors and run on travis
2019-10-31 15:49:28 -07:00
Sam El-Husseini
6170b77027 Add langfiles build task in gulp (#3331)
* Add build-langfiles to gulp, and add uncompressed + langfiles to build when building all
2019-10-24 11:11:03 -04:00
Sam El-Husseini
6d8f22f39a Add a lint plugin to ensure we only use ES5 syntax. (#3160)
* Add a lint plugin to ensure we only use ES5 only syntax.
2019-10-21 21:26:26 -04:00
Rachel Fenichel
1e2eb6bd60 Review cleanup 2019-10-21 10:46:34 -07:00
Rachel Fenichel
3946b56a26 Add script to bump minor version number
Creates a new branch, bumps, builds, tags, and pushes the new branch and new tag.
2019-10-18 14:24:06 -07:00
Rachel Fenichel
f56bed5f50 Merge pull request #3285 from rachel-fenichel/release_tasks_2
Add task to create a release candidate.
2019-10-18 11:37:37 -06:00
Rachel Fenichel
9fd4f8edff Change branch name format 2019-10-18 10:23:55 -07:00
Neil Fraser
8e5797e2bc Add missing utils.global (#3286)
Compile is breaking due to the JS environment sweep.
2019-10-18 02:00:31 -07:00
Rachel Fenichel
5507a1eb88 Add task to create a release candidate. 2019-10-17 15:45:43 -07:00
Rachel Fenichel
90f6948c27 Remove unnecessary stash 2019-10-17 10:21:55 -06:00
Rachel Fenichel
17f47ee2cb Make git-sync-master as well as git-sync-develop 2019-10-16 11:29:54 -06:00