Commit Graph

191 Commits

Author SHA1 Message Date
Christopher Allen
3e9878169b chore(build): Update compiler & linter input language to ES2020 (#5745)
Also:

- Ensure that the `comma-dangle` rule will not be applied to
function parameter lists (even when multi-line).

- Update tests/node/.eslintrc.json to make the environment
node-specific and not pinned to es6.
2021-11-30 01:31:03 +00:00
Christopher Allen
25a4561e21 Migrate blocks/colour.js to goog.module syntax (#5738)
* Migrate blocks/colour.js to goog.module

* Migrate blocks/colour.js named requires

This causes an apparently harmless reordering of the files for the
first chunk.

* clang-format blocks/colour.js
2021-11-29 22:51:06 +00:00
Christopher Allen
985af10f6e chore(build): Use chunked compilation (#5721)
* chore(build): Add "all" modules for blocks & generators

These modules (Blockly.blocks.all and Blockly.<Generator>.all) will
be the entry points for the corresponding chunks.

They also make it easier to pull in all the modules in each package
(e.g. for playground and tests).

It is necessary to set the Closure Compiler dependency_mode to
SORT_ONLY as otherwise it tries to compile the "all" modules before
their dependencies, which fails.

The only impact on the _compressed.js files is the addition of a short
string to the very end of each file, e.g.:

    var module$exports$Blockly$JavaScript$all={};

* chore(deps): Add devDependency on closure-calculate-chunks

* feat(build): First pass at chunked complation

Add a new buildCompiled gulp target (npm run build:compiled) that
uses closure-calculate-chunks to do chunked compliation of core/,
blocks/ and generators/ all in a single pass.

This work is incomplete: the resulting *_compressed.js files don't
(yet) have UMD wrappers.

* chore(build): Generate chunk wrappers

A first pass; this does not have support for a namespace object yet.

* refactor(build): Use chunked compilation by default

Remove old "compressed" gulp tasks in favour of new "compiled" task.

* chore(build): Remove cruft from buildCompiled

Remove unneeded `done` parameter and commented-out options that had
been cargo-culted from the old build pipeline.

* fix(build): Fix test failures caused by new build pipeline

- Exclude closure/goog/base.js from compiler input; use
  externs/goog-externs.js instead.

- Have the build:debug and build:strict targets only build the first
  chunk (blockly_compressed.js).

- Fix namespace entries for blocks and generators.

* fix(build): Fix build failures on node v12

closure-calculate-chunks requires node.js v14 or later.

When running on node.js v14 or later have getChunkOptions save
the output of closure-calculate-chunks to
scripts/gulpfiles/chunks.json.  When running on older versions of
node.js have it use this checked-in, cached output instead of
attempting to run closure-calculate-chunks.

* chore(build): enable --rename_prefix_namespace

This will allow modules in blocks/ and generators/ to use
goog.require to obtain the exports object of goog.modules from
core/.

* fix(build): Always build all chunks

The previous commit enabled --rename_prefix_namespace option to
Closure Compiler, and this causes the buildCompressed target to
work fine when run without --debug or --strict, but adding either
of those flags (as for example when `npm test` runs
`npm run build:debug`) causes an issue:

- Because of many compiler errors in blocks/ and generators/,
  a previous commit added a hack to only build the first chunk
  when doing debug/strict builds.

- When asked to build only one chunk, Closure Compiler ignores the
  --rename_prefix_namespace flag, because it 'correctly' infers
  that there are no later chunks that will need to access global
  variables from the first chunk.

- This causes a test failure, because `npm test` first runs
  `npm run build`, which generates a valid blockly_compressed.js,
  but this is then overrwritten by an invalid one when it next runs
  `npm run build:debug`.

  (The invalid one is missing all `$.` prefixes on 'global' variables,
  including on Blockly, so the wrapper's last two lines -
  "$.Blockly.internal_ = $;" and "return $.Blockly" - fail.)

The fix is to add appropriate @suppress annotations to blocks/*.js and
generators/**/*.js and then remove the first-chunk-only hack.

* refactor(build): Just build once

Since the previous commit caused `npm run build:debug` to do
everything that `... build:compressed` does - and to produce
byte-for-byte identical output - it doesn't make sense to run
both when testing.  To that end:

- Replace the build:debug and build:strict package scripts that
  did `gulp buildCompressed --...` with new scripts build-debug
  and build-strict that do `gulp build --...` instead.

  (The target names are changed so as to extend our existing naming
  convention as follows: a target named "foo:bar" does some sub-part
  of the job done by target "foo", but a target named "foo-bar" does
  all the work of the target "foo" with some extra options.)

- build:debug:log and build:strict:log are similarly replaced with
  build-debug-log and build-strict-log.

- Modify run_all_tests.js to just do `npm run build-debug` instead of
  doing both `npm run build` and `npm run build:debug`.

- Also remove the 'build:blocks' script that should have been removed
  when the buildBlocks gulp task was deleted previously.

* refactor(build): Compile with base_minimal.js instead of base.js

Introduce a (very!) cut-down version of closure/goog/base.js named
base_minimal.js that is used as input to the compiler as an
alternative to using externs/goog-externs.js (which will be deleted
once the buildAdvancedCompilationTest target has been updated).

This will allow use of goog.setTestOnly since it will now exist in
compiled mode, and allows the changes made in 5b112db to filter
base.js out of the files for the first chunk to be reverted.
(It also obliges a change to the compiled-mode check in blockly.js.)

* fix(build): Fix buildAdvanceCompilationTest

- In build_tasks.js:
  - Replace the old compile() function with a new one factored out of
    buildCompiled().
  - Update buildAdvancedCompilationTest to use the new compile()
    and other helpers created in the meantime.
  - Remove no-longer-used maybeAddClosureLibrary().

- Remove externs/{block,generator,goog}-externs.js, which are no longer
  used by any compile pipeline.

- Update core/blockly.js to fix issue with detection of compiled mode
  when using ADVANCED_OPTIMISATIONS.

- Update only other use of globalThis, in core/utils/xml.js, to
  consistently treat it as a dictionary object.

- Update instructions in tests/compile/index.html.

This commit is sort-of-a-prerequisite to #5602; test:compile:advanced
was previously working but the generated `main_compresed.js` would
throw errors upon loading.
2021-11-29 17:50:17 +00:00
Christopher Allen
335ff199d7 refactor: Update uncompiled-mode dependency loading for playground, tests (#5715)
* chore: rename module Blockly.blocks.Lists to ....lists

All the other Blockly.blocks modules have lower-case names.  This
one being named with an upper-case initial appears to have been a
typo on my part.

This module name is not mentioned anywhere else in the source code
(though it will be soon!) so no other files need to be edited.
Further, it does not appear anywhere in the last release (which
before PR #5696) so it is not necessary to add an entry in
renamings.js for this change.

* chore(build): Rationalise deps.js, deps.mocha.js

* Include blocks/*.js (Blockly.blocks.*) in tests/deps.js, since
  these modules are used in the playground.  (They are goog.provide
  modules loaded via <script> tags, so their absence from deps.js
  does not cause errors - but it will when they are migrated to
  goog.module and must be loaded via goog.require.)

* Filter the entries in deps.mocha.js so that it includes only the
  additional mocha test modules (i.e. those not mentioned in deps.js
  already).

* refactor: Load blocks and generators using goog.require
2021-11-17 00:04:45 +00:00
alschmiedt
6448528e9a chore: applies fixes to clang format (#5677)
* chore: update the clang-format version

* chore: specify the clang format version to use

* chore: remove style tag since it will default to file
2021-11-05 12:55:09 -07:00
alschmiedt
8a89e080fd chore: adds a check for properly formatted files (#5624)
* chore: add check for clang format

* chore: updates clang format script
2021-10-25 11:43:59 -07:00
dependabot[bot]
063c8ad91e chore: Bump google-closure-compiler from 20210601.0.0 to 20211006.0.0 (#5592)
* Bump google-closure-compiler from 20210601.0.0 to 20211006.0.0

Bumps [google-closure-compiler](https://github.com/google/closure-compiler-npm) from 20210601.0.0 to 20211006.0.0.
- [Release notes](https://github.com/google/closure-compiler-npm/releases)
- [Commits](https://github.com/google/closure-compiler-npm/compare/v20210601.0.0...v20211006.0.0)

---
updated-dependencies:
- dependency-name: google-closure-compiler
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Remove deleted option from Closure Compiler arguments

The closure compiler no longer supports the undefinedNames warning
group (and indeed even prior to deletion it didn't do anything for
some time).

Per @lauraharker, enabling checkVars, missingProperties and
strictMissingProperties gives about the same check coverage; we
already enable the first two, while the third was already listed
but commented out for the time being.

* Provide externs for base.js functions

Per comment: fixes compiler errors from build:debug.

* Fix erroneous import

Blockly.serialization.ISerializer uses named exports, so we need to
destructure the import.

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Christopher Allen <cpcallen+git@google.com>
2021-10-15 22:30:14 +01:00
Neil Fraser
c929b3015b chore: Convert == to === and != to !== where possible. (#5599) 2021-10-15 09:17:04 -07:00
alschmiedt
1504c999b8 fix: updates the release task to build typings for msgs (#5495) 2021-09-22 17:08:26 -07:00
kozbial
6fc90213a6 Merge branch 'develop' into merge-develop-to-goog_module 2021-09-21 16:04:33 -07:00
Rachel Fenichel
d83eb1364b fix: enable missingRequire in build_tasks (#5510) 2021-09-21 13:19:55 -07:00
alschmiedt
28fadf8c15 Sets flag to true (#5454) 2021-09-17 11:15:20 -07:00
alschmiedt
5b1586ee1b test: update mocha tests to use goog_module (#5440)
* Use goog.module in mocha tests

* Fix compiler warnings

* Make test helpers a module

* Name test modules Blockly.test.*

This is to be more consistent with how non-test modules are named.

Also remove top-level goog.require of TestHelpers (now
Blockly.test.helpers) since requiring a side-effect-less module does
nothing.

* Convert block_test.js and comment_test.js to goog.module syntax

* Address PR comments

* Goog modulify tests

* Goog modulify toolbox helpers

* Fixes imports and moves common tests from workspace_test.js to a helper file.

* Update test deps after rebase

Co-authored-by: Christopher Allen <cpcallen+git@google.com>
2021-09-16 13:00:38 -07:00
Maribeth Bottorff
744e0ecb3d Quiet the typings script unless you tell it to be verbose. (#5362) 2021-08-13 12:58:18 -07:00
Maribeth Bottorff
50e470ccb1 Quiet the typings script unless you tell it to be verbose. (#5359) 2021-08-12 10:15:35 -07:00
Christopher Allen
4c40378b9a Use goog.requireType when importing interfaces (etc.) (#5343)
* Use goog.requireType when importing I* interfaces

Interfaces have no code, so should never be referred to outside of
(JSDoc) comments, and so the modules that define only interfaces never
need to be goog.require'd - goog.requireType is always sufficient.

This commit fixes imports of all modules whose name matches
/(.*\.)?I[A-Z]*/ - i.e., the hungarian-notation named ones in
core/interfaces/.

* Use goog.requireType when only using import for type specifications

Where a module is imported only to used in JSDoc comments it can
(and should) be goog.requireType'd instead of goog.require'd.

* Remove spurious eslint-disable no-unused-vars

There were a few cases where modules were being imported with
goog.require (because they are referred to in code, not just JSDoc
comments) but were prefaced by a spurious eslint suppress.

Remove these, restoring the invariant that an import gets an eslint
if and only if it is a requireType.

* Remove obsolete Closure Compiler error group

stricterMissingRequire has been superceded by missingRequire, and now
causes a Java null pointer exception if supplied.
2021-08-11 18:50:45 +01:00
kozbial
2a62c06a92 Update script to use python3 command 2021-08-10 16:43:08 -07:00
Neil Fraser
52d280e615 Recompile of JSON to JS messages.
Delete unused Klingon message.
Remove '.' from message that confusingly appears after a filename.
2021-08-05 10:39:22 -07:00
Monica Kozbial
96420f0daf Add command to deploy beta version of demos (#4958) 2021-07-21 11:53:52 -07:00
Christopher Allen
2bdf2627af Configure eslint and Closure Compiler to accept ES6 2021-07-13 23:54:36 +01:00
Christopher Allen
d313ec804a Merge branch 'develop' into goog.module-prep
This resolves a conflict in `blockly_uncompressed.js`, and missing
updates to `test/deps.js`, caused by PR #5041.
2021-07-13 22:16:35 +01:00
Christopher Allen
c42c5e72b4 Remove obsolete TODO
No more `execSync('mkdir -p')` so no more need to check if it works on
Windows.
2021-07-13 16:11:54 +01:00
Christopher Allen
ebd6559822 Fix incorrectly-reverted change to mkdirSync call
An intermediate version of this code read:

    fs.mkdir(demoStaticTmpDir, {recursive: true}), done);

but apparently `fs.mkdir` doesn't honour the recursive option, so I
tried to revert the change but munged it instead.

This commit cleans up the mess I made.
2021-07-13 16:11:54 +01:00
Christopher Allen
85b50fcb73 Use fs.mkdirSync instead of execSync('mkdir -p ...')
Also consistently use {recursive: true} in case directory is nested
in an also-not-yet-existent directory.
2021-07-13 16:11:54 +01:00
Christopher Allen
f5a9f2cf2b Load dependency graph from tests/deps.js in uncompiled mode
* New gulp task buildDeps (npm run build:deps) to create tests/deps.js.
* Old gulp task buildUncompressed is deleted.
* blockly_uncompressed.js is now handwritten.
  * And simplified; in particular, BLOCKLY_BOOT is gone.
  * And linted!
* For consistency with the Closure documentation and base.js,
  consistently refer to what blockly_uncompressed.js is used for as
  "uncompiled mode" rather than "uncompressed mode" (but don't yet
  rename it to blockly_uncompiled.js)
2021-07-12 02:28:09 +01:00
Christopher Allen
6ccff4431d Fix line lengths
Including by using the node_modules/.bin/closure2ts wrapper provided by
typescript-closure-tools instead of running the source .js directly.
2021-07-10 13:02:47 +01:00
Sam El-Husseini
6136d93cdf Fix msg typings script and move to typings build directory 2021-07-10 13:02:47 +01:00
Christopher Allen
c45929db22 Rename internal recompile function to rebuildAll 2021-07-09 23:08:59 +01:00
Christopher Allen
ffd0252947 Merge branch 'develop' into build-elsewhere
Fix conflict in package.json.
2021-07-09 14:13:23 +01:00
Christopher Allen
9e72378b69 Finish renaming BUILD_DIR; make tests more error-prone
I had forgotten that I needed to change the value of BUILD_DIR
in several different places.

Added comments warning future editors about this as well as filing
issue #5007 to track fixing this properly.

Despite being misconfigured and therefore failing, the typescript
and metadata test scripts were exiting with status 0, indicating
successful completion.  These have been fixed so they should fail
on any error, including misconfiguration.
2021-07-09 14:06:59 +01:00
Christopher Allen
1e4fa8005a Fix comment TODO formatting. 2021-07-08 17:29:26 +01:00
Christopher Allen
311b2c16d8 Rename BUILD_DIR to "build" (was "built")
Per discussion in PR #4968, the dirname "build" is widely used for
this purpose.
2021-07-08 16:55:15 +01:00
Christopher Allen
ec14cc3ed8 Package build output; fix node tests.
- Reconfigure package_tasks.js to use BUILD_DIR from config.js
  (i.e., build/) rather than repository root as source of files to
  package.

  - Add check to packageTasks.package to ensure that certain required
    files exist in BUILD_DIR, to verify that buildTasks.build and
    typings.typings have been run.

  - Fix packageUMDBundle to use generated, rather than checked-in
    version of en.js.

  - Fix packageDTS to use the generated (rather than checked-in)
    versions of blockly.d.ts and msg/*.d.ts.

- Modify run_all_tests.sh to run packageTasks.package before running
  node tests, since they depend on it.  Previously this was only
  working because 'npm install' runs the 'prepare' script, which would
  run the 'package' script - so the code being tested by the node tests
  was not the current source but whatever precomipled code had
  previously been checked in.

- Remove the 'prepare' script from package.json, since it is no longer
  needed (and is now broken, since it requires that build and typings
  have been done first.)  Note that no scripts at all are included in
  the version of package.json that is created in dist/ and subsequently
  included in the published npms, so this deletion does not affect what
  happens when the Blockly npm in installed - it only affects what
  happens when 'npm install' is run after the blockly repo is cloned.

- Factor out the actual recompilation steps from releaseTasks.recompile.

- Rename releaseTasks.recomple to recompileDevelop, since it deals
  specifically with the develop branch.  (The npm script name is
  unchanged.)

- Ensure that a full recompile and repackage is done before publishing
  (beta and non-beta) npms.
2021-07-08 16:34:50 +01:00
Rachel Fenichel
23602bb177 Initial commit for appengine deploy action 2021-07-07 17:57:48 -07:00
Christopher Allen
e6929decde Also check in sourcemaps
The .js.map files generated by buildCompressed, buildBlocks etc. were
not being copied back by checkinBuilt.
2021-07-07 15:36:39 +01:00
Christopher Allen
37ecce8d80 Have npm run typings use built directory; fix typescript tests
* Modify scripts/gulpfiles/typings.js to write typings to BUILD_DIR.
* Modify tests/scripts/compile_typings.sh to check compilability of
  resulting output from BUILD_DIR.
* Rename checkin script to checkin:built, add a checkin:typings script
  to do the same for .d.ts files, and a new checkin script to do both.
* Have recompile run checkin:typings.
2021-06-30 15:34:54 +01:00
Christopher Allen
1aa35ef438 Merge branch 'develop' into build-elsewhere 2021-06-29 16:22:13 +01:00
Christopher Allen
c521fa9bfa Fix release process
The documented release process is to do npm run recompile, merge the
resulting branch to develop, and then do npm run relase, which does
not do another build.

This process should probably be changed, but for the moment ensure
that npm run recompile (as well as npm run package:beta) runs
buildTasks.checkinBuilt after each .build to preserve the old procedure.
2021-06-23 13:11:29 +01:00
Christopher Allen
e8eb30fa8b Script to clean release directory
You can now do npm run clean:buildDir, ... clean:releaseDir, or just
... clean, which does both.

The release directory is automatically cleaned before packaging
commences.
2021-06-23 13:07:20 +01:00
Christopher Allen
961319b3eb Script to clean build directory 2021-06-22 18:10:02 +01:00
Christopher Allen
e2c4a9b7f9 Script to copy built files from BUILD_DIR to repository
This is to allow built files to be checked in.
2021-06-22 18:08:23 +01:00
Christopher Allen
25f073d0e3 Build msg/js/*.js langfiles in BUILD_DIR 2021-06-22 12:04:30 +01:00
Christopher Allen
a03cd29b68 Separate script to rebuild msg/json/en.json, qqq.json etc.
There are some files in msg/json/ (currently en.json, qqq.json,
constants.json and synonyms.json) that are generated by
scripts/i18n/js_to_json.py as part of the language file build process
- but this only needs to be done when messages.js is updated and
and usually requires some manual cleanup, so remove this step from the
existing buildLangfiles gulp script and create a separate command
('npm run generate:langfiles') to do this when required.
2021-06-22 10:59:24 +01:00
Christopher Allen
4111073292 Write build output to built/ instead of repository root
I have verified that

   npm run build && npm run package

produces an identical dist/ directory compared to the one produced prior
to this and the previous commit.
2021-06-22 10:59:13 +01:00
Christopher Allen
082fd1fc6f Move build and package directory config into new config.js
Make the destination directories for certain build/package/release
steps more easily (and centrally) configurable.

This only deals with building *_compressed* files;
blockly_uncompressed.js and the various msg/js/*.js files are not
affected by this commit.
2021-06-22 10:55:38 +01:00
Christopher Allen
b348e18dfb Don't use path.join to create globs for gulp.src
Per the gulp documentation[1], globs passed to gulp.src should use '/'
as the path separator regardless of the path separator used on whatever
OS we are running on.

[1] https://gulpjs.com/docs/en/getting-started/explaining-globs#segments-and-separators
2021-06-21 19:13:59 +01:00
Monica Kozbial
a15bbf1700 Revert #4697 (#4917) 2021-06-16 09:36:39 -07:00
Neil Fraser
76b5517008 Use null-prototype objects for maps
A {} has a bunch of names already defined on it (like ‘toString’).  When using an object as a map with arbitrary keys, it should not inherit from Object.prototype.
2021-06-09 11:41:42 -07:00
Neil Fraser
d919b0af21 Update JSDoc from Array.<> to Array<>
Same with Object.
2021-05-27 21:30:26 -07:00
Sam El-Husseini
d83279008b Remove dark theme in favour of @blockly/theme-dark (#4697)
* Remove dark theme in favour of @blockly/theme-dark
2021-04-21 13:33:59 -07:00