* fix(build): Correctly handle out-of-order chunks
It turns out that closure-calculate-chunks does not guarantee that
calculated chunks will be output in the same order as the
entrypoints, so modify getChunkOptions so that it no longer makes
that assumption.
* refactor(build): Introduce NAMESPACE_PROPERTY; rename NAMESPACE_OBJECT
Rename the constant NAMESPACE_OBJECT to NAMESPACE_VARIABLE to
better explain its actual meaning, and introduce
NAMESPACE_PROPERTY to specify what property the namespace object
will be stored in (and change the previous value, "internal_",
to "__namespace__" to reduce the chance of conflicts with
properties created by the output of Closure Compiler).
* refactor(build): Always save namespace object on chunk exports object
This is so that chunks whose parent chunk is not the root chunk
(chunks[0]) can obtain the namespace object. (See following commit.)
* fix(build): Correct handling of chunk dependencies
Previously getChunkOptions and chunkWrapper incorrectly assumed
that a chunk could have more than one dependency.
In fact, each chunk can have only a single dependency, which is
its parent chunk. It is used only to retrieve the namespace
object, which is saved on to the exports object for the chunk so
that any child chunk(s) can obtain it.
Update getChunkOptions and chunkWrapper (making the latter
longer but more readable) accordingly.
* refactor(build): Rename/repurpose chunk.exports -> .reexport
And remove chunk.importAs, since it was no longer being used
anywhere.
* fix: remove unnecessary s from reexports
Co-authored-by: alschmiedt <aschmiedt@google.com>
* refactor(tests): Migrate tests/compile/main.js to goog.module
* feat(tests): Enable --debug for test:compile:advanced; demote errors
* Enable the --debug flag when running the
buildAdvancedCompilationTest gulp task.
* Remove partialAlias diagnostic group from `JSC_ERROR`, demoting
JSC_PARTIAL_NAMESPACE to warning.
Reverts "Revert 'fix(tests): Enable --debug for test:compile:advanced;
fix some errors (#5959)'".
Because of our mis-use of @package, and the removal of the hack
that made it work reasonably (in PR #5918), compilation was
generating a large number of JSC_BAD_PACKAGE_PROPERTY_ACCESS
warnings, making it hard to see any other warnings/errors that
might be generated.
To make compiler diagnostics more useful, disable the visibility
diagnostic group entirely for the time being.
Also improve the documentation for JSCOMP_ERROR and the newly-
-introduced JSCOMP_WARNING and JSCOMP_OFF.
...and rename Blockly.blocks.all (blocks/all.js) to
Blockly.libraryBlocks (blocks/blocks.js
BREAKING CHANGE: (only) because the exports object from the
`blocks_compressed.js` chunk will be accessed as
`Blockly.libraryBlocks` instead of `Blockly.blocks.all` when the
chunk is loaded in a browser via a `<script>` tag. There will
be no changes visible when the chunk is loaded via ES module
`import` or CJS `require`.
* fix(build): Correctly handle deep export paths
A problem can occur when loading chunks in a browser: although
factory() will create the full exported path on $, and thus the
assignment
root.<path> = factory(...) { ...; return $.<path> }
will normally be a do-nothing in every chunk except the first, if
the exported path (e.g. Blockly.blocks.all) is more than one level
deeper than any previously-existing path (e.g. Blockly), this will
fail because root.<path> is evaluated before calling factory(), and
so the left hand side will will evaluate to a undefined reference
(e.g. undefined.all) and TypeError will be thrown before the call
to factory is evaluated.
To fix this, call factory first and store the exports object in a
variable before assigning it to the exported path.
Fixes#5932.
* refactor: Stop flattening Blockly sources during compilation in preparation for move to ES modules
* fix: Remove obsolete accessControls and add comment about visibility changes
* refactor: Provide a BlockDefinition type
* refactor: Split defineBlocksWithJsonArray
Split defineBlocksWithJsonArray into:
- createBlockDefinitionsFromJsonArray, which creates BlockDefinitions
from a (possibly JSON-originated) POJsO array, having no side-effects
except possibly issuing warnings to the console.
- defineBlocks, which add any dictionary of BlockDefinitions to
the Blocks dictionary.
* feat(blocks): Export block definitions per-module
- Define all blocks in a local blocks dictionary, often using
createBlockDefinitionFromJsonArray, without registering them.
- Separately, use defineBlocks to register the exported
BlockDefinitions at the end of each Blockly.blocks.*
module.
- In Blockly.blocks.all, create a blocks export that combines all
of the blocks exports from the individual blocks modules.
* chore: have format script run clang-format on blocks/ too
* refactor(blocks): Make loopTypes a Set
This is likely to slightly improve performance, especially if there
are many entries.
* refactor(blocks): Re-export individual block modules from Blockly.blocks.all
* fix!(blocks): Have blocks_compressed.js export Blockly.blocks.all
Previously the value obtained by
const blocks = require('blockly/blocks'); // Node.js
or
import blocks from 'blockly/blocks.js'; // ES Modules
would be the block definitions dictionary (Blockly.Blocks).
Change this so that it is instead the export object from
Blockly.blocks.all.
This means you can now access loopTypes via:
import blocks from 'blockly/blocks.js';
blocks.loops.loopTypes.add('my_loop_blocktype');
This is a breaking change for any code which depended on the value
that was exported by blocks_compressed.js.
BREAKING CHANGE: the exports provided by blocks_compressed.js (and
therefore by `import ... from 'blockly/blocks'`) have changed; see above.
* chore(build): add a script to run the ts compiler and send the new files to closure compiler
* chore(build): clean up tsconfig
* chore: apply suggestions from code review
Cleanup from Chris
Co-authored-by: Christopher Allen <cpcallen+github@gmail.com>
Co-authored-by: Christopher Allen <cpcallen+github@gmail.com>
* chore: Remove requires that duplicate those in blockly.js
* refactor: Remove core/requires.js, make blockly.js be entry point
It sort of makes sense, doesn't it?
The only remaining require not duplicated in `blockly.js` was that for
`Blockly.themes.Classic`, but it is already required by
`Blockly.themes` (itself required by `Blockly`).
* chore: Clean up NPM package module wrappers
- Slightly improve documentation for each file based on helpful
explanations given by @samelhusseini.
- Removed redundant code---e.g., loading `javascript_compressed.js`
creates and sets Blockly.JavaScript as a side effect, so there is
no need to set `Blockly.JavaScript = BlocklyJavaScript` in
`dist/javascript.js` (generated from `scripts/package/javascript.js`).
- Remove possibly harmful code---e.g., `Blockly.Msg` is initialised
with a null-prototype object in `blockly_compressed.js` and that
initial object should under no circumstances be replaced.
- Remvoe downright misleading code---e.g., `dist/blocks.js` previously
_appeared_ to replace Blockly.Blocks with an empty object, but in
fact the `Blockly` name referred at that point to the exports object
from `blocks_compressed.js`, which would randomly get a useless
`{}`-valued `.Blocks` property tacked on to it; similarly, code in
`dist/browser.js` (generated from `scripts/package/browser/index.js`)
appeared to copy definitions from `BlocklyBlocks` to `Blockly.Blocks`,
but the former would always be (the aforementioned) empty object,
making this code ineffective.
* chore: Improve chunk definition / UMD generation
Make several improvements to the chunks global and chunkWrapper
function:
- Document chunk definition format (and improve the names of
of the documented properties).
- Replace the chunk `.namespace` property with two others:
- `.exports` names the variable/property to be returned by the
factory function, and which will be set on the global object if
the module is loaded in a browser.
- `.importAs` names the parameter that this chunk's exports value
is to be passed to the factory function of other chunks which
depend on this one. (This needs to be different because e.g.
`Blockly.blocks` is not a valid parameter name.)
- Change the definition for the blocks chunk to export Blockly.Blocks
(i.e., the block definition dictionary) as blocks_compressed.js
did previous to PR #5721 (chunked compilation), rather than the
(empty and soon to vanish) Blockly.blocks namespace object.
This is a win for backwards compatibility, though it does mean that
if we want to expose the `loopTypes` export from `blocks/loops.js`
we will need to find a different way to do so.
Also:
- Ensure that the `comma-dangle` rule will not be applied to
function parameter lists (even when multi-line).
- Update tests/node/.eslintrc.json to make the environment
node-specific and not pinned to es6.
* chore(build): Add "all" modules for blocks & generators
These modules (Blockly.blocks.all and Blockly.<Generator>.all) will
be the entry points for the corresponding chunks.
They also make it easier to pull in all the modules in each package
(e.g. for playground and tests).
It is necessary to set the Closure Compiler dependency_mode to
SORT_ONLY as otherwise it tries to compile the "all" modules before
their dependencies, which fails.
The only impact on the _compressed.js files is the addition of a short
string to the very end of each file, e.g.:
var module$exports$Blockly$JavaScript$all={};
* chore(deps): Add devDependency on closure-calculate-chunks
* feat(build): First pass at chunked complation
Add a new buildCompiled gulp target (npm run build:compiled) that
uses closure-calculate-chunks to do chunked compliation of core/,
blocks/ and generators/ all in a single pass.
This work is incomplete: the resulting *_compressed.js files don't
(yet) have UMD wrappers.
* chore(build): Generate chunk wrappers
A first pass; this does not have support for a namespace object yet.
* refactor(build): Use chunked compilation by default
Remove old "compressed" gulp tasks in favour of new "compiled" task.
* chore(build): Remove cruft from buildCompiled
Remove unneeded `done` parameter and commented-out options that had
been cargo-culted from the old build pipeline.
* fix(build): Fix test failures caused by new build pipeline
- Exclude closure/goog/base.js from compiler input; use
externs/goog-externs.js instead.
- Have the build:debug and build:strict targets only build the first
chunk (blockly_compressed.js).
- Fix namespace entries for blocks and generators.
* fix(build): Fix build failures on node v12
closure-calculate-chunks requires node.js v14 or later.
When running on node.js v14 or later have getChunkOptions save
the output of closure-calculate-chunks to
scripts/gulpfiles/chunks.json. When running on older versions of
node.js have it use this checked-in, cached output instead of
attempting to run closure-calculate-chunks.
* chore(build): enable --rename_prefix_namespace
This will allow modules in blocks/ and generators/ to use
goog.require to obtain the exports object of goog.modules from
core/.
* fix(build): Always build all chunks
The previous commit enabled --rename_prefix_namespace option to
Closure Compiler, and this causes the buildCompressed target to
work fine when run without --debug or --strict, but adding either
of those flags (as for example when `npm test` runs
`npm run build:debug`) causes an issue:
- Because of many compiler errors in blocks/ and generators/,
a previous commit added a hack to only build the first chunk
when doing debug/strict builds.
- When asked to build only one chunk, Closure Compiler ignores the
--rename_prefix_namespace flag, because it 'correctly' infers
that there are no later chunks that will need to access global
variables from the first chunk.
- This causes a test failure, because `npm test` first runs
`npm run build`, which generates a valid blockly_compressed.js,
but this is then overrwritten by an invalid one when it next runs
`npm run build:debug`.
(The invalid one is missing all `$.` prefixes on 'global' variables,
including on Blockly, so the wrapper's last two lines -
"$.Blockly.internal_ = $;" and "return $.Blockly" - fail.)
The fix is to add appropriate @suppress annotations to blocks/*.js and
generators/**/*.js and then remove the first-chunk-only hack.
* refactor(build): Just build once
Since the previous commit caused `npm run build:debug` to do
everything that `... build:compressed` does - and to produce
byte-for-byte identical output - it doesn't make sense to run
both when testing. To that end:
- Replace the build:debug and build:strict package scripts that
did `gulp buildCompressed --...` with new scripts build-debug
and build-strict that do `gulp build --...` instead.
(The target names are changed so as to extend our existing naming
convention as follows: a target named "foo:bar" does some sub-part
of the job done by target "foo", but a target named "foo-bar" does
all the work of the target "foo" with some extra options.)
- build:debug:log and build:strict:log are similarly replaced with
build-debug-log and build-strict-log.
- Modify run_all_tests.js to just do `npm run build-debug` instead of
doing both `npm run build` and `npm run build:debug`.
- Also remove the 'build:blocks' script that should have been removed
when the buildBlocks gulp task was deleted previously.
* refactor(build): Compile with base_minimal.js instead of base.js
Introduce a (very!) cut-down version of closure/goog/base.js named
base_minimal.js that is used as input to the compiler as an
alternative to using externs/goog-externs.js (which will be deleted
once the buildAdvancedCompilationTest target has been updated).
This will allow use of goog.setTestOnly since it will now exist in
compiled mode, and allows the changes made in 5b112db to filter
base.js out of the files for the first chunk to be reverted.
(It also obliges a change to the compiled-mode check in blockly.js.)
* fix(build): Fix buildAdvanceCompilationTest
- In build_tasks.js:
- Replace the old compile() function with a new one factored out of
buildCompiled().
- Update buildAdvancedCompilationTest to use the new compile()
and other helpers created in the meantime.
- Remove no-longer-used maybeAddClosureLibrary().
- Remove externs/{block,generator,goog}-externs.js, which are no longer
used by any compile pipeline.
- Update core/blockly.js to fix issue with detection of compiled mode
when using ADVANCED_OPTIMISATIONS.
- Update only other use of globalThis, in core/utils/xml.js, to
consistently treat it as a dictionary object.
- Update instructions in tests/compile/index.html.
This commit is sort-of-a-prerequisite to #5602; test:compile:advanced
was previously working but the generated `main_compresed.js` would
throw errors upon loading.
* chore: rename module Blockly.blocks.Lists to ....lists
All the other Blockly.blocks modules have lower-case names. This
one being named with an upper-case initial appears to have been a
typo on my part.
This module name is not mentioned anywhere else in the source code
(though it will be soon!) so no other files need to be edited.
Further, it does not appear anywhere in the last release (which
before PR #5696) so it is not necessary to add an entry in
renamings.js for this change.
* chore(build): Rationalise deps.js, deps.mocha.js
* Include blocks/*.js (Blockly.blocks.*) in tests/deps.js, since
these modules are used in the playground. (They are goog.provide
modules loaded via <script> tags, so their absence from deps.js
does not cause errors - but it will when they are migrated to
goog.module and must be loaded via goog.require.)
* Filter the entries in deps.mocha.js so that it includes only the
additional mocha test modules (i.e. those not mentioned in deps.js
already).
* refactor: Load blocks and generators using goog.require
* Bump google-closure-compiler from 20210601.0.0 to 20211006.0.0
Bumps [google-closure-compiler](https://github.com/google/closure-compiler-npm) from 20210601.0.0 to 20211006.0.0.
- [Release notes](https://github.com/google/closure-compiler-npm/releases)
- [Commits](https://github.com/google/closure-compiler-npm/compare/v20210601.0.0...v20211006.0.0)
---
updated-dependencies:
- dependency-name: google-closure-compiler
dependency-type: direct:development
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
* Remove deleted option from Closure Compiler arguments
The closure compiler no longer supports the undefinedNames warning
group (and indeed even prior to deletion it didn't do anything for
some time).
Per @lauraharker, enabling checkVars, missingProperties and
strictMissingProperties gives about the same check coverage; we
already enable the first two, while the third was already listed
but commented out for the time being.
* Provide externs for base.js functions
Per comment: fixes compiler errors from build:debug.
* Fix erroneous import
Blockly.serialization.ISerializer uses named exports, so we need to
destructure the import.
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Christopher Allen <cpcallen+git@google.com>
* Use goog.module in mocha tests
* Fix compiler warnings
* Make test helpers a module
* Name test modules Blockly.test.*
This is to be more consistent with how non-test modules are named.
Also remove top-level goog.require of TestHelpers (now
Blockly.test.helpers) since requiring a side-effect-less module does
nothing.
* Convert block_test.js and comment_test.js to goog.module syntax
* Address PR comments
* Goog modulify tests
* Goog modulify toolbox helpers
* Fixes imports and moves common tests from workspace_test.js to a helper file.
* Update test deps after rebase
Co-authored-by: Christopher Allen <cpcallen+git@google.com>
* Use goog.requireType when importing I* interfaces
Interfaces have no code, so should never be referred to outside of
(JSDoc) comments, and so the modules that define only interfaces never
need to be goog.require'd - goog.requireType is always sufficient.
This commit fixes imports of all modules whose name matches
/(.*\.)?I[A-Z]*/ - i.e., the hungarian-notation named ones in
core/interfaces/.
* Use goog.requireType when only using import for type specifications
Where a module is imported only to used in JSDoc comments it can
(and should) be goog.requireType'd instead of goog.require'd.
* Remove spurious eslint-disable no-unused-vars
There were a few cases where modules were being imported with
goog.require (because they are referred to in code, not just JSDoc
comments) but were prefaced by a spurious eslint suppress.
Remove these, restoring the invariant that an import gets an eslint
if and only if it is a requireType.
* Remove obsolete Closure Compiler error group
stricterMissingRequire has been superceded by missingRequire, and now
causes a Java null pointer exception if supplied.
* New gulp task buildDeps (npm run build:deps) to create tests/deps.js.
* Old gulp task buildUncompressed is deleted.
* blockly_uncompressed.js is now handwritten.
* And simplified; in particular, BLOCKLY_BOOT is gone.
* And linted!
* For consistency with the Closure documentation and base.js,
consistently refer to what blockly_uncompressed.js is used for as
"uncompiled mode" rather than "uncompressed mode" (but don't yet
rename it to blockly_uncompiled.js)
You can now do npm run clean:buildDir, ... clean:releaseDir, or just
... clean, which does both.
The release directory is automatically cleaned before packaging
commences.
There are some files in msg/json/ (currently en.json, qqq.json,
constants.json and synonyms.json) that are generated by
scripts/i18n/js_to_json.py as part of the language file build process
- but this only needs to be done when messages.js is updated and
and usually requires some manual cleanup, so remove this step from the
existing buildLangfiles gulp script and create a separate command
('npm run generate:langfiles') to do this when required.
Make the destination directories for certain build/package/release
steps more easily (and centrally) configurable.
This only deals with building *_compressed* files;
blockly_uncompressed.js and the various msg/js/*.js files are not
affected by this commit.
A {} has a bunch of names already defined on it (like ‘toString’). When using an object as a map with arbitrary keys, it should not inherit from Object.prototype.
