- Use Path objects from pathlib to handle file paths properly
- Generate output files in the same directory as input file
- Create output filenames with "new_" prefix while preserving original stem name
- Ensure script works with files from any location in the filesystem
* chore: Add puppeteer-core as a dev dependency.
* fix: Make mocha tests run in a fake-focused window.
* fix: Make `test:mocha:interactive` use the same gulp codepath as `test`.
* refactor(build): Rename "package" gulp task (but not npm script) to "pack"
This is to avoid an issue due to "package" being a reserved word
in JavaScript, and therefore not a valid export identifier.
* refactor(build): Convert gulpfile.js from CJS to ESM.
* refactor(build): Convert scripts/gulpfiles/*.js from CJS to ESM
* fix(build): Fix eslint warning for @license tag in gulpfile.mjs
* chore(build): Remove unused imports
* fix(build): Fix incorrect import of gulp-gzip
* fix(build): Fix incorrect sourcemaps import reference
By default gulp's src() treats files as UTF-8, which corrupts binary
files like MP3s when copying them for packaging. This makes it treat
them as binary.
* chore: rename .eslintrc.js to eslint.config.js
* chore: Rename eslint.config.js to eslint.config.mjs.
* refactor: Migrate ESLint config to new flat format.
* chore: Remove old per-directory and global ignore ESLint config files.
* fix: Allowlist JSDoc tag aliases.
* fix: Don't require @license in tests/*.
* fix: Add NodeJS globals to several files that run under Node.
* chore: Remove now-unneeded ESLint directives in core.
* chore: Remove invalid/unneeded ESLint directives.
* fix: Fix invalid use of `await` outside of an `async` function.
* fix: Improve screenshot error message.
* fix: Update ESLint config file to not warn on existing violations.
* chore: Remove suppressions of rules that weren't triggering.
* chore: Fix package-lock.json.
* feat(scripts): Create script to help with CJS -> ESM migration
This is based on js2ts, but considerably simplified since we
no longer need to deal with goog.module IDs.
* feat(scripts): Add support for module.exports = {...}
Support (optionally renaming) assignments to module.exports, in
addition to the existing support for simple exports property
assignmenets.
* fix(scripts): Typo corrections + other improvments for PR #8197
* chore(scripts): Format
Set "type": "commonjs" in the generated dist/package.json file,
since the .js files in the package root are CJS. This should be a
no-op since that's the default, but by setting it explicitly we
ensure that any change to the repository top-level
package.json to set "type": "module" won't break the
published package accidentally.
* feat(build)!: Introduce ESM entrypoints for chunks
Introduce an "import" conditional export for each of the chunk
entrypoints (blockly/core, blockly/blocks, blockly/javascript
etc.), and point these at wrappers created by build_tasks.js
that import the corresponding <chunk>_compressed.js file and
export its named exports.
BREAKING CHANGE:
Importing Blockly via
import Blockly from 'blockly/core';
(and similarly for the other chunk entrypoints) has worked until
now because most build tools (including Webpack in particular)
fuilfil the request for the default export of a CJS module by
providing the module.exports object, rather than an
explicitly-named default export as they would for an ES module.
Since core/blockly.ts (the notional entrypoint for blockly/core)
does not provide a default export, the wrappers created by this
PR do not either.
Code of the above form will therefore break, and should be updated
to use a wildcard:
import * as Blockly from 'blockly/core';
* feat(build)!: Introduce main package ESM entrypoint
Introduce an "import" conditional export for the top-level
package entrypoint (blockly), and point it at a wrappers
created by build_tasks.js that imports the existing index.js
file.
BREAKING CHANGE:
Importing Blockly via
import Blockly from 'blockly';
has worked until now because most build tools (including Webpack
in particular) fuilfil the request for the default export of a
CJS module by providing the module.exports object, rather than an
explicitly-named default export as they would for an ES module.
Since core/blockly.ts does not provide a default export, the
wrapper created by this PR does not either.
Code of the above form will therefore break, and should be updated
to use a wildcard:
import * as Blockly from 'blockly';
* feat(build)!: Introduce ESM entrypoints for langfiles
Introduce an "import" conditional export for each of the
langfile entrypoints (msg/en, msg/fr, etc.),, and point them
at wrappers created by build_tasks.js that import the
existing <lang>.js file.
BREAKING CHANGE:
Importing languages via
import en from 'blockly/msg/en';
has worked until now because most build tools (including Webpack
in particular) fuilfil the request for the default export of a
CJS module by providing the module.exports object, rather than an
explicitly-named default export as they would for an ES module.
Code of the above form will therefore break, and should be updated
to use a wildcard:
import * as en from 'blockly/msg/en';
* fix(typings): Remove bogus .d.ts file.
For some reason we had a typings/msg/yue.d.ts that did not
correxpond to any msg/json/yue.json. Delete it.
* Correct list's "find" and "get" blocks help.
Both blocks were using the same help URL.
Two unrelated results of recompiling messages:
* Allow unwanted 'qqq' messages to be reverted.
* 'constants.json' is unchanged, but sorted.
Also improve python json dumping.
This is in effect a partial rollback of PR #7822.
This should solve issues encountered by users of bunders that don't
support exports at all (e.g. browserify) as well as ones that don't
support it in certain circumstances (e.g., when using webpack's
resolve.alias configuration option to alias 'blockly' to
'node_modules/blockly', as we formerly did in most plugins, which
causes webpack to ignore blockly's package.json entirely).
Assumptions:
- Such bundlers will _completely_ ignore the exports declaration.
- The bundles are intended to be used in a browser—or at least not
in node.js—so the core entrypoint never needs to route to
core-node.js. This is reasonable since there's little reason to
bundle code for node.js, and node.js has supported the exports
clause since at least v12, considerably older than any version of
node.js we officially support.
- It suffices to provide only a CJS entrypoint (because we can only
provide CJS or ESM, not both. (We could in future switch to
providing only an ESM entrypoint instead, though.)
* fix(typings): Remove bogus .d.ts files; add new languages
PR #3821 added .d.ts files for every file in msg/json/, but several
of these are internal utility files rather than translations, and
do not result in a langfile being output by create_messages.py
when building langfiles.
In the meantime we have added a few new languages that are being
published but which have (until now) not had the corresponding
type declarations.
* feat(build)!: Add exports section to package.json
Add an exports stanza to package.json, enumerating existing
entrypoints in a new format.
- The original main entrypoint, index.js, is removed since the
exports section can point directly at node.js or browser.js.
- No change made (yet) to other entrypoints (core, blocks,
generators); these will be dealt with in a subsequent PR.
- The msg/en entrypoint is included in the top-level package.json
as an example; entries for all other languages created as part
of the packageJSON package task.
BREAKING CHANGE: The introduction of an exports stanza means that
correctly-behaved tools (node.js, bundlers like webpack, etc.)
will only allow importing of the specified entrypoints. Here
is the full list of permitted entrypoints that can be imported
or required:
- blockly
- blockly/core
- blockly/blocks
- blockly/dart
- blockly/lua
- blockly/javascript
- blockly/php
- blockly/python
- blockly/msg/<lang>, for all supported language codes <lang>
(e.g blockly/msg/en, blockly/msg/fr, blockly/msg/de, etc.)
If you previously impored any other paths from the blockly package
you will need to update your imports. Here are the most common
paths that may have been used, and their correct replacements:
| If you previously imported: | Import instead: |
| -------------------------------- | -------------------------- |
| blockly/index.js | blockly |
| blockly/node.js | blockly |
| blockly/browser.js | blockly |
| blockly/blockly.min | This file should only be loaded as a <script>. |
| blockly/core.js | blockly/core |
| blockly/core-browser.js | blockly/core |
| blockly/blockly_compressed.js | blockly/core |
| blockly/blocks.js | blockly/blocks |
| blockly/blocks_compressed.js | blockly/blocks |
| blockly/dart.js | blockly/dart |
| blockly/dart_compressed.js | blockly/dart |
| blockly/lua.js | blockly/lua |
| blockly/lua_compressed.js | blockly/lua |
| blockly/javascript.js | blockly/javascript |
| blockly/javascript_compressed.js | blockly/javascript |
| blockly/php.js | blockly/php |
| blockly/php_compressed.js | blockly/php |
| blockly/python.js | blockly/python |
| blockly/python_compressed.js | blockly/python |
| blockly/msg/en.js | blockly/msg/en |
* fix(build): Use package-paths (blockly/*) in wrapper imports
Use 'blockly/core' instead of './core' when importing core into
other wrappers (and similarly for other entries in package.json
exports stanza), so that (e.g.) dist/javascript.js won't
import dist/core.js (the node.js version that loads jsdom) when
being loaded in a browser environment.
This fixes an issue where blockly attempts to load jsdom even
in browser environments because the browser stanza in
package.json, which caused attempts to load core.js to load
core-browser.js instead in browser environments, was removed
in a previous commit.
* refactor(build): Remove unnecessray wrappers
Remove pointless wrapper modules that no longer server any
purpose; use exports stanza in package.json to point directly to
compiled chunks where possible.
* refactor(build)!: Eliminate separate browser and node entrypoints
Combine scripts/package/browser/index.js (becomes dist/browser.js)
and scripts/package/node/index.js (becomes dist/node.js) into
a single environment-agnostic index.js.
BREAKING CHANGE: Historically, importing the main 'blockly' package
would import 'blockly/core', 'blockly/blocks', 'blockly/en' and
'blockly/javascript' - and additionally, in node.js, also import
'blockly/dart', 'blockly/lua', 'blockly/php' and 'blockly/python'.
Now the main 'blockly' package entrypoint never loads any of the
generator modules.
This change has been made because of changes to generator exports
made in blockly v9.0.0 that make necessary to always separately
import generator modules.
Note that this change does not affect loading the blockly package
via <script src="https://unpkg.com/blockly"; that continues to
load to blockly.min.js, which includes javascript_compressed.js
and (due to being loaded as a script) makes it available via
Blockly.JavaScript.
* refactor(build): Simplify core entrypoint wrapper for node.js
Move scripts/package/node/core.js to scripts/package/core-node.js,
and have it packaged as dist/core-node.js rather than dist/core.js
- without a UMD wrapper, since it will always be loaded as a CJS
module.
* chore(build): Remove disused packageCommonJS helper
* refactor(build): Use subpath pattern (wildcard) for msg/* exports
Use a subpath pattern (wildcard) for the msg/* entrypoints,
obviating the need for special handling in packageJSON.
* fix(tests): Fix node tests
run_node_test.js previously directly require()d the dist/blockly.js
and dist/javascript.js wrapper module, which no longer exist.
Change it to require('blockly-test') (and …blockly-test/javascript)
and create a symlink ./node_modules/blocky-test -> dist/ to satisfy
this.
* fix(build): Add types: and default: entries to exports['./core']
In the 'blockly/core' export:
- Replace the browser: entrypoint with a default: one.
- Add a types: entrypoint for core.
* fix(build): Revert "refactor: Remove $build$src infix from munged paths"
This is a mostly-manual revert of commit
06d78af6a4 to fix an issue where
the generated sourcemaps are missing the inline copies of the
original .ts source files.
* chore: Update metadata for 2020 Q3 release
This is being done a second time as the revert of 06d78af causes a
significant increase in the size of the build products.
* refactor(build): Don't use closure-calculate-chunks
Rewrite the getChunkOptions function to not use
closure-calculate-chunks, but instead just chunk the input files
(more or less) by subdirectory: first chunk is core/, second is
blocks/, etc.
This does make a material change to blockly_compressed.js,
because we end up feeding several empty modules that contain
only typescript interface declarations and which tsc
compiles to "export {};" in the input to Closure Compiler
(closure-calculate-chunks is smart enough to notice that
no other module depends on these), which results in ~1.7KiB of
superflous
var module$build$src$core$interfaces$i_ast_node_location_svg={};
declarations. This can be avoided by filtering such empty modules
out but that has been left for a future commit.
This adds the glob NPM package as a dev dependency, but gulp
and several other existing dev dependencies already depend on
it.
Build time is sped up by about a factor of 3x, due to removal
of the buildDeps step that was really slow:
$ time npm run build
before:
real 0m24.410s
user 0m16.010s
sys 0m1.140s
after:
real 0m8.397s
user 0m11.976s
sys 0m0.694s
* chore(build): Remove buildDeps task
* refactor: Remove $build$src infix from munged paths
Closure Compiler renames module globals so that they do not
clash when multiple modules are bundled together. It does so
by adding a "$$module$build$src$path$to$module" suffix (with
the module object istelf being named simply
"$module$build$src$path$to$module").
By changing the gulp.src base option to be build/src/ instead
of ./ (referring to the repostiory root), Closure Compiler
obligingly shortens all of these munged named by removing the
"$build$src" infix, reducing the size of the compressed chunks
by about 10%; blockly_compressed.js goes from 900595 to 816667
bytes.
* chore(build): Compute module object munged name from entrypoint
- Add modulePath to compute the munged name of the entrypoint
module from the entrypoint module filename, and use this
instead of hard-coded chunk.exports paths.
- Be more careful about when we are using poxix vs. OS-specific
paths, especially TSC_OUTPUT_PATH which is regularly passed
to gulp.src, which is documented as taking only posix paths.
- Rename one existing variable modulePath -> entryPath to try
to avoid confusion with new modulePath function.
* fix(build): Restore erroneously-deleted filter function
This was deleted in PR #7406 as it was mainly being used to
filter core/ vs. test/mocha/ deps into separate deps files -
but it turns out also to be used for filtering error
messages too. Oops.
* refactor(tests): Migrate advanced compilation test to ES Modules
* refactor(build): Migrate main.js to TypeScript
This turns out to be pretty straight forward, even if it would
cause crashing if one actually tried to import this module
instead of just feeding it to Closure Compiler.
* chore(build): Remove goog.declareModuleId calls
Replace goog.declareModuleId calls with a comment recording the
former module ID for posterity (or at least until we decide
how to reformat the renamings file.
* chore(tests): Delete closure/goog/*
For the moment we still need something to serve as base.js for
the benefit of closure-make-deps, so we keep a vestigial
base.js around, containing only the @provideGoog declaration.
* refactor(build): Remove vestigial base.js
By changing slightly the command line arguments to
closure-make-deps and closure-calculate-chunks the need to have
any base.js is eliminated.
* chore: Typo fix for PR #7415
* refactor(tests): Use shims instead of bootstrap to load Blockly
- Modify tests/generators/index.html to import the test shims
instead of using bootstrap.js to load Blockly.
- Modify test/generators/webdriver.js to have it wait for the
workspace to exist before calling loadSelected(). There was
previously a race which index.html had been winning, but
now webdriver.js is winning (and the tests failing because
there is no workspace yet when start() is called.
* chore(tests): Delete bootstrap.js etc.
- Delete bootstrap.js, bootstrap_helper.js, and bootstrap_done.mjs.
- Remove remaining references to bootstrap.js
* refactor(build): Remove deps npm script
buildDeps is now only needed by buildCompiled, not ever for
runnning in uncompressed mode, so:
- Remove the deps gulp task (and the deps npm script.
- Have the minify task run buildJavaScript and buildDeps directly.
Additionally, the buildAdvanceCompilationTest target hasn't
needed deps.js for some time (if ever), so skip having it run
buildDeps entirely.
* refactor(build): Repatriate DEPS_FILE to build_tasks.js
Since this is no longer used anywhere else it doesn't need to
live in common.js.
* fix(scripts): Remove vestigial references to deps.mocha.js
* docs(tests): Add additional explanatory note
* fix(build): Have buildShims clean up up after itself
We need to create a build/package.json file to allow node.js to
load build/src/core/blockly.js and the other chunk entry points
as ES modules (it forcibly assumes .js means CJS even if one is
trying to import, unless package.json says {"type": "module"}),
but this interferes with scripts/migration/js2ts doing a
require('build/deps.js'), which is _not_ an ES module.
Specific error message was:
/Users/cpcallen/src/blockly/scripts/migration/js2ts:56
require(path.resolve(__dirname, '../../build/deps.js'));
^
Error [ERR_REQUIRE_ESM]: require() of ES Module
/Users/cpcallen/src/blockly/build/deps.js from /Users/cpcallen/src/blockly/scripts/migration/js2ts
not supported.
deps.js is treated as an ES module file as it is a .js file whose
nearest parent package.json contains "type": "module" which
declares all .js files in that package scope as ES modules.
Instead rename deps.js to end in .cjs, change the requiring code
to use dynamic import() which is available in all CommonJS
modules, or change "type": "module" to "type": "commonjs" in
/Users/cpcallen/src/blockly/build/package.json to treat all .js
files as CommonJS (using .mjs for all ES modules instead).
at Object.<anonymous> (/Users/cpcallen/src/blockly/scripts/migration/js2ts:56:1) {
code: 'ERR_REQUIRE_ESM'
}
* chore(tests): Reorder to put interesting script nearer top of file
* chore(tests): Add missing imports of closure/goog/goog.js
These modules were depending on being loaded via the
debug module loader, which cannot be used without first loading
base.js as a script, and thereby defining goog.declareModuleId
as a side effect—but if they are to be loaded via direct import
statements then they need to actually import their own
dependencies.
This is a temporary measure as soon the goog.declareMouleId
calls can themselves be deleted.
* refactor(tests): Use import instead of bootstrap to load Blockly
* chores(build): Stop generating deps.mocha.js
This file was only needed by tests/mocha/index.html's use of
the debug module loader (via bootstrap.js), which has now been
removed.
* chore(tests): Remove unneeded goog.declareModuleId calls
These were only needed because these modules were previously
being loaded by goog.require and/or goog.bootstrap.
* chores(tests): Remove dead code
We are fully committed to proper modules now.
In PR #7380 it was suggested[1] that the shims be renamed from
(e.g.) blockly.mjs to blockly.loader.mjs, and in commit 6f930f5
this was duly done, but alas one place was overlooked.
The problem was not spotted in local testing because the
blockly.mjs module that the blocks and generators chunks were
attempting to import did still exist on disk, left over from
before the change was made.
Running npm run clean would have revealed the issue but alas
that was not done.
[1] https://github.com/google/blockly/pull/7380#discussion_r1291667037
* refactor(build): Simplify implementation of posixPath
* fix(closure): Make safe to import in node.js
Make the implementation declareModlueId safe to import and
execute in node.js, which does not provide a window global.
(N.B. because this is an ESM and therefore automatically
strict, using window?.goog?.declareModuleId doesn't work
because window being undefined is an early error.)
* feat(tests): Introduce chunk loading shims
- Add a buildShims task to build_tasks.js that, for each chunk,
creates a correspondingly-named build/<chunk>.mjs that will
either (in uncompressed mode) import and reexport that chunk's
entry point module (e.g. core/blockly.js) or (in compressed
mode) load dist/<chunk>_compressed.js using a <script> tag
and then export the corresponding properties on the chunk's
exports object.
- Provide helper methods used by these shims in
tests/scripts/loading.mjs, including code to detect whether
to load in compressed or uncompressed mode.
- Add a quote() function to scripts/helpers.js, used by
buildShims. This is copied from tests/bootstrap_helper.js,
which will be removed in a later commit.
* refactor(tests): Update playground.html to use new loading shims
* refactor(tests): Update advanced_playground.html to use new loading shims
* refactor(tests): Update multi_playground.html to use new loading shims
* chore(tests): Delete playgrounds/shared_procedures.html
Shared procedure support was moved to a plugin and this should
have been removed from core along with it.
* docs(tests): Typo corrections.
* chore(tests): Add ".loader" infix to shim filenames.
Per suggestion on PR #7380, have buildShims name the shims
${chunk.name}.loader.mjs instead of just `${chunk.name}.mjs`.
* chore: change the way we look for upstream remote
* chore: use js instead of shell
thanks christopher
Co-authored-by: Christopher Allen <cpcallen+github@gmail.com>
* chore: store the found upstream name
---------
Co-authored-by: Christopher Allen <cpcallen+github@gmail.com>
* fix: update build path for windows
When using single quote on windows, e.g. 'build/src', the folder
are created with a single quote at the beginning `'build` and end
`src'`. This commit fixes this issue.
* fix: update python command and folder separator
Ensure that when running on windows, python command is python and
not python3. Also, separators are normalized to posix style `/` even on
windows system
* fix: add global PYTHON constant to run python command
* fix: simplify `path.sep` to forwadslash since it is cross-platform
* fix(syntax): replace double quote with single quote
