By default gulp's src() treats files as UTF-8, which corrupts binary
files like MP3s when copying them for packaging. This makes it treat
them as binary.
(cherry picked from commit dd6be31a8e)
Set "type": "commonjs" in the generated dist/package.json file,
since the .js files in the package root are CJS. This should be a
no-op since that's the default, but by setting it explicitly we
ensure that any change to the repository top-level
package.json to set "type": "module" won't break the
published package accidentally.
This is in effect a partial rollback of PR #7822.
This should solve issues encountered by users of bunders that don't
support exports at all (e.g. browserify) as well as ones that don't
support it in certain circumstances (e.g., when using webpack's
resolve.alias configuration option to alias 'blockly' to
'node_modules/blockly', as we formerly did in most plugins, which
causes webpack to ignore blockly's package.json entirely).
Assumptions:
- Such bundlers will _completely_ ignore the exports declaration.
- The bundles are intended to be used in a browser—or at least not
in node.js—so the core entrypoint never needs to route to
core-node.js. This is reasonable since there's little reason to
bundle code for node.js, and node.js has supported the exports
clause since at least v12, considerably older than any version of
node.js we officially support.
- It suffices to provide only a CJS entrypoint (because we can only
provide CJS or ESM, not both. (We could in future switch to
providing only an ESM entrypoint instead, though.)
* fix(typings): Remove bogus .d.ts files; add new languages
PR #3821 added .d.ts files for every file in msg/json/, but several
of these are internal utility files rather than translations, and
do not result in a langfile being output by create_messages.py
when building langfiles.
In the meantime we have added a few new languages that are being
published but which have (until now) not had the corresponding
type declarations.
* feat(build)!: Add exports section to package.json
Add an exports stanza to package.json, enumerating existing
entrypoints in a new format.
- The original main entrypoint, index.js, is removed since the
exports section can point directly at node.js or browser.js.
- No change made (yet) to other entrypoints (core, blocks,
generators); these will be dealt with in a subsequent PR.
- The msg/en entrypoint is included in the top-level package.json
as an example; entries for all other languages created as part
of the packageJSON package task.
BREAKING CHANGE: The introduction of an exports stanza means that
correctly-behaved tools (node.js, bundlers like webpack, etc.)
will only allow importing of the specified entrypoints. Here
is the full list of permitted entrypoints that can be imported
or required:
- blockly
- blockly/core
- blockly/blocks
- blockly/dart
- blockly/lua
- blockly/javascript
- blockly/php
- blockly/python
- blockly/msg/<lang>, for all supported language codes <lang>
(e.g blockly/msg/en, blockly/msg/fr, blockly/msg/de, etc.)
If you previously impored any other paths from the blockly package
you will need to update your imports. Here are the most common
paths that may have been used, and their correct replacements:
| If you previously imported: | Import instead: |
| -------------------------------- | -------------------------- |
| blockly/index.js | blockly |
| blockly/node.js | blockly |
| blockly/browser.js | blockly |
| blockly/blockly.min | This file should only be loaded as a <script>. |
| blockly/core.js | blockly/core |
| blockly/core-browser.js | blockly/core |
| blockly/blockly_compressed.js | blockly/core |
| blockly/blocks.js | blockly/blocks |
| blockly/blocks_compressed.js | blockly/blocks |
| blockly/dart.js | blockly/dart |
| blockly/dart_compressed.js | blockly/dart |
| blockly/lua.js | blockly/lua |
| blockly/lua_compressed.js | blockly/lua |
| blockly/javascript.js | blockly/javascript |
| blockly/javascript_compressed.js | blockly/javascript |
| blockly/php.js | blockly/php |
| blockly/php_compressed.js | blockly/php |
| blockly/python.js | blockly/python |
| blockly/python_compressed.js | blockly/python |
| blockly/msg/en.js | blockly/msg/en |
* fix(build): Use package-paths (blockly/*) in wrapper imports
Use 'blockly/core' instead of './core' when importing core into
other wrappers (and similarly for other entries in package.json
exports stanza), so that (e.g.) dist/javascript.js won't
import dist/core.js (the node.js version that loads jsdom) when
being loaded in a browser environment.
This fixes an issue where blockly attempts to load jsdom even
in browser environments because the browser stanza in
package.json, which caused attempts to load core.js to load
core-browser.js instead in browser environments, was removed
in a previous commit.
* refactor(build): Remove unnecessray wrappers
Remove pointless wrapper modules that no longer server any
purpose; use exports stanza in package.json to point directly to
compiled chunks where possible.
* refactor(build)!: Eliminate separate browser and node entrypoints
Combine scripts/package/browser/index.js (becomes dist/browser.js)
and scripts/package/node/index.js (becomes dist/node.js) into
a single environment-agnostic index.js.
BREAKING CHANGE: Historically, importing the main 'blockly' package
would import 'blockly/core', 'blockly/blocks', 'blockly/en' and
'blockly/javascript' - and additionally, in node.js, also import
'blockly/dart', 'blockly/lua', 'blockly/php' and 'blockly/python'.
Now the main 'blockly' package entrypoint never loads any of the
generator modules.
This change has been made because of changes to generator exports
made in blockly v9.0.0 that make necessary to always separately
import generator modules.
Note that this change does not affect loading the blockly package
via <script src="https://unpkg.com/blockly"; that continues to
load to blockly.min.js, which includes javascript_compressed.js
and (due to being loaded as a script) makes it available via
Blockly.JavaScript.
* refactor(build): Simplify core entrypoint wrapper for node.js
Move scripts/package/node/core.js to scripts/package/core-node.js,
and have it packaged as dist/core-node.js rather than dist/core.js
- without a UMD wrapper, since it will always be loaded as a CJS
module.
* chore(build): Remove disused packageCommonJS helper
* refactor(build): Use subpath pattern (wildcard) for msg/* exports
Use a subpath pattern (wildcard) for the msg/* entrypoints,
obviating the need for special handling in packageJSON.
* fix(tests): Fix node tests
run_node_test.js previously directly require()d the dist/blockly.js
and dist/javascript.js wrapper module, which no longer exist.
Change it to require('blockly-test') (and …blockly-test/javascript)
and create a symlink ./node_modules/blocky-test -> dist/ to satisfy
this.
* fix(build): Add types: and default: entries to exports['./core']
In the 'blockly/core' export:
- Replace the browser: entrypoint with a default: one.
- Add a types: entrypoint for core.
Temporarily exclude the generated .d.ts files for blocks/* and
generators/* from the package.
This will in due course replace the (very simplistic) hand-written
versions in typings/, but for now they are not referenced
anywhere a developer's tooling should be looking, and contain
in some cases actually incorrect typings (e.g., in unmigrated
blocks files, the blocks export is typed as ObjectConstructor,
which is wrong), so do not include them in the package least they
cause problems for the unwary.
* Add tsick.js to rewrite enums.
tsc generates JavaScript which is incompatible with the Closure Compiler's advanced optimizations.
* Remove unused 'outputCode' variable.
* Rename 'run_X_in_browser.js' to 'webdriver.js'
The Mocha and generator tests can both be run either manually or via our webdriver. In all cases they run in a browser. These two 'run_X_in_browser.js' files only apply to webdriver, thus they are confusingly named.
Also delete completely unused (and broken) `run_all_tests.sh`
* Linting improvements to mocha/webdriver.js
Still not at 100%. Complains about require/module/process/__dirname not being defined in multiple places.
* runTestBlock -> runTestFunction
'Block' means something very different in Blockly.
* Removal of `var` from scripts.
* Add webdriver test to verify compile test worked.
* Resolve conficts with 'develop'.
* Address PR comments.
* feat(build): Make build tasks invoke their prerequisites
- Divide gulp targets into three kinds: main sequence,
manually invokable, and script-only. The first two categories
automatically invoke their prerequisites.
- Give (most of) the affected gulp targets shorter and more memorable
names that could become their npm script names in future.
* feat(build): Make package tasks invoke their prerequisites
Have the package task invoke the cleanBuildDir (as well as
cleanPackageDir) and build tasks. Remove the checkBuildDir
task as it is now redundant since a fresh build is done every
time.
* feat(build): Make git tasks invoke their prerequisites
* feat(build): Make cleanup, license [sic] tasks invoke their prerequisites
Turns out they don't have any, so this commit just classifies
their gulp targets according to the established scheme.
* feat(build): Make appengine tasks invoke their prerequisites
In this case prepareDeployDir will eventually depend on package
but does not for now.
* feat(build): Have npm scripts run npm ci first where applicable
Have any npm script that have external effects (e.g. publishing an
npm package, pushing a new version to appengine, or updating GitHub
Pages) start by running npm ci to ensure that all dependencies are
up-to-date with respect to package-lock.json.
(This is done by npm and not a gulp script because gulp itself
might need updating. So might npm, but that is less likely to
make any difference to what gets published/pushed.)
* chore(build): have tests use package target
Have the tests just run the package target (with debug flags)
since that runs the the build target automatically.
* feat(tests): Write Closure Compiler output directly to dist/
Since they are already UMD-wrapped, have Closure Compiler write
output chunks directly to RELEASE_DIR, i.e. dist/.
* chore(tests): Use freshly-build files in compressed mode.
Use the freshly-built build/*_compresssed.js files when bootstrapping
in compressed mode, rather than using the checked-in files in the
repository root.
This helps ensure that compressed and uncompressed mode will be
testing (as closely as possible) the same code.
Obsoletes #6218 (though the issues discussed there have not actually
yet been addressed in this branch).
* chore(build): Write intermediate langfiles to build/msg
Write the results of create_messages.py to build/msg instead of
build/msg/js.
* fix(build): Use build/msg/en.js instead of msg/messages.js in tests
This has no direct effect but fixes a long-standing misdesign
where we are testing against the input to, rather than the output
of, the language file processing pipeline.
* feat(demos): Use freshly-built files
Use the freshly-built dist/*_compresssed.js and build/msg/* files
rather than using the checked-in files in the repository root.
This helps ensure that these demos are using the most recent
version of Blockly (even in the develop branch).
* fix(build): Update appengine deployment to include built files
Modify the prepareDemos task as follows:
- Use the git index instead of HEAD, so that most local changes
will be applied (without copying whatever .gitignored cruft
might be in the local directory).
- Run clean and build and then copy build/msg and
dist/*_compressed.js* to the deploy directory.
This fixes the problem created by the previous commit, wherein the
demos relied on built files that were not being deployed to
appengine.
* fix(build): Update GitHub Pages deployment to include built files
Modify the updateGithubPages task to run clean and build and
then git add build/msg dist/*_compressed.js*, so that they will
be included in the deployed pages.
This fixes the problem created by the previous^2 commit,
wherein the demos relied on built files that were not being
deployed to GitHub Pages.
* chore(build): Remove build products from repository
Remove *_compressed.js* and msg/js/* from the blockly repository.
Also remove the now-obsolete checkinBuilt gulp task.
* chore(build): Apply relevant changes to test_tasks.js
Apply changes made to run_all_tests.sh and check_metadata.sh to
the corresponding parts of their JS replacements in
test_tasks.js.
* chore(build): Make updates suggested in PR #6475
- Remove `clean:builddir` and `clean:releasedir` - `clean`
is sufficient.
- Remove duplicate `require` from `appengine_tasks.js`.
* feat(build): Use shorter npm script names
Since scripts that run build tasks now automatically run their
prerequisite tasks, the previous naming scheme of task `build`
running all the `build:subtask`s no longe really makes very
much sense.
Additionally, following a chat discussion, there seems to be a
rough consensus to use "messages" to refer to the .json input
files, and "langfiles" to the generated .js output files.
Consequently, simplify npm script names by renaming as follows:
- "generate:langfiles" -> "messages"
- "build:langfiles" -> "langfiles"
- "build:js" -> "tsc"
- "build:deps" -> "deps"
- "build:compiled" -> "minify"
- "build:compressed": delete this synonym for "build:compiled",
("minify" was chosen as agnostic to Closure Compiler vs. WebPack.)
* chores(build): Add deprecation notice for old scripts
To reduce potential confusion/frustration, restore the previous
npm scripts but have them display a deprecation notice instead
(note that npm prints the script contents before running it, so
echo is not needed).
* docs(build): Add comments distinguishing 'messages' from 'langfiles'
* chore: regen msg dts files
* fix: package tasks not packaging msg.d.ts files
* fix: add setLocale to blockly.ts
* chore: format
* chore: move setLocale
* chore: add comment about setLocale not being useful when used with script tags
* chore: format
* chore(build): Add inline sources to sourcemaps
* chore(build): Don't package sources
Since the sources are now inline in the sourcemaps, they no longer
need to be package separately.
* fix: loading messages in the browser
* chore: fix comment
* fix: change unwrapped message files to write to a new object, rather than Blockly.Msg
* fix: fixup exports
* fix: PR comments
* fix: change to use for-in loop
* fix: ES6 compatibility and formatting
* chore: Clean up NPM package module wrappers
- Slightly improve documentation for each file based on helpful
explanations given by @samelhusseini.
- Removed redundant code---e.g., loading `javascript_compressed.js`
creates and sets Blockly.JavaScript as a side effect, so there is
no need to set `Blockly.JavaScript = BlocklyJavaScript` in
`dist/javascript.js` (generated from `scripts/package/javascript.js`).
- Remove possibly harmful code---e.g., `Blockly.Msg` is initialised
with a null-prototype object in `blockly_compressed.js` and that
initial object should under no circumstances be replaced.
- Remvoe downright misleading code---e.g., `dist/blocks.js` previously
_appeared_ to replace Blockly.Blocks with an empty object, but in
fact the `Blockly` name referred at that point to the exports object
from `blocks_compressed.js`, which would randomly get a useless
`{}`-valued `.Blocks` property tacked on to it; similarly, code in
`dist/browser.js` (generated from `scripts/package/browser/index.js`)
appeared to copy definitions from `BlocklyBlocks` to `Blockly.Blocks`,
but the former would always be (the aforementioned) empty object,
making this code ineffective.
* chore: Improve chunk definition / UMD generation
Make several improvements to the chunks global and chunkWrapper
function:
- Document chunk definition format (and improve the names of
of the documented properties).
- Replace the chunk `.namespace` property with two others:
- `.exports` names the variable/property to be returned by the
factory function, and which will be set on the global object if
the module is loaded in a browser.
- `.importAs` names the parameter that this chunk's exports value
is to be passed to the factory function of other chunks which
depend on this one. (This needs to be different because e.g.
`Blockly.blocks` is not a valid parameter name.)
- Change the definition for the blocks chunk to export Blockly.Blocks
(i.e., the block definition dictionary) as blocks_compressed.js
did previous to PR #5721 (chunked compilation), rather than the
(empty and soon to vanish) Blockly.blocks namespace object.
This is a win for backwards compatibility, though it does mean that
if we want to expose the `loopTypes` export from `blocks/loops.js`
we will need to find a different way to do so.
- Reconfigure package_tasks.js to use BUILD_DIR from config.js
(i.e., build/) rather than repository root as source of files to
package.
- Add check to packageTasks.package to ensure that certain required
files exist in BUILD_DIR, to verify that buildTasks.build and
typings.typings have been run.
- Fix packageUMDBundle to use generated, rather than checked-in
version of en.js.
- Fix packageDTS to use the generated (rather than checked-in)
versions of blockly.d.ts and msg/*.d.ts.
- Modify run_all_tests.sh to run packageTasks.package before running
node tests, since they depend on it. Previously this was only
working because 'npm install' runs the 'prepare' script, which would
run the 'package' script - so the code being tested by the node tests
was not the current source but whatever precomipled code had
previously been checked in.
- Remove the 'prepare' script from package.json, since it is no longer
needed (and is now broken, since it requires that build and typings
have been done first.) Note that no scripts at all are included in
the version of package.json that is created in dist/ and subsequently
included in the published npms, so this deletion does not affect what
happens when the Blockly npm in installed - it only affects what
happens when 'npm install' is run after the blockly repo is cloned.
- Factor out the actual recompilation steps from releaseTasks.recompile.
- Rename releaseTasks.recomple to recompileDevelop, since it deals
specifically with the develop branch. (The npm script name is
unchanged.)
- Ensure that a full recompile and repackage is done before publishing
(beta and non-beta) npms.
The documented release process is to do npm run recompile, merge the
resulting branch to develop, and then do npm run relase, which does
not do another build.
This process should probably be changed, but for the moment ensure
that npm run recompile (as well as npm run package:beta) runs
buildTasks.checkinBuilt after each .build to preserve the old procedure.
You can now do npm run clean:buildDir, ... clean:releaseDir, or just
... clean, which does both.
The release directory is automatically cleaned before packaging
commences.
Make the destination directories for certain build/package/release
steps more easily (and centrally) configurable.
This only deals with building *_compressed* files;
blockly_uncompressed.js and the various msg/js/*.js files are not
affected by this commit.
