chore(build): Remove build products from the Blockly repository (#6475)

* feat(build): Make build tasks invoke their prerequisites

  - Divide gulp targets into three kinds: main sequence,
    manually invokable, and script-only.  The first two categories
    automatically invoke their prerequisites.
  - Give (most of) the affected gulp targets shorter and more memorable
    names that could become their npm script names in future.

* feat(build): Make package tasks invoke their prerequisites

  Have the package task invoke the cleanBuildDir (as well as
  cleanPackageDir) and build tasks.  Remove the checkBuildDir
  task as it is now redundant since a fresh build is done every
  time.

* feat(build): Make git tasks invoke their prerequisites

* feat(build): Make cleanup, license [sic] tasks invoke their prerequisites

  Turns out they don't have any, so this commit just classifies
  their gulp targets according to the established scheme.

* feat(build): Make appengine tasks invoke their prerequisites

  In this case prepareDeployDir will eventually depend on package
  but does not for now.

* feat(build): Have npm scripts run npm ci first where applicable

  Have any npm script that have external effects (e.g. publishing an
  npm package, pushing a new version to appengine, or updating GitHub
  Pages) start by running npm ci to ensure that all dependencies are
  up-to-date with respect to package-lock.json.

  (This is done by npm and not a gulp script because gulp itself
  might need updating.  So might npm, but that is less likely to
  make any difference to what gets published/pushed.)

* chore(build): have tests use package target

  Have the tests just run the package target (with debug flags)
  since that runs the the build target automatically.

* feat(tests): Write Closure Compiler output directly to dist/

  Since they are already UMD-wrapped, have Closure Compiler write
  output chunks directly to RELEASE_DIR, i.e. dist/.

* chore(tests): Use freshly-build files in compressed mode.

  Use the freshly-built build/*_compresssed.js files when bootstrapping
  in compressed mode, rather than using the checked-in files in the
  repository root.

  This helps ensure that compressed and uncompressed mode will be
  testing (as closely as possible) the same code.

  Obsoletes #6218 (though the issues discussed there have not actually
  yet been addressed in this branch).

* chore(build): Write intermediate langfiles to build/msg

  Write the results of create_messages.py to build/msg instead of
  build/msg/js.

* fix(build): Use build/msg/en.js instead of msg/messages.js in tests

  This has no direct effect but fixes a long-standing misdesign
  where we are testing against the input to, rather than the output
  of, the language file processing pipeline.

* feat(demos): Use freshly-built files

  Use the freshly-built dist/*_compresssed.js and build/msg/* files
  rather than using the checked-in files in the repository root.

  This helps ensure that these demos are using the most recent
  version of Blockly (even in the develop branch).

* fix(build): Update appengine deployment to include built files

  Modify the prepareDemos task as follows:

  - Use the git index instead of HEAD, so that most local changes
    will be applied (without copying whatever .gitignored cruft
    might be in the local directory).
  - Run clean and build and then copy build/msg and
    dist/*_compressed.js* to the deploy directory.

  This fixes the problem created by the previous commit, wherein the
  demos relied on built files that were not being deployed to
  appengine.

* fix(build): Update GitHub Pages deployment to include built files

  Modify the updateGithubPages task to run clean and build and
  then git add build/msg dist/*_compressed.js*, so that they will
  be included in the deployed pages.

  This fixes the problem created by the previous^2 commit,
  wherein the demos relied on built files that were not being
  deployed to GitHub Pages.

* chore(build): Remove build products from repository

  Remove *_compressed.js* and msg/js/* from the blockly repository.
  Also remove the now-obsolete checkinBuilt gulp task.

* chore(build): Apply relevant changes to test_tasks.js

  Apply changes made to run_all_tests.sh and check_metadata.sh to
  the corresponding parts of their JS replacements in
  test_tasks.js.

* chore(build): Make updates suggested in PR #6475

  - Remove `clean:builddir` and `clean:releasedir` - `clean`
    is sufficient.
  - Remove duplicate `require` from `appengine_tasks.js`.

* feat(build): Use shorter npm script names

  Since scripts that run build tasks now automatically run their
  prerequisite tasks, the previous naming scheme of task `build`
  running all the `build:subtask`s no longe really makes very
  much sense.

  Additionally, following a chat discussion, there seems to be a
  rough consensus to use "messages" to refer to the .json input
  files, and "langfiles" to the generated .js output files.

  Consequently, simplify npm script names by renaming as follows:

  - "generate:langfiles" -> "messages"
  - "build:langfiles" -> "langfiles"
  - "build:js" -> "tsc"
  - "build:deps" -> "deps"
  - "build:compiled" -> "minify"
  - "build:compressed": delete this synonym for "build:compiled",

  ("minify" was chosen as agnostic to Closure Compiler vs. WebPack.)

* chores(build): Add deprecation notice for old scripts

  To reduce potential confusion/frustration, restore the previous
  npm scripts but have them display a deprecation notice instead
  (note that npm prints the script contents before running it, so
  echo is not needed).

* docs(build): Add comments distinguishing 'messages' from 'langfiles'

demos/blockfactory/index.html

@@ -4,10 +4,10 @@
   <meta charset="utf-8">
   <meta name="viewport" content="target-densitydpi=device-dpi, height=660, initial-scale=1.0, maximum-scale=1.0, user-scalable=no" />
   <title>Blockly Demo: Blockly Developer Tools</title>
-  <script src="../../blockly_compressed.js"></script>
-  <script src="../../javascript_compressed.js"></script>
-  <script src="../../msg/js/en.js"></script>
-  <script src="../../blocks_compressed.js"></script>
+  <script src="../../dist/blockly_compressed.js"></script>
+  <script src="../../dist/blocks_compressed.js"></script>
+  <script src="../../dist/javascript_compressed.js"></script>
+  <script src="../../build/msg/en.js"></script>
   <script src="analytics.js"></script>
   <script src="block_definition_extractor.js"></script>
   <script src="factory_utils.js"></script>

demos/blockfactory_old/index.html

@@ -6,8 +6,8 @@
   <title>Blockly Demo: Block Factory</title>
   <script src="/storage.js"></script>
   <script src="factory.js"></script>
-  <script src="../../blockly_compressed.js"></script>
-  <script src="../../msg/js/en.js"></script>
+  <script src="../../dist/blockly_compressed.js"></script>
+  <script src="../../build/msg/en.js"></script>
   <script src="blocks.js"></script>
   <style>
     html, body {

demos/code/code.js

@@ -621,6 +621,6 @@ Code.discard = function() {
 // Load the Code demo's language strings.
 document.write('<script src="msg/' + Code.LANG + '.js"></script>\n');
 // Load Blockly's language strings.
-document.write('<script src="../../msg/js/' + Code.LANG + '.js"></script>\n');
+document.write('<script src="../../build/msg/' + Code.LANG + '.js"></script>\n');
 
 window.addEventListener('load', Code.init);

demos/code/index.html

@@ -7,13 +7,13 @@
   <title>Blockly Demo:</title>
   <link rel="stylesheet" href="style.css">
   <script src="/storage.js"></script>
-  <script src="../../blockly_compressed.js"></script>
-  <script src="../../blocks_compressed.js"></script>
-  <script src="../../javascript_compressed.js"></script>
-  <script src="../../python_compressed.js"></script>
-  <script src="../../php_compressed.js"></script>
-  <script src="../../lua_compressed.js"></script>
-  <script src="../../dart_compressed.js"></script>
+  <script src="../../dist/blockly_compressed.js"></script>
+  <script src="../../dist/blocks_compressed.js"></script>
+  <script src="../../dist/javascript_compressed.js"></script>
+  <script src="../../dist/python_compressed.js"></script>
+  <script src="../../dist/php_compressed.js"></script>
+  <script src="../../dist/lua_compressed.js"></script>
+  <script src="../../dist/dart_compressed.js"></script>
   <script src="code.js"></script>
 </head>
 <body>

demos/minimap/index.html

@@ -3,9 +3,9 @@
 <head>
   <meta charset="utf-8">
   <title>Blockly Demo: Minimap</title>
-  <script src="../../blockly_compressed.js"></script>
-  <script src="../../blocks_compressed.js"></script>
-  <script src="../../msg/js/en.js"></script>
+  <script src="../../dist/blockly_compressed.js"></script>
+  <script src="../../dist/blocks_compressed.js"></script>
+  <script src="../../build/msg/en.js"></script>
   <script src="minimap.js"></script>
   <style>
     body {

demos/mobile/android/app/build.gradle

@@ -28,16 +28,30 @@ task copyBlocklyHtmlFile(type: Copy) {
     rename('index.html', 'webview.html')
 }
 
-task copyBlocklyMoreFiles(type: Copy) {
+task copyBlocklyDistFiles(type: Copy) {
+    from('../../../../dist') {
+        include 'blockly_compressed.js', 'blocks_compressed.js'
+    }
+    into project(':app').file('./src/main/assets/blockly')
+}
+
+task copyBlocklyBuildFiles(type: Copy) {
+    from('../../../../build') {
+        include 'msg/**'
+    }
+    into project(':app').file('./src/main/assets/blockly')
+}
+
+task copyBlocklyMediaFiles(type: Copy) {
     from('../../../..') {
-        include 'blockly_compressed.js', 'blocks_compressed.js', 'msg/js/**', 'media/**'
+        include 'media/**'
         exclude 'media/test_*'
     }
     into project(':app').file('./src/main/assets/blockly')
 }
 
 project.afterEvaluate {
-    preBuild.dependsOn(copyBlocklyHtmlFile, copyBlocklyMoreFiles)
+    preBuild.dependsOn(copyBlocklyHtmlFile, copyBlocklyBuildFiles, copyBlocklyDistFiles, copyBlocklyMediaFiles)
 }
 
 dependencies {

demos/mobile/html/index.html

@@ -15,7 +15,7 @@
   <script src="blockly_compressed.js"></script>
   <script src="blocks_compressed.js"></script>
   <!-- TODO: Select msg file based on locale. -->
-  <script src="msg/js/en.js"></script>
+  <script src="msg/en.js"></script>
   <script src="toolbox_standard.js"></script>
 </head>
 <body>
@@ -28,4 +28,4 @@
         });
   </script>
 </body>
-</html>
+</html>

demos/mobile/html/ln_resources.sh

@@ -3,20 +3,20 @@
 # Create symbolic links in this directory for the
 # Blockly library files used by this demo's index.html.
 
-if [[ ! -e ../../../blockly_compressed.js ]]; then
+if [[ ! -e ../../../dist/blockly_compressed.js ]]; then
     echo "ERROR: Could not locate blockly_compressed.js. Run from demos/mobile/html/" 1>&2
     exit 1 # terminate and indicate error
 fi
 
 if [ ! -L blockly_compressed.js ]; then
-    ln -s ../../../blockly_compressed.js blockly_compressed.js
+    ln -s ../../../dist/blockly_compressed.js blockly_compressed.js
 fi
 if [ ! -L blocks_compressed.js ]; then
-    ln -s ../../../blocks_compressed.js blocks_compressed.js
+    ln -s ../../../dist/blocks_compressed.js blocks_compressed.js
 fi
 if [ ! -L media ]; then
     ln -s ../../../media media
 fi
 if [ ! -L msg ]; then
-    ln -s ../../../msg msg
+    ln -s ../../../build/msg msg
 fi

demos/mobile/ios/cp_resources.sh

@@ -5,10 +5,10 @@ BLOCKLY_ROOT=../../..
 IOS_RESOURCES=Resources/Non-Localized/Blockly
 
 MORE_FILES_TO_COPY=(
-  "blockly_compressed.js"
-  "blocks_compressed.js"
+  "dist/blockly_compressed.js"
+  "dist/blocks_compressed.js"
   "media"
-  "msg/js"
+  "build/msg"
   )
 
 mkdir -p $IOS_RESOURCES/media

demos/storage/index.html

@@ -4,9 +4,9 @@
   <meta charset="utf-8">
   <title>Blockly Demo: Cloud Storage</title>
   <script src="/storage.js"></script>
-  <script src="../../blockly_compressed.js"></script>
-  <script src="../../blocks_compressed.js"></script>
-  <script src="../../msg/js/en.js"></script>
+  <script src="../../dist/blockly_compressed.js"></script>
+  <script src="../../dist/blocks_compressed.js"></script>
+  <script src="../../build/msg/en.js"></script>
   <style>
     body {
       background-color: #fff;