Files
blockly/core/field_image.ts
Ben Henning d0ad9343f0 feat: Add initial support for screen readers (experimental) (#9280)
## The basics

- [x] I [validated my changes](https://developers.google.com/blockly/guides/contribute/core#making_and_verifying_a_change)

## The details
### Resolves

Fixes part of #8207
Fixes part of #3370

### Proposed Changes

This introduces initial broad ARIA integration in order to enable at least basic screen reader support when using keyboard navigation.

Largely this involves introducing ARIA roles and labels in a bunch of places, sometimes done in a way to override normal built-in behaviors of the accessibility node tree in order to get a richer first-class output for Blockly (such as for blocks and workspaces).

### Reason for Changes

ARIA is the fundamental basis for configuring how focusable nodes in Blockly are represented to the user when using a screen reader. As such, all focusable nodes requires labels and roles in order to correctly communicate their contexts.

The specific approach taken in this PR is to simply add labels and roles to all nodes where obvious with some extra work done for `WorkspaceSvg` and `BlockSvg` in order to represent blocks as a tree (since that seems to be the best fitting ARIA role per those available: https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/Reference/Roles). The custom work specifically for blocks includes:
- Overriding the role description to be 'block' rather than 'tree item' (which is the default).
- Overriding the position, level, and number of sibling counts since those are normally determined based on the DOM tree and blocks are not laid out in the tree the same way they are visually or logically (so these computations were incorrect). This is also the reason for a bunch of extra computation logic being introduced.

One note on some of the labels being nonsensical (e.g. 'DoNotOverride?'): this was done intentionally to try and ensure _all_ focusable nodes (that can be focused) have labels, even when the specifics of what that label should be aren't yet clear. More components had these temporary labels until testing revealed how exactly they would behave from a screen reader perspective (at which point their roles and labels were updated as needed). The temporary labels act as an indicator when navigating through the UI, and some of the nodes can't easily be reached (for reasons) and thus may never actually need a label. More work is needed in understanding both what components need labels and what those labels should be, but that will be done beyond this PR.

### Test Coverage

No tests are added to this as it's experimental and not a final implementation.

The keyboard navigation tests are failing due to a visibility expansion of `connectionCandidate` in `BlockDragStrategy`. There's no way to avoid this breakage, unfortunately. Instead, this PR will be merged and then https://github.com/google/blockly-keyboard-experimentation/pull/684 will be finalized and merged to fix it. There's some additional work that will happen both in that branch and in a later PR in core Blockly to integrate the two experimentation branches as part of #9283 so that CI passes correctly for both branches.

### Documentation

No documentation is needed at this time.

### Additional Information

This work is experimental and is meant to serve two purposes:
- Provide a foundation for testing and iterating the core screen reader experience in Blockly.
- Provide a reference point for designing a long-term solution that accounts for all requirements collected during user testing.

This code should never be merged into `develop` as it stands. Instead, it will be redesigned with maintainability, testing, and correctness in mind at a future date (see https://github.com/google/blockly-keyboard-experimentation/discussions/673).
2025-08-06 15:28:45 -07:00

317 lines
8.4 KiB
TypeScript

/**
* @license
* Copyright 2012 Google LLC
* SPDX-License-Identifier: Apache-2.0
*/
/**
* Image field. Used for pictures, icons, etc.
*
* @class
*/
// Former goog.module ID: Blockly.FieldImage
import {Field, FieldConfig} from './field.js';
import * as fieldRegistry from './field_registry.js';
import * as aria from './utils/aria.js';
import * as dom from './utils/dom.js';
import * as parsing from './utils/parsing.js';
import {Size} from './utils/size.js';
import {Svg} from './utils/svg.js';
/**
* Class for an image on a block.
*/
export class FieldImage extends Field<string> {
/**
* Vertical padding below the image, which is included in the reported height
* of the field.
*/
private static readonly Y_PADDING = 1;
protected readonly imageHeight: number;
/** The function to be called when this field is clicked. */
private clickHandler: ((p1: FieldImage) => void) | null = null;
/** The rendered field's image element. */
protected imageElement: SVGImageElement | null = null;
/**
* Editable fields usually show some sort of UI indicating they are
* editable. This field should not.
*/
override readonly EDITABLE = false;
/**
* Used to tell if the field needs to be rendered the next time the block is
* rendered. Image fields are statically sized, and only need to be
* rendered at initialization.
*/
protected override isDirty_ = false;
/** Whether to flip this image in RTL. */
private flipRtl = false;
/** Alt text of this image. */
private altText = '';
/**
* @param src The URL of the image.
* Also accepts Field.SKIP_SETUP if you wish to skip setup (only used by
* subclasses that want to handle configuration and setting the field value
* after their own constructors have run).
* @param width Width of the image.
* @param height Height of the image.
* @param alt Optional alt text for when block is collapsed.
* @param onClick Optional function to be called when the image is
* clicked. If onClick is defined, alt must also be defined.
* @param flipRtl Whether to flip the icon in RTL.
* @param config A map of options used to configure the field.
* See the [field creation documentation]{@link
* https://developers.google.com/blockly/guides/create-custom-blocks/fields/built-in-fields/image#creation}
* for a list of properties this parameter supports.
*/
constructor(
src: string | typeof Field.SKIP_SETUP,
width: string | number,
height: string | number,
alt?: string,
onClick?: (p1: FieldImage) => void,
flipRtl?: boolean,
config?: FieldImageConfig,
) {
super(Field.SKIP_SETUP);
const imageHeight = Number(parsing.replaceMessageReferences(height));
const imageWidth = Number(parsing.replaceMessageReferences(width));
if (isNaN(imageHeight) || isNaN(imageWidth)) {
throw Error(
'Height and width values of an image field must cast to' + ' numbers.',
);
}
if (imageHeight <= 0 || imageWidth <= 0) {
throw Error(
'Height and width values of an image field must be greater' +
' than 0.',
);
}
/** The size of the area rendered by the field. */
this.size_ = new Size(imageWidth, imageHeight + FieldImage.Y_PADDING);
/**
* Store the image height, since it is different from the field height.
*/
this.imageHeight = imageHeight;
if (typeof onClick === 'function') {
this.clickHandler = onClick;
}
if (src === Field.SKIP_SETUP) return;
if (config) {
this.configure_(config);
} else {
this.flipRtl = !!flipRtl;
this.altText = parsing.replaceMessageReferences(alt) || '';
}
this.setValue(parsing.replaceMessageReferences(src));
}
/**
* Configure the field based on the given map of options.
*
* @param config A map of options to configure the field based on.
*/
protected override configure_(config: FieldImageConfig) {
super.configure_(config);
if (config.flipRtl) this.flipRtl = config.flipRtl;
if (config.alt) {
this.altText = parsing.replaceMessageReferences(config.alt);
}
}
/**
* Create the block UI for this image.
*/
override initView() {
this.imageElement = dom.createSvgElement(
Svg.IMAGE,
{
'height': this.imageHeight + 'px',
'width': this.size_.width + 'px',
'alt': this.altText,
},
this.fieldGroup_,
);
this.imageElement.setAttributeNS(
dom.XLINK_NS,
'xlink:href',
this.value_ as string,
);
if (this.fieldGroup_) {
dom.addClass(this.fieldGroup_, 'blocklyImageField');
}
if (this.clickHandler) {
this.imageElement.style.cursor = 'pointer';
}
const element = this.getFocusableElement();
aria.setRole(element, aria.Role.IMAGE);
aria.setState(
element,
aria.State.LABEL,
this.name ? `Image ${this.name}` : 'Image',
);
}
override updateSize_() {}
// NOP
/**
* Ensure that the input value (the source URL) is a string.
*
* @param newValue The input value.
* @returns A string, or null if invalid.
*/
protected override doClassValidation_(newValue?: any): string | null {
if (typeof newValue !== 'string') {
return null;
}
return newValue;
}
/**
* Update the value of this image field, and update the displayed image.
*
* @param newValue The value to be saved. The default validator guarantees
* that this is a string.
*/
protected override doValueUpdate_(newValue: string) {
this.value_ = newValue;
if (this.imageElement) {
this.imageElement.setAttributeNS(dom.XLINK_NS, 'xlink:href', this.value_);
}
}
/**
* Get whether to flip this image in RTL
*
* @returns True if we should flip in RTL.
*/
override getFlipRtl(): boolean {
return this.flipRtl;
}
/**
* Set the alt text of this image.
*
* @param alt New alt text.
*/
setAlt(alt: string | null) {
if (alt === this.altText) {
return;
}
this.altText = alt || '';
if (this.imageElement) {
this.imageElement.setAttribute('alt', this.altText);
}
}
/**
* Check whether this field should be clickable.
*
* @returns Whether this field is clickable.
*/
isClickable(): boolean {
// Images are only clickable if they have a click handler and fulfill the
// contract to be clickable: enabled and attached to an editable block.
return super.isClickable() && !!this.clickHandler;
}
/**
* If field click is called, and click handler defined,
* call the handler.
*/
protected override showEditor_() {
if (this.clickHandler) {
this.clickHandler(this);
}
}
/**
* Set the function that is called when this image is clicked.
*
* @param func The function that is called when the image is clicked, or null
* to remove.
*/
setOnClickHandler(func: ((p1: FieldImage) => void) | null) {
this.clickHandler = func;
}
/**
* Use the `getText_` developer hook to override the field's text
* representation.
* Return the image alt text instead.
*
* @returns The image alt text.
*/
protected override getText_(): string | null {
return this.altText;
}
/**
* Construct a FieldImage from a JSON arg object,
* dereferencing any string table references.
*
* @param options A JSON object with options (src, width, height, alt, and
* flipRtl).
* @returns The new field instance.
* @nocollapse
* @internal
*/
static override fromJson(options: FieldImageFromJsonConfig): FieldImage {
if (!options.src || !options.width || !options.height) {
throw new Error(
'src, width, and height values for an image field are' +
'required. The width and height must be non-zero.',
);
}
// `this` might be a subclass of FieldImage if that class doesn't override
// the static fromJson method.
return new this(
options.src,
options.width,
options.height,
undefined,
undefined,
undefined,
options,
);
}
}
fieldRegistry.register('field_image', FieldImage);
FieldImage.prototype.DEFAULT_VALUE = '';
/**
* Config options for the image field.
*/
export interface FieldImageConfig extends FieldConfig {
flipRtl?: boolean;
alt?: string;
}
/**
* fromJson config options for the image field.
*/
export interface FieldImageFromJsonConfig extends FieldImageConfig {
src?: string;
width?: number;
height?: number;
}