OpenXR: Added a fallback modifier implementation to the hand tracking demo, if hand tracking is not available

2025-12-16 13:30:07 +01:00 · 2025-03-31 12:42:21 +11:00
parent 5557b10cfa
commit debdec8469
9 changed files with 310 additions and 32 deletions
--- a/xr/openxr_hand_tracking_demo/README.md
+++ b/xr/openxr_hand_tracking_demo/README.md
@@ -6,7 +6,9 @@ Language: GDScript

 Renderer: Compatibility

-> Note: this demo requires Godot 4.3 or later
+> [!NOTE]
+>
+> This demo requires Godot 4.3 or later

 ## Screenshots

@@ -88,7 +90,18 @@ This split is applied because:
 * positioning is always within the local space of the XROrigin3D node
 * there are many use cases where the positioning may be ignored or modified

-> Note that the trackers used for the hand tracking API are `/user/hand_tracker/left` and `/user/hand_tracker/right`.
+> [!NOTE]
+>
+> The trackers used for the hand tracking API are `/user/hand_tracker/left`
+> and `/user/hand_tracker/right`.
+
+> [!NOTE]
+>
+> There are new project settings in Godot 4.4.
+> If you upgrade this project to Godot 4.4 or later,
+> you need to open the Project Settings window,
+> go to the OpenXR settings
+> and enable the hand tracking source extensions.

 ## (Half) body Tracking API

@@ -96,11 +109,24 @@ Just an honerable mention of this, this is not part of this demo but Godot now a
 for half and full body tracking that includes hand tracking. This functionality however is only
 available on a limited number of XR runtimes.

+## Fallback skeleton modifier
+
+If the XR runtime doesn't support hand tracking, or doesn't support the controller data source,
+this demo implements a fallback modifier that animates the fingers based on trigger
+and grip input from the action map.
+The hand is now also positioned based on poses accessed through the action map.
+
+By default we use the palm pose, which is properly defined in the OpenXR action map.
+
+As support for the palm pose is optional, we fallback on the grip pose if needed.
+While positioning matches closely with the palm pose, orientation of the grip pose
+differs between XR runtimes and can cause misalignment of the hand mesh.
+
 ## Action map

 As mentioned, we're using the action map here for input however when optical hand tracking is used
-we rely on OpenXRs hand interaction profile extension. Without support for this extension this demo
-will not fully function.
+we rely on OpenXRs hand interaction profile extension.
+Without support for this extension this demo will not fully function.

 This can be solved by checking that no interaction profile has been bound to our XRController3D node,
 and performing our own gesture detection.
@@ -114,9 +140,12 @@ will soon see wide adoption, this is left out of the demo.

 We are not using the default action map and instead have created an action map specific to this use case.

-There are only two actions needed for this example:
- `pose` is used to position the XRController3D nodes and mapped to the grip pose in most cases
+There are a number of actions needed for this example:
+- `palm_pose` is used to position the XRController3D nodes and our hand fallback by default.
+- `grip_pose` is used when the palm pose is not supported.
 - `pickup` is used as the input for picking up an object, and mapped accordingly.
+- `trigger` is used purely for animating our index finger.
+- `haptic` is currently not used in the demo but defined for future use.

 The pickup logic itself is split into two components:

--- a/xr/openxr_hand_tracking_demo/assets/gltf/LeftHandHumanoid.gltf.import
+++ b/xr/openxr_hand_tracking_demo/assets/gltf/LeftHandHumanoid.gltf.import
@@ -32,8 +32,5 @@ animation/remove_immutable_tracks=true
 animation/import_rest_as_RESET=false
 import_script/path=""
 _subresources={}
-fbx/importer=0
-fbx/allow_geometry_helper_nodes=false
-fbx/embedded_image_handling=1
 gltf/naming_version=1
 gltf/embedded_image_handling=1
--- a/xr/openxr_hand_tracking_demo/assets/gltf/RightHandHumanoid.gltf.import
+++ b/xr/openxr_hand_tracking_demo/assets/gltf/RightHandHumanoid.gltf.import
@@ -32,8 +32,5 @@ animation/remove_immutable_tracks=true
 animation/import_rest_as_RESET=false
 import_script/path=""
 _subresources={}
-fbx/importer=0
-fbx/allow_geometry_helper_nodes=false
-fbx/embedded_image_handling=1
 gltf/naming_version=1
 gltf/embedded_image_handling=1
--- a/xr/openxr_hand_tracking_demo/hand_controller.gd
+++ b/xr/openxr_hand_tracking_demo/hand_controller.gd
@@ -0,0 +1,16 @@
+extends XRController3D
+
+# Check if we can use our palm pose or should fallback to our grip pose.
+
+
+# Called every frame. 'delta' is the elapsed time since the previous frame.
+func _process(delta):
+	var controller_tracker : XRControllerTracker = XRServer.get_tracker(tracker)
+	if controller_tracker:
+		var new_pose : String = "palm_pose"
+		var xr_pose : XRPose = controller_tracker.get_pose(new_pose)
+		if not xr_pose or xr_pose.tracking_confidence == XRPose.XR_TRACKING_CONFIDENCE_NONE:
+			new_pose = "grip"
+
+		if pose != new_pose:
+			pose = new_pose
--- a/xr/openxr_hand_tracking_demo/hand_info.gd
+++ b/xr/openxr_hand_tracking_demo/hand_info.gd
@@ -17,7 +17,14 @@ func _process(delta):
 		var profile = controller_tracker.profile.replace("/interaction_profiles/", "").replace("/", " ")
 		text += "\nProfile: " + profile + "\n"

-		var pose : XRPose = controller_tracker.get_pose("pose")
+		var pose : XRPose = controller_tracker.get_pose("palm_pose")
+		if pose and pose.tracking_confidence != XRPose.XR_TRACKING_CONFIDENCE_NONE:
+			text +=" - Using palm pose\n"
+		else:
+			pose = controller_tracker.get_pose("grip")
+			if pose:
+				text +=" - Using grip pose\n"
+
 		if pose:
 			if pose.tracking_confidence == XRPose.XR_TRACKING_CONFIDENCE_NONE:
 				text += "- No tracking data\n"
--- a/xr/openxr_hand_tracking_demo/hand_mesh.gd
+++ b/xr/openxr_hand_tracking_demo/hand_mesh.gd
@@ -0,0 +1,41 @@
+extends XRNode3D
+
+## Detect which tracker we should use to position our hand, we prefer
+## hand tracking as this works in unison with our hand skeleton updates.
+
+## Hand for which to get our tracking data.
+@export_enum("Left","Right") var hand : int = 0
+
+# Called every frame. 'delta' is the elapsed time since the previous frame.
+func _process(delta):
+	var new_tracker : String
+
+	# Check if our hand tracker is usable
+	new_tracker = "/user/hand_tracker/left" if hand == 0 \
+		else "/user/hand_tracker/right"
+	var hand_tracker : XRHandTracker = XRServer.get_tracker(new_tracker)
+	if hand_tracker and hand_tracker.has_tracking_data:
+		if tracker != new_tracker:
+			print("Switching to left hand tracker" if hand == 0 \
+				else "Switching to right hand tracker")
+			tracker = new_tracker
+			pose = "default"
+
+		return
+
+	# Else fallback to our controller tracker
+	new_tracker = "left_hand" if hand == 0 else "right_hand"
+	var controller_tracker : XRControllerTracker = XRServer.get_tracker(new_tracker)
+	if controller_tracker:
+		if tracker != new_tracker:
+			print("Switching to left controller tracker" if hand == 0 \
+				else "Switching to right controller tracker")
+			tracker = new_tracker
+
+		var new_pose : String = "palm_pose"
+		var xr_pose : XRPose = controller_tracker.get_pose(new_pose)
+		if not xr_pose or xr_pose.tracking_confidence == XRPose.XR_TRACKING_CONFIDENCE_NONE:
+			new_pose = "grip"
+
+		if pose != new_pose:
+			pose = new_pose
--- a/xr/openxr_hand_tracking_demo/main.tscn
+++ b/xr/openxr_hand_tracking_demo/main.tscn
@@ -1,4 +1,4 @@
-[gd_scene load_steps=14 format=3 uid="uid://br3bss6kac8pa"]
+[gd_scene load_steps=17 format=3 uid="uid://br3bss6kac8pa"]

 [ext_resource type="PackedScene" uid="uid://d22k0sp2hinew" path="res://assets/gltf/LeftHandHumanoid.gltf" id="2_3hxem"]
 [ext_resource type="Script" path="res://start_vr.gd" id="2_5rtkn"]
@@ -6,8 +6,11 @@
 [ext_resource type="PackedScene" uid="uid://byif52d1xkl3u" path="res://pickup/pickup_handler.tscn" id="3_sg1io"]
 [ext_resource type="Texture2D" uid="uid://b1waowk6l76ap" path="res://assets/images/pattern.png" id="4_3x0ea"]
 [ext_resource type="PackedScene" uid="uid://dtabh705qyufu" path="res://hand_info.tscn" id="5_wlhtu"]
+[ext_resource type="Script" path="res://hand_controller.gd" id="6_e5cto"]
 [ext_resource type="PackedScene" uid="uid://hanl00aqvu7u" path="res://objects/table.tscn" id="6_rfmma"]
 [ext_resource type="PackedScene" uid="uid://cerkxyasq8t8b" path="res://objects/box.tscn" id="7_6sqt7"]
+[ext_resource type="Script" path="res://hand_mesh.gd" id="7_yj2fr"]
+[ext_resource type="Script" path="res://xr_hand_fallback_modifier_3d.gd" id="8_gdsnk"]

 [sub_resource type="ProceduralSkyMaterial" id="ProceduralSkyMaterial_eyx45"]
 sky_horizon_color = Color(0.64625, 0.65575, 0.67075, 1)
@@ -88,8 +91,9 @@ transform = Transform3D(1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1.7, 0)
 [node name="LeftHandController" type="XRController3D" parent="XROrigin3D"]
 transform = Transform3D(1, 0, 0, 0, 1, 0, 0, 0, 1, -0.5, 1, -0.5)
 tracker = &"left_hand"
-pose = &"pose"
+pose = &"palm_pose"
 show_when_tracked = true
+script = ExtResource("6_e5cto")

 [node name="PickupHandler" parent="XROrigin3D/LeftHandController" instance=ExtResource("3_sg1io")]
 transform = Transform3D(1, 0, 0, 0, 1, 0, 0, 0, 1, 0.05, 0, 0)
@@ -98,8 +102,9 @@ pickup_action = "pickup"
 [node name="RightHandController" type="XRController3D" parent="XROrigin3D"]
 transform = Transform3D(1, 0, 0, 0, 1, 0, 0, 0, 1, 0.5, 1, -0.5)
 tracker = &"right_hand"
-pose = &"pose"
+pose = &"palm_pose"
 show_when_tracked = true
+script = ExtResource("6_e5cto")

 [node name="PickupHandler" parent="XROrigin3D/RightHandController" instance=ExtResource("3_sg1io")]
 transform = Transform3D(1, 0, 0, 0, 1, 0, 0, 0, 1, -0.05, 0, 0)
@@ -108,21 +113,64 @@ pickup_action = "pickup"
 [node name="LeftHandMesh" type="XRNode3D" parent="XROrigin3D"]
 transform = Transform3D(1, 0, 0, 0, 1, 0, 0, 0, 1, -0.5, 1, -0.5)
 tracker = &"/user/hand_tracker/left"
-show_when_tracked = true
+script = ExtResource("7_yj2fr")

 [node name="LeftHandHumanoid2" parent="XROrigin3D/LeftHandMesh" instance=ExtResource("2_3hxem")]

 [node name="XRHandModifier3D" type="XRHandModifier3D" parent="XROrigin3D/LeftHandMesh/LeftHandHumanoid2/LeftHandHumanoid/Skeleton3D" index="1"]

+[node name="XRHandFallbackModifier3D" type="SkeletonModifier3D" parent="XROrigin3D/LeftHandMesh/LeftHandHumanoid2/LeftHandHumanoid/Skeleton3D" index="2"]
+_import_path = NodePath("")
+unique_name_in_owner = false
+process_mode = 0
+process_priority = 0
+process_physics_priority = 0
+process_thread_group = 0
+physics_interpolation_mode = 0
+auto_translate_mode = 0
+editor_description = ""
+transform = Transform3D(1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0)
+rotation_edit_mode = 0
+rotation_order = 2
+top_level = false
+visible = true
+visibility_parent = NodePath("")
+active = true
+influence = 1.0
+script = ExtResource("8_gdsnk")
+grip_action = "pickup"
+
 [node name="RightHandMesh" type="XRNode3D" parent="XROrigin3D"]
 transform = Transform3D(1, 0, 0, 0, 1, 0, 0, 0, 1, 0.5, 1, -0.5)
 tracker = &"/user/hand_tracker/right"
-show_when_tracked = true
+script = ExtResource("7_yj2fr")
+hand = 1

 [node name="RightHandHumanoid2" parent="XROrigin3D/RightHandMesh" instance=ExtResource("3_oifi1")]

 [node name="XRHandModifier3D" type="XRHandModifier3D" parent="XROrigin3D/RightHandMesh/RightHandHumanoid2/RightHandHumanoid/Skeleton3D" index="1"]
 hand_tracker = &"/user/hand_tracker/right"

+[node name="XRHandFallbackModifier3D" type="SkeletonModifier3D" parent="XROrigin3D/RightHandMesh/RightHandHumanoid2/RightHandHumanoid/Skeleton3D" index="2"]
+_import_path = NodePath("")
+unique_name_in_owner = false
+process_mode = 0
+process_priority = 0
+process_physics_priority = 0
+process_thread_group = 0
+physics_interpolation_mode = 0
+auto_translate_mode = 0
+editor_description = ""
+transform = Transform3D(1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0)
+rotation_edit_mode = 0
+rotation_order = 2
+top_level = false
+visible = true
+visibility_parent = NodePath("")
+active = true
+influence = 1.0
+script = ExtResource("8_gdsnk")
+grip_action = "pickup"
+
 [editable path="XROrigin3D/LeftHandMesh/LeftHandHumanoid2"]
 [editable path="XROrigin3D/RightHandMesh/RightHandHumanoid2"]
--- a/xr/openxr_hand_tracking_demo/openxr_action_map.tres
+++ b/xr/openxr_hand_tracking_demo/openxr_action_map.tres
@@ -1,8 +1,8 @@
-[gd_resource type="OpenXRActionMap" load_steps=23 format=3 uid="uid://dydgx5ktpcmdl"]
+[gd_resource type="OpenXRActionMap" load_steps=38 format=3 uid="uid://dydgx5ktpcmdl"]

 [sub_resource type="OpenXRAction" id="OpenXRAction_ywi2s"]
-resource_name = "pose"
-localized_name = "Pose"
+resource_name = "palm_pose"
+localized_name = "Palm pose"
 action_type = 3
 toplevel_paths = PackedStringArray("/user/hand/left", "/user/hand/right")

@@ -17,14 +17,25 @@ resource_name = "pickup"
 localized_name = "Pickup"
 toplevel_paths = PackedStringArray("/user/hand/left", "/user/hand/right")

+[sub_resource type="OpenXRAction" id="OpenXRAction_ibecu"]
+resource_name = "trigger"
+localized_name = "Trigger"
+toplevel_paths = PackedStringArray("/user/hand/left", "/user/hand/right")
+
+[sub_resource type="OpenXRAction" id="OpenXRAction_nhsps"]
+resource_name = "grip_pose"
+localized_name = "Grip pose"
+action_type = 3
+toplevel_paths = PackedStringArray("/user/hand/left", "/user/hand/right")
+
 [sub_resource type="OpenXRActionSet" id="OpenXRActionSet_c2hwm"]
 resource_name = "godot"
 localized_name = "Godot action set"
-actions = [SubResource("OpenXRAction_ywi2s"), SubResource("OpenXRAction_vayrd"), SubResource("OpenXRAction_h3dsb")]
+actions = [SubResource("OpenXRAction_ywi2s"), SubResource("OpenXRAction_vayrd"), SubResource("OpenXRAction_h3dsb"), SubResource("OpenXRAction_ibecu"), SubResource("OpenXRAction_nhsps")]

 [sub_resource type="OpenXRIPBinding" id="OpenXRIPBinding_r6sxc"]
 action = SubResource("OpenXRAction_ywi2s")
-paths = PackedStringArray("/user/hand/left/input/grip/pose", "/user/hand/right/input/grip/pose")
+paths = PackedStringArray("/user/hand/left/input/palm_ext/pose", "/user/hand/right/input/palm_ext/pose")

 [sub_resource type="OpenXRIPBinding" id="OpenXRIPBinding_j0h30"]
 action = SubResource("OpenXRAction_vayrd")
@@ -34,9 +45,13 @@ paths = PackedStringArray("/user/hand/left/output/haptic", "/user/hand/right/out
 action = SubResource("OpenXRAction_h3dsb")
 paths = PackedStringArray("/user/hand/left/input/select/click", "/user/hand/right/input/select/click")

+[sub_resource type="OpenXRIPBinding" id="OpenXRIPBinding_l06qe"]
+action = SubResource("OpenXRAction_nhsps")
+paths = PackedStringArray("/user/hand/left/input/grip/pose", "/user/hand/right/input/grip/pose")
+
 [sub_resource type="OpenXRInteractionProfile" id="OpenXRInteractionProfile_643al"]
 interaction_profile_path = "/interaction_profiles/khr/simple_controller"
-bindings = [SubResource("OpenXRIPBinding_r6sxc"), SubResource("OpenXRIPBinding_j0h30"), SubResource("OpenXRIPBinding_govyu")]
+bindings = [SubResource("OpenXRIPBinding_r6sxc"), SubResource("OpenXRIPBinding_j0h30"), SubResource("OpenXRIPBinding_govyu"), SubResource("OpenXRIPBinding_l06qe")]

 [sub_resource type="OpenXRIPBinding" id="OpenXRIPBinding_fd646"]
 action = SubResource("OpenXRAction_h3dsb")
@@ -44,15 +59,19 @@ paths = PackedStringArray("/user/hand/left/input/grasp_ext/value", "/user/hand/r

 [sub_resource type="OpenXRIPBinding" id="OpenXRIPBinding_wkfos"]
 action = SubResource("OpenXRAction_ywi2s")
+paths = PackedStringArray("/user/hand/left/input/palm_ext/pose", "/user/hand/right/input/palm_ext/pose")
+
+[sub_resource type="OpenXRIPBinding" id="OpenXRIPBinding_qhuhw"]
+action = SubResource("OpenXRAction_nhsps")
 paths = PackedStringArray("/user/hand/left/input/grip/pose", "/user/hand/right/input/grip/pose")

 [sub_resource type="OpenXRInteractionProfile" id="OpenXRInteractionProfile_nhhi0"]
 interaction_profile_path = "/interaction_profiles/ext/hand_interaction_ext"
-bindings = [SubResource("OpenXRIPBinding_fd646"), SubResource("OpenXRIPBinding_wkfos")]
+bindings = [SubResource("OpenXRIPBinding_fd646"), SubResource("OpenXRIPBinding_wkfos"), SubResource("OpenXRIPBinding_qhuhw")]

 [sub_resource type="OpenXRIPBinding" id="OpenXRIPBinding_gv55f"]
 action = SubResource("OpenXRAction_ywi2s")
-paths = PackedStringArray("/user/hand/left/input/grip/pose", "/user/hand/right/input/grip/pose")
+paths = PackedStringArray("/user/hand/left/input/palm_ext/pose", "/user/hand/right/input/palm_ext/pose")

 [sub_resource type="OpenXRIPBinding" id="OpenXRIPBinding_mp0xr"]
 action = SubResource("OpenXRAction_h3dsb")
@@ -62,13 +81,21 @@ paths = PackedStringArray("/user/hand/left/input/squeeze/value", "/user/hand/rig
 action = SubResource("OpenXRAction_vayrd")
 paths = PackedStringArray("/user/hand/left/output/haptic", "/user/hand/right/output/haptic")

+[sub_resource type="OpenXRIPBinding" id="OpenXRIPBinding_ma8yg"]
+action = SubResource("OpenXRAction_ibecu")
+paths = PackedStringArray("/user/hand/left/input/trigger/value", "/user/hand/right/input/trigger/value")
+
+[sub_resource type="OpenXRIPBinding" id="OpenXRIPBinding_ov3kx"]
+action = SubResource("OpenXRAction_nhsps")
+paths = PackedStringArray("/user/hand/left/input/grip/pose", "/user/hand/right/input/grip/pose")
+
 [sub_resource type="OpenXRInteractionProfile" id="OpenXRInteractionProfile_wrsh4"]
 interaction_profile_path = "/interaction_profiles/oculus/touch_controller"
-bindings = [SubResource("OpenXRIPBinding_gv55f"), SubResource("OpenXRIPBinding_mp0xr"), SubResource("OpenXRIPBinding_86te5")]
+bindings = [SubResource("OpenXRIPBinding_gv55f"), SubResource("OpenXRIPBinding_mp0xr"), SubResource("OpenXRIPBinding_86te5"), SubResource("OpenXRIPBinding_ma8yg"), SubResource("OpenXRIPBinding_ov3kx")]

 [sub_resource type="OpenXRIPBinding" id="OpenXRIPBinding_n476d"]
 action = SubResource("OpenXRAction_ywi2s")
-paths = PackedStringArray("/user/hand/left/input/grip/pose", "/user/hand/right/input/grip/pose")
+paths = PackedStringArray("/user/hand/left/input/palm_ext/pose", "/user/hand/right/input/palm_ext/pose")

 [sub_resource type="OpenXRIPBinding" id="OpenXRIPBinding_dh54r"]
 action = SubResource("OpenXRAction_h3dsb")
@@ -78,22 +105,58 @@ paths = PackedStringArray("/user/hand/left/input/squeeze/value", "/user/hand/rig
 action = SubResource("OpenXRAction_vayrd")
 paths = PackedStringArray("/user/hand/left/output/haptic", "/user/hand/right/output/haptic")

+[sub_resource type="OpenXRIPBinding" id="OpenXRIPBinding_738eg"]
+action = SubResource("OpenXRAction_ibecu")
+paths = PackedStringArray("/user/hand/left/input/trigger/value", "/user/hand/right/input/trigger/value")
+
+[sub_resource type="OpenXRIPBinding" id="OpenXRIPBinding_2p0gb"]
+action = SubResource("OpenXRAction_nhsps")
+paths = PackedStringArray("/user/hand/left/input/grip/pose", "/user/hand/right/input/grip/pose")
+
 [sub_resource type="OpenXRInteractionProfile" id="OpenXRInteractionProfile_lah6t"]
 interaction_profile_path = "/interaction_profiles/valve/index_controller"
-bindings = [SubResource("OpenXRIPBinding_n476d"), SubResource("OpenXRIPBinding_dh54r"), SubResource("OpenXRIPBinding_tc2nk")]
+bindings = [SubResource("OpenXRIPBinding_n476d"), SubResource("OpenXRIPBinding_dh54r"), SubResource("OpenXRIPBinding_tc2nk"), SubResource("OpenXRIPBinding_738eg"), SubResource("OpenXRIPBinding_2p0gb")]

 [sub_resource type="OpenXRIPBinding" id="OpenXRIPBinding_vhdol"]
 action = SubResource("OpenXRAction_ywi2s")
-paths = PackedStringArray("/user/hand/left/input/grip/pose", "/user/hand/right/input/grip/pose")
+paths = PackedStringArray("/user/hand/left/input/palm_ext/pose", "/user/hand/right/input/palm_ext/pose")

 [sub_resource type="OpenXRIPBinding" id="OpenXRIPBinding_pwv0l"]
 action = SubResource("OpenXRAction_h3dsb")
 paths = PackedStringArray("/user/hand/left/input/squeeze/value", "/user/hand/right/input/squeeze/value")

+[sub_resource type="OpenXRIPBinding" id="OpenXRIPBinding_rg58l"]
+action = SubResource("OpenXRAction_nhsps")
+paths = PackedStringArray("/user/hand/left/input/grip/pose", "/user/hand/right/input/grip/pose")
+
 [sub_resource type="OpenXRInteractionProfile" id="OpenXRInteractionProfile_51rtw"]
 interaction_profile_path = "/interaction_profiles/microsoft/hand_interaction"
-bindings = [SubResource("OpenXRIPBinding_vhdol"), SubResource("OpenXRIPBinding_pwv0l")]
+bindings = [SubResource("OpenXRIPBinding_vhdol"), SubResource("OpenXRIPBinding_pwv0l"), SubResource("OpenXRIPBinding_rg58l")]
+
+[sub_resource type="OpenXRIPBinding" id="OpenXRIPBinding_plc23"]
+action = SubResource("OpenXRAction_ywi2s")
+paths = PackedStringArray("/user/hand/right/input/palm_ext/pose", "/user/hand/left/input/palm_ext/pose")
+
+[sub_resource type="OpenXRIPBinding" id="OpenXRIPBinding_eus86"]
+action = SubResource("OpenXRAction_h3dsb")
+paths = PackedStringArray("/user/hand/left/input/squeeze/value", "/user/hand/right/input/squeeze/value")
+
+[sub_resource type="OpenXRIPBinding" id="OpenXRIPBinding_75kc0"]
+action = SubResource("OpenXRAction_vayrd")
+paths = PackedStringArray("/user/hand/left/output/haptic", "/user/hand/right/output/haptic")
+
+[sub_resource type="OpenXRIPBinding" id="OpenXRIPBinding_dkyu7"]
+action = SubResource("OpenXRAction_ibecu")
+paths = PackedStringArray("/user/hand/left/input/trigger/value", "/user/hand/right/input/trigger/value")
+
+[sub_resource type="OpenXRIPBinding" id="OpenXRIPBinding_65s55"]
+action = SubResource("OpenXRAction_nhsps")
+paths = PackedStringArray("/user/hand/left/input/grip/pose", "/user/hand/right/input/grip/pose")
+
+[sub_resource type="OpenXRInteractionProfile" id="OpenXRInteractionProfile_f3qcx"]
+interaction_profile_path = "/interaction_profiles/bytedance/pico4_controller"
+bindings = [SubResource("OpenXRIPBinding_plc23"), SubResource("OpenXRIPBinding_eus86"), SubResource("OpenXRIPBinding_75kc0"), SubResource("OpenXRIPBinding_dkyu7"), SubResource("OpenXRIPBinding_65s55")]

 [resource]
 action_sets = [SubResource("OpenXRActionSet_c2hwm")]
-interaction_profiles = [SubResource("OpenXRInteractionProfile_643al"), SubResource("OpenXRInteractionProfile_nhhi0"), SubResource("OpenXRInteractionProfile_wrsh4"), SubResource("OpenXRInteractionProfile_lah6t"), SubResource("OpenXRInteractionProfile_51rtw")]
+interaction_profiles = [SubResource("OpenXRInteractionProfile_643al"), SubResource("OpenXRInteractionProfile_nhhi0"), SubResource("OpenXRInteractionProfile_wrsh4"), SubResource("OpenXRInteractionProfile_lah6t"), SubResource("OpenXRInteractionProfile_51rtw"), SubResource("OpenXRInteractionProfile_f3qcx")]
--- a/xr/openxr_hand_tracking_demo/xr_hand_fallback_modifier_3d.gd
+++ b/xr/openxr_hand_tracking_demo/xr_hand_fallback_modifier_3d.gd
@@ -0,0 +1,80 @@
+class_name XRHandFallbackModifier3D
+extends SkeletonModifier3D
+
+## This node implements a fallback if the hand tracking API is not available
+## or if one of the data sources is not supported by the XR runtime.
+## It uses trigger and grip inputs from the normal controller tracker to
+## animate the index finger and bottom 3 fingers respectively.
+
+## Action to use for animating index finger.
+@export var trigger_action : String = "trigger"
+
+## Action to use for animating bottom 3 fingers.
+@export var grip_action : String = "grip"
+
+# Called by our skeleton logic when this modifier needs to apply its modifications.
+func _process_modification() -> void:
+	# Get our skeleton.
+	var skeleton: Skeleton3D = get_skeleton()
+	if !skeleton:
+		return
+
+	# Find our parent controller
+	var parent = get_parent()
+	while parent and not parent is XRNode3D:
+		parent = parent.get_parent()
+	if !parent:
+		return
+
+	# Check if we have an active hand tracker,
+	# if so, we don't need our fallback!
+	var xr_parent : XRNode3D = parent
+	if not xr_parent.tracker in [ "left_hand", "right_hand" ]:
+		return
+
+	var trigger : float = 0.0
+	var grip : float = 0.0
+
+	# Check our tracker for trigger and grip values
+	var tracker : XRControllerTracker = XRServer.get_tracker(xr_parent.tracker)
+	if tracker:
+		var trigger_value : Variant = tracker.get_input(trigger_action)
+		if trigger_value:
+			trigger = trigger_value
+
+		var grip_value : Variant = tracker.get_input(grip_action)
+		if grip_value:
+			grip = grip_value
+
+	# Now position bones
+	var bone_count = skeleton.get_bone_count()
+	for i in bone_count:
+		var t : Transform3D = skeleton.get_bone_rest(i)
+
+		# We animate based on bone_name.
+		# For now just hardcoded values.
+		# Note that we position all bones in case we need to reset some.
+		var bone_name = skeleton.get_bone_name(i)
+		if bone_name == "LeftHand":
+			# Offset to center our palm, this requires the use of the palm pose!
+			t.origin += Vector3(-0.015, 0.0, 0.04)
+		elif bone_name == "RightHand":
+			# Offset to center our palm, this requires the use of the palm pose!
+			t.origin += Vector3(0.015, 0.0, 0.04)
+		elif bone_name == "LeftIndexDistal" or bone_name == "LeftIndexIntermediate" \
+			or bone_name == "RightIndexDistal" or bone_name == "RightIndexIntermediate":
+			var r : Transform3D
+			t = t * r.rotated(Vector3(1.0, 0.0, 0.0), deg_to_rad(45.0) * trigger)
+		elif bone_name == "LeftIndexProximal" or bone_name == "RightIndexProximal":
+			var r : Transform3D
+			t = t * r.rotated(Vector3(1.0, 0.0, 0.0), deg_to_rad(20.0) * trigger)
+		elif bone_name == "LeftMiddleDistal" or bone_name == "LeftMiddleIntermediate" or bone_name == "LeftMiddleProximal" \
+			or bone_name == "RightMiddleDistal" or bone_name == "RightMiddleIntermediate" or bone_name == "RightMiddleProximal" \
+			or bone_name == "LeftRingDistal" or bone_name == "LeftRingIntermediate" or bone_name == "LeftRingProximal" \
+			or bone_name == "RightRingDistal" or bone_name == "RightRingIntermediate" or bone_name == "RightRingProximal" \
+			or bone_name == "LeftLittleDistal" or bone_name == "LeftLittleIntermediate" or bone_name == "LeftLittleProximal" \
+			or bone_name == "RightLittleDistal" or bone_name == "RightLittleIntermediate" or bone_name == "RightLittleProximal":
+			var r : Transform3D
+			t = t * r.rotated(Vector3(1.0, 0.0, 0.0), deg_to_rad(90.0) * grip)
+
+		skeleton.set_bone_pose(i, t)