Hand Tracking

If the platform supports WebXR Hand Input, then an input source can have associated hand data, which is exposed as an XrHand, and its data in the form of XrFingers and XrJoints for an application developer to use, such as wrist, fingers, joints, tips and events for detecting when hands lose/restore tracking.

Model

Creating a basic hand model:

const joints = [];
const hand = inputSource.hand;

if (hand) {
    for (let i = 0; i < hand.joints.length; i++) {
        const entity = new pc.Entity();
        entity.joint = hand.joints[i];
        entity.addComponent('render', { type: 'box' });
        parent.addChild(entity);
        joints.push(entity);
    }
}

Updates

Every frame, joint data can change position, rotation, and other details.

for (let i = 0; i < joints.length; i++) {
    const entity = joints[i];
    const joint = entity.joint;
    const radius = joint.radius * 2;
    entity.setLocalScale(radius, radius, radius);
    entity.setPosition(joint.getPosition());
    entity.setRotation(joint.getRotation());
}

Tracking

Hand tracking is subject to the reliability and sophistication of the underlying system. There might be cases when tracking is not possible due to obstructions between cameras and hands, or when hands interlock in a complex way. While Computer Vision techniques are improving, when designing content with hands as an input source, their shortcomings should be taken into consideration.

Skinning

A skinned mesh for a hand can used. You can check out this project as an example: