API Reference

Class List

XrDepthSensing

Extends: EventHandler

Depth Sensing provides depth information which is reconstructed using the underlying AR system. It provides the ability to query depth values (CPU path) or access a depth texture (GPU path). Depth information can be used (not limited to) for reconstructing real world geometry, virtual object placement, occlusion of virtual objects by real world geometry and more.

// CPU path
var depthSensing = app.xr.depthSensing;
if (depthSensing.available) {
    // get depth in the middle of the screen, value is in meters
    var depth = depthSensing.getDepth(depthSensing.width / 2, depthSensing.height / 2);
}
// GPU path, attaching texture to material
material.diffuseMap = depthSensing.texture;
material.setParameter('matrix_depth_uv', depthSensing.uvMatrix.data);
material.setParameter('depth_raw_to_meters', depthSensing.rawValueToMeters);
material.update();

// update UV transformation matrix on depth texture resize
depthSensing.on('resize', function () {
    material.setParameter('matrix_depth_uv', depthSensing.uvMatrix.data);
    material.setParameter('depth_raw_to_meters', depthSensing.rawValueToMeters);
});
// GLSL shader to unpack depth texture
varying vec2 vUv0;

uniform sampler2D texture_depthSensingMap;
uniform mat4 matrix_depth_uv;
uniform float depth_raw_to_meters;

void main(void) {
    // transform UVs using depth matrix
    vec2 texCoord = (matrix_depth_uv * vec4(vUv0.xy, 0.0, 1.0)).xy;

    // get luminance alpha components from depth texture
    vec2 packedDepth = texture2D(texture_depthSensingMap, texCoord).ra;

    // unpack into single value in millimeters
    float depth = dot(packedDepth, vec2(255.0, 256.0 * 255.0)) * depth_raw_to_meters; // m

    // normalize: 0m to 8m distance
    depth = min(depth / 8.0, 1.0); // 0..1 = 0..8

    // paint scene from black to white based on distance
    gl_FragColor = vec4(depth, depth, depth, 1.0);
}

Summary

Properties

available

True if depth sensing information is available.

height

Height of depth texture or 0 if not available.

rawValueToMeters

Multiply this coefficient number by raw depth value to get depth in meters.

supported

True if Depth Sensing is supported.

texture

Texture that contains packed depth information.

uvMatrix

4x4 matrix that should be used to transform depth texture UVs to normalized UVs in a shader.

width

Width of depth texture or 0 if not available.

Methods

getDepth

Get depth value from depth information in meters.

Events

available

Fired when depth sensing data becomes available.

resize

Fired when the depth sensing texture been resized.

unavailable

Fired when depth sensing data becomes unavailable.

Inherited

Methods

fire

Fire an event, all additional arguments are passed on to the event listener.

hasEvent

Test if there are any handlers bound to an event name.

off

Detach an event handler from an event.

on

Attach an event handler to an event.

once

Attach an event handler to an event.

Details

Constructor

XrDepthSensing(manager)

Depth Sensing provides depth information which is reconstructed using the underlying AR system. It provides the ability to query depth values (CPU path) or access a depth texture (GPU path). Depth information can be used (not limited to) for reconstructing real world geometry, virtual object placement, occlusion of virtual objects by real world geometry and more.

// CPU path
var depthSensing = app.xr.depthSensing;
if (depthSensing.available) {
    // get depth in the middle of the screen, value is in meters
    var depth = depthSensing.getDepth(depthSensing.width / 2, depthSensing.height / 2);
}
// GPU path, attaching texture to material
material.diffuseMap = depthSensing.texture;
material.setParameter('matrix_depth_uv', depthSensing.uvMatrix.data);
material.setParameter('depth_raw_to_meters', depthSensing.rawValueToMeters);
material.update();

// update UV transformation matrix on depth texture resize
depthSensing.on('resize', function () {
    material.setParameter('matrix_depth_uv', depthSensing.uvMatrix.data);
    material.setParameter('depth_raw_to_meters', depthSensing.rawValueToMeters);
});
// GLSL shader to unpack depth texture
varying vec2 vUv0;

uniform sampler2D texture_depthSensingMap;
uniform mat4 matrix_depth_uv;
uniform float depth_raw_to_meters;

void main(void) {
    // transform UVs using depth matrix
    vec2 texCoord = (matrix_depth_uv * vec4(vUv0.xy, 0.0, 1.0)).xy;

    // get luminance alpha components from depth texture
    vec2 packedDepth = texture2D(texture_depthSensingMap, texCoord).ra;

    // unpack into single value in millimeters
    float depth = dot(packedDepth, vec2(255.0, 256.0 * 255.0)) * depth_raw_to_meters; // m

    // normalize: 0m to 8m distance
    depth = min(depth / 8.0, 1.0); // 0..1 = 0..8

    // paint scene from black to white based on distance
    gl_FragColor = vec4(depth, depth, depth, 1.0);
}

Parameters

managerXrManager

WebXR Manager.

Properties

booleanavailable

True if depth sensing information is available.

if (app.xr.depthSensing.available) {
    var depth = app.xr.depthSensing.getDepth(x, y);
}
numberheight

Height of depth texture or 0 if not available.

numberrawValueToMeters

Multiply this coefficient number by raw depth value to get depth in meters.

material.setParameter('depth_raw_to_meters', depthSensing.rawValueToMeters);
booleansupported

True if Depth Sensing is supported.

Texturetexture

Texture that contains packed depth information. The format of this texture is PIXELFORMAT_L8_A8. It is UV transformed based on the underlying AR system which can be normalized using XrDepthSensing#uvMatrix.

material.diffuseMap = depthSensing.texture;
// GLSL shader to unpack depth texture
varying vec2 vUv0;

uniform sampler2D texture_depthSensingMap;
uniform mat4 matrix_depth_uv;
uniform float depth_raw_to_meters;

void main(void) {
    // transform UVs using depth matrix
    vec2 texCoord = (matrix_depth_uv * vec4(vUv0.xy, 0.0, 1.0)).xy;

    // get luminance alpha components from depth texture
    vec2 packedDepth = texture2D(texture_depthSensingMap, texCoord).ra;

    // unpack into single value in millimeters
    float depth = dot(packedDepth, vec2(255.0, 256.0 * 255.0)) * depth_raw_to_meters; // m

    // normalize: 0m to 8m distance
    depth = min(depth / 8.0, 1.0); // 0..1 = 0m..8m

    // paint scene from black to white based on distance
    gl_FragColor = vec4(depth, depth, depth, 1.0);
}
Mat4uvMatrix

4x4 matrix that should be used to transform depth texture UVs to normalized UVs in a shader. It is updated when the depth texture is resized. Refer to XrDepthSensing#resize.

material.setParameter('matrix_depth_uv', depthSensing.uvMatrix.data);
numberwidth

Width of depth texture or 0 if not available.

Methods

getDepth(u, v)

Get depth value from depth information in meters. UV is in range of 0..1, with origin in top-left corner of a texture.

var depth = app.xr.depthSensing.getDepth(u, v);
if (depth !== null) {
    // depth in meters
}

Parameters

unumber

U coordinate of pixel in depth texture, which is in range from 0.0 to 1.0 (left to right).

vnumber

V coordinate of pixel in depth texture, which is in range from 0.0 to 1.0 (top to bottom).

Returns

number, null

Depth in meters or null if depth information is currently not available.

Events

available

Fired when depth sensing data becomes available.

resize

Fired when the depth sensing texture been resized. The XrDepthSensing#uvMatrix needs to be updated for relevant shaders.

depthSensing.on('resize', function () {
    material.setParameter('matrix_depth_uv', depthSensing.uvMatrix);
});

Parameters

widthnumber

The new width of the depth texture in pixels.

heightnumber

The new height of the depth texture in pixels.

unavailable

Fired when depth sensing data becomes unavailable.

Inherited

Methods

fire(name, [arg1], [arg2], [arg3], [arg4], [arg5], [arg6], [arg7], [arg8])

Fire an event, all additional arguments are passed on to the event listener.

obj.fire('test', 'This is the message');

Parameters

nameobject

Name of event to fire.

arg1*

First argument that is passed to the event handler.

arg2*

Second argument that is passed to the event handler.

arg3*

Third argument that is passed to the event handler.

arg4*

Fourth argument that is passed to the event handler.

arg5*

Fifth argument that is passed to the event handler.

arg6*

Sixth argument that is passed to the event handler.

arg7*

Seventh argument that is passed to the event handler.

arg8*

Eighth argument that is passed to the event handler.

Returns

EventHandler

Self for chaining.

hasEvent(name)

Test if there are any handlers bound to an event name.

obj.on('test', function () { }); // bind an event to 'test'
obj.hasEvent('test'); // returns true
obj.hasEvent('hello'); // returns false

Parameters

namestring

The name of the event to test.

Returns

boolean

True if the object has handlers bound to the specified event name.

off([name], [callback], [scope])

Detach an event handler from an event. If callback is not provided then all callbacks are unbound from the event, if scope is not provided then all events with the callback will be unbound.

var handler = function () {
};
obj.on('test', handler);

obj.off(); // Removes all events
obj.off('test'); // Removes all events called 'test'
obj.off('test', handler); // Removes all handler functions, called 'test'
obj.off('test', handler, this); // Removes all hander functions, called 'test' with scope this

Parameters

namestring

Name of the event to unbind.

callbackcallbacks.HandleEvent

Function to be unbound.

scopeobject

Scope that was used as the this when the event is fired.

Returns

EventHandler

Self for chaining.

on(name, callback, [scope])

Attach an event handler to an event.

obj.on('test', function (a, b) {
    console.log(a + b);
});
obj.fire('test', 1, 2); // prints 3 to the console

Parameters

namestring

Name of the event to bind the callback to.

callbackcallbacks.HandleEvent

Function that is called when event is fired. Note the callback is limited to 8 arguments.

scopeobject

Object to use as 'this' when the event is fired, defaults to current this.

Returns

EventHandler

Self for chaining.

once(name, callback, [scope])

Attach an event handler to an event. This handler will be removed after being fired once.

obj.once('test', function (a, b) {
    console.log(a + b);
});
obj.fire('test', 1, 2); // prints 3 to the console
obj.fire('test', 1, 2); // not going to get handled

Parameters

namestring

Name of the event to bind the callback to.

callbackcallbacks.HandleEvent

Function that is called when event is fired. Note the callback is limited to 8 arguments.

scopeobject

Object to use as 'this' when the event is fired, defaults to current this.

Returns

EventHandler

Self for chaining.