3D Gaussian Splatting

3D Gaussian Splatting is a relatively new technique for capturing and rendering photorealistic volumetric point clouds. Since the technique relies on photogrammetry, it is very quick, cheap and easy to generate high-quality rendered scenes.

Working with Gaussian Splats

Creating Splats

Splats are stored in binary PLY files. There are a number of ways you can generate your own splats.

1. Use a Capture App

Use a splat capture app such as Polycam or Luma. With Luma, export to 'Gaussian Splat' and extract the PLY file from the downloaded ZIP file. With Polycam, export to 'splat PLY'.

2. Use the Inria Tools

Inria's SIGGRAPH 2023 paper links to a toolset on GitHub that you can download and use to generate splats from a set of images. These tools are more complicated to set up and use but they do offer fine-grained control over the splat generation process.

Editing Splats

Captured splats normally need to be edited to some degree. The generation process can often create splats in the wrong location (sometimes referred to as 'floaters') so it's useful to be able to remove these stray splats. It may also be desirable to isolate a particular object (such as a person) in a captured splat and completely remove the background.

PlayCanvas provides a powerful 3D Gaussian Splat editor called SuperSplat. SuperSplat is open-sourced under an MIT license on GitHub.

Importing Splats

To import your PLY splat file:

1. Drag it into the Editor's Asset Panel.
2. Drag the created GSplat asset into the Viewport. This will auto-create an Entity in the Hierarchy with a GSplat component with your GSplat asset assigned to it.

パフォーマンス

Rendering splats can be expensive on both the CPU and GPU. Here are some strategies to achieve good performance:

• Be mindful of the number of Gaussians in your scene since every Gaussian is sorted on camera depth every frame. You can check the number contained within a particular GSplat asset by using the Inspector. Use SuperSplat to trim unwanted Gaussians from your PLY files.
• Disable Anti-Alias in the Scene Settings. Anti-aliasing is GPU intensive and offers little benefit for rendering splats.
• Disable Device Pixel Ratio in the Scene Settings. This will reduce the overall number of pixels that the GPU has to process.

制限事項

There are some limitations to keep in mind when working with splats:

1. Fog has no effect.
2. Dynamic lights have no effect.
3. Image Based Lighting has no effect.
4. Splats do not cast shadows.