ThreePipe

A new way to work with three.js, 3D models and rendering on the web.

ThreePipe — Github — Examples — API Reference — WebGi

ThreePipe is a 3D framework built on top of three.js in TypeScript with a focus on rendering quality, modularity, and extensibility.

Key features include:

• Simple, intuitive API for creating 3D model viewers/configurators/editors on web pages, with many built-in presets for common workflows and use-cases.
• Companion editor to create, edit and configure 3D scenes in the browser.
• Modular architecture that allows you to easily extend the viewer, scene objects, materials, shaders, rendering, post-processing and serialization with custom functionality.
• Plugin system along with a rich library of built-in plugins that allows you to easily add new features to the viewer.
• uiconfig compatibility to automatically generate configuration UIs in the browser.
• Modular rendering pipeline with built-in deferred rendering, post-processing, RGBM HDR rendering, etc.
• Material extension framework to modify/inject/build custom shader code into existing materials at runtime from plugins.
• Extendable asset import, export and management pipeline with built-in support for gltf, glb, obj+mtl, fbx, materials(pmat/bmat), json, zip, png, jpeg, svg, webp, ktx2, ply, 3dm and many more.
• Automatic serialization of all viewer and plugin settings in GLB(with custom extensions) and JSON formats.
• Automatic disposal of all three.js resources with built-in reference management.

Examples

Code samples and demos covering various usecases and test are present in the examples folder.

Try them: https://threepipe.org/examples/

View the source code by pressing the code button on the top left of the example page.

To make changes and run the example, click on the CodePen button on the top right of the source code.

Table of Contents

• ThreePipe
  • Examples
  • Table of Contents
  • Getting Started
    • HTML/JS Quickstart (CDN)
    • React
    • Vue.js
    • Svelte
    • NPM/YARN Package
      • Installation
      • Loading a 3D Model
  • License
  • Status
  • Documentation (API Reference)
  • WebGi
  • Contributing
• Features
  • File Formats
  • Loading files
    • 3D models
    • Materials
    • Images/Textures
    • zip files
    • txt, json files
    • Data URLs
    • Local files, File and Blob
    • Background, Environment maps
    • SVG strings
  • Exporting files
    • Exporting 3D models
    • Exporting Materials
    • Exporting Canvas Images
    • Exporting Images/Textures
    • Exporting Render Targets
  • Render Pipeline
  • Material Extension
  • UI Configuration
  • Serialization
  • Plugin System
• Viewer API
  • ThreeViewer
  • RenderManager
  • RootScene
  • ICamera
  • AssetManager
    • AssetImporter
    • AssetExporter
    • MaterialManager
  • Other classes and interfaces
• Plugins
  • TonemapPlugin - Add tonemap to the final screen pass
  • DropzonePlugin - Drag and drop local files to import and load
  • ProgressivePlugin - Post-render pass to blend the last frame with the current frame
  • DepthBufferPlugin - Pre-rendering of depth buffer
  • NormalBufferPlugin - Pre-rendering of normal buffer
  • GBufferPlugin - Pre-rendering of depth-normal and flags buffers in a single pass
  • SSAOPlugin - Add SSAO(Screen Space Ambient Occlusion) for physical materials.
  • CanvasSnapshotPlugin - Add support for taking snapshots of the canvas
  • PickingPlugin - Adds support for selecting objects in the viewer with user interactions and selection widgets
  • TransformControlsPlugin - Adds support for moving, rotating and scaling objects in the viewer with interactive widgets
  • ContactShadowGroundPlugin - Adds a ground plane at runtime with contact shadows
  • GLTFAnimationPlugin - Add support for playing and seeking gltf animations
  • PopmotionPlugin - Integrates with popmotion.io library for animation/tweening
  • CameraViewPlugin - Add support for saving, loading, animating, looping between camera views
  • RenderTargetPreviewPlugin - Preview any render target in a UI panel over the canvas
  • GeometryUVPreviewPlugin - Preview UVs of any geometry in a UI panel over the canvas
  • FrameFadePlugin - Post-render pass to smoothly fade to a new rendered frame over time
  • VignettePlugin - Add Vignette effect by patching the final screen pass
  • ChromaticAberrationPlugin - Add Chromatic Aberration effect by patching the final screen pass
  • FilmicGrainPlugin - Add Filmic Grain effect by patching the final screen pass
  • NoiseBumpMaterialPlugin - Sparkle Bump/Noise Bump material extension for PhysicalMaterial
  • CustomBumpMapPlugin - Custom Bump Map material extension for PhysicalMaterial
  • ClearcoatTintPlugin - Clearcoat Tint material extension for PhysicalMaterial
  • FragmentClippingExtensionPlugin - Fragment/SDF Clipping material extension for PhysicalMaterial
  • HDRiGroundPlugin - Add support for ground projected hdri/skybox to the webgl background shader.
  • VirtualCamerasPlugin - Add support for rendering virtual cameras before the main one every frame.
  • EditorViewWidgetPlugin - Adds an interactive ViewHelper/AxisHelper that syncs with the main camera.
  • Object3DWidgetsPlugin - Automatically create light and camera helpers/gizmos when they are added to the scene.
  • Object3DGeneratorPlugin - Provides UI and API to create scene objects like lights, cameras, meshes, etc.
  • DeviceOrientationControlsPlugin - Adds a controlsMode to the mainCamera for device orientation controls(gyroscope rotation control).
  • PointerLockControlsPlugin - Adds a controlsMode to the mainCamera for pointer lock controls.
  • ThreeFirstPersonControlsPlugin - Adds a controlsMode to the mainCamera for first person controls from threejs.
  • Rhino3dmLoadPlugin - Add support for loading .3dm files
  • PLYLoadPlugin - Add support for loading .ply files
  • STLLoadPlugin - Add support for loading .stl files
  • KTX2LoadPlugin - Add support for loading .ktx2 files
  • KTXLoadPlugin - Add support for loading .ktx files
  • SimplifyModifierPlugin - Boilerplate for plugin to simplify geometries
  • MeshOptSimplifyModifierPlugin - Simplify geometries using meshoptimizer library
• Packages
  • @threepipe/plugin-tweakpane Tweakpane UI Plugin
  • @threepipe/plugin-blueprintjs BlueprintJs UI Plugin
  • @threepipe/plugin-tweakpane-editor - Tweakpane Editor Plugin
  • @threepipe/plugins-extra-importers - Plugin for loading more file types supported by loaders in three.js
  • @threepipe/plugin-blend-importer - Blender to add support for loading .blend file
  • @threepipe/plugin-geometry-generator - Generate parametric geometry types that can be re-generated from UI/API.
  • @threepipe/plugin-gaussian-splatting - Gaussian Splatting plugin for loading and rendering splat files

Getting Started

HTML/JS Quickstart (CDN)

<canvas id="three-canvas" style="width: 800px; height: 600px;"></canvas>
<script type="module">
  import {ThreeViewer, DepthBufferPlugin} from 'https://threepipe.org/dist/index.mjs'

  const viewer = new ThreeViewer({canvas: document.getElementById('three-canvas')})

  // Add some plugins 
  viewer.addPluginSync(new DepthBufferPlugin())
  
  // Load an environment map
  const envPromise = viewer.setEnvironmentMap('https://threejs.org/examples/textures/equirectangular/venice_sunset_1k.hdr')
  const modelPromise = viewer.load('https://threejs.org/examples/models/gltf/DamagedHelmet/glTF/DamagedHelmet.gltf', {
    autoCenter: true,
    autoScale: true,
  })

  Promise.all([envPromise, modelPromise]).then(([env, model]) => {
    console.log('Loaded', model, env, viewer)
  })
</script>

Check it in action: https://threepipe.org/examples/#html-js-sample/

Check out the details about the ThreeViewer API and more plugins below.

React

A sample react component in tsx to render a model with an environment map.

import React from 'react'
function ThreeViewerComponent({src, env}: {src: string, env: string}) {
  const canvasRef = React.useRef(null)
  React.useEffect(() => {
    const viewer = new ThreeViewer({canvas: canvasRef.current})

    const envPromise = viewer.setEnvironmentMap(env)
    const modelPromise = viewer.load(src)
    Promise.all([envPromise, modelPromise])
    return () => {
      viewer.dispose()
    }
  }, [])
  return (
     <canvas id="three-canvas" style={{width: 800, height: 600}} ref={canvasRef} />
  )
}

Check it in action: https://threepipe.org/examples/#react-tsx-sample/

Other examples in js: https://threepipe.org/examples/#react-js-sample/ and jsx: https://threepipe.org/examples/#react-jsx-sample/

Vue.js

A sample vue.js component in js to render a model with an environment map.

const ThreeViewerComponent = {
  setup() {
    const canvasRef = ref(null);

    onMounted(() => {
      const viewer = new ThreeViewer({ canvas: canvasRef.value });

      const envPromise = viewer.setEnvironmentMap('https://threejs.org/examples/textures/equirectangular/venice_sunset_1k.hdr');
      const modelPromise = viewer.load('https://threejs.org/examples/models/gltf/DamagedHelmet/glTF/DamagedHelmet.gltf');

      Promise.all([envPromise, modelPromise])

      onBeforeUnmount(() => {
        viewer.dispose();
      });
    });

    return { canvasRef };
  },
};

Check it in action: https://threepipe.org/examples/#vue-html-sample/

Another example with Vue SFC(Single file component): https://threepipe.org/examples/#vue-sfc-sample/

Svelte

A sample svelte component in js to render a model with an environment map.

<script>
    import {onDestroy, onMount} from 'svelte';
    import {ThreeViewer} from 'threepipe'; 

    let canvasRef;
    let viewer;
    onMount(() => {
        viewer = new ThreeViewer({canvas: canvasRef});

        const envPromise = viewer.setEnvironmentMap('https://threejs.org/examples/textures/equirectangular/venice_sunset_1k.hdr');
        const modelPromise = viewer.load('https://threejs.org/examples/models/gltf/DamagedHelmet/glTF/DamagedHelmet.gltf');

        Promise.all([envPromise, modelPromise])
    });
    onDestroy(() => viewer.dispose())
</script>

<canvas bind:this={canvasRef} id="three-canvas" style="width: 800px; height: 600px"></canvas>

Check it in action: https://threepipe.org/examples/#svelte-sample/

NPM/YARN

Installation

npm install threepipe

Loading a 3D Model

First, create a canvas element in your HTML page:

<canvas id="three-canvas" style="width: 800px; height: 600px;"></canvas>

Then, import the viewer and create a new instance:

import {ThreeViewer, IObject3D} from 'threepipe'

// Create a viewer
const viewer = new ThreeViewer({canvas: document.getElementById('three-canvas') as HTMLCanvasElement})

// Load an environment map
await viewer.setEnvironmentMap('https://threejs.org/examples/textures/equirectangular/venice_sunset_1k.hdr')

// Load a model
const result = await viewer.load<IObject3D>('https://threejs.org/examples/models/gltf/DamagedHelmet/glTF/DamagedHelmet.gltf', {
    autoCenter: true,
    autoScale: true,
})

That's it! You should now see a 3D model on your page.

The 3D model can be opened in the editor to view and edit the scene settings, objects, materials, lights, cameras, post-processing, etc. and exported as a GLB file. All settings are automatically serialized and saved in the GLB file, which can be loaded into the viewer. Any plugins used in the editor can be added to the viewer to add the same functionality. The plugin data is automatically loaded(if the plugin is added) when the model is added to the scene.

The viewer initializes with a Scene, Camera, Camera controls(Orbit Controls), several importers, exporters and a default rendering pipeline. Additional functionality can be added with plugins.

Check out the GLTF Load example to see it in action or to check the JS equivalent code: https://threepipe.org/examples/#gltf-load/

Check out the Plugins section below to learn how to add additional functionality to the viewer.

License

The core framework(src, dist, examples folders) and any plugins without a separate license are under the Free Apache 2.0 license.

Some plugins(in the plugins folder) might have different licenses. Check the individual plugin documentation and the source folder/files for more details.

Status

The project is in alpha stage and under active development. Many features will be added but the core API will not change significantly in future releases.

Check out WebGi for an advanced tailor-made solution for e-commerce, jewelry, automobile, apparel, furniture etc.

Documentation

Check the list of all functions, classes and types in the API Reference Docs.

WebGi

Check out WebGi - Premium Photo-realistic 3D rendering framework and tools for web applications and online commerce along with custom modules and rendering solutions for e-commerce, jewelry, automobile, apparel, furniture and other retail applications.

Homepage — Docs

Contributing

Contributions to ThreePipe are welcome and encouraged! Feel free to open issues and pull requests on the GitHub repository.

Features

File Formats

ThreePipe Asset Manager supports the import of the following file formats out of the box:

• Models: gltf, glb, obj+mtl, fbx, drc
• Materials: mat, pmat, bmat (json based), registered material template slugs
• Images: webp, png, jpeg, jpg, svg, ico, avif, hdr, exr
• Misc: json, vjson, zip, txt

Plugins can add additional formats:

• Models
  • 3dm - Using Rhino3dmLoadPlugin
  • ply - Using PLYLoadPlugin
  • usdz - Using USDZLoadPlugin
  • stl - Using STLLoadPlugin
  • ktx - Using KTXLoadPlugin
  • ktx2 - Using KTX2LoadPlugin

Plugins to support more model formats are available in the package @threepipe/plugins-extra-importers including .3ds, .3mf, .collada, .amf, .bvh, .vox, .gcode, .mdd, .pcd, .tilt, .wrl, .mpd, .vtk, .xyz

Loading files

ThreePipe uses the AssetManager to load files. The AssetManager has support for loading files from URLs, local files and data URLs. The AssetManager also adds support for loading files from a zip archive. The zip files are automatically unzipped, and the files are loaded from the zip archive.

viewer.load() is a high-level wrapper for loading files from the AssetManager. It automatically adds the loaded object to the scene and returns a promise that resolves to the loaded object, the materials are also automatically registered to the material manager.

AssetManager internally uses AssetImporter, which provides a low-level API for managing three.js LoadingManager and adding and registering loaders for different file types.

If the purpose is not to add files to the scene then viewer.assetManager.importer.import() method can be used to import files from the AssetImporter. viewer.assetManager.loadImported()](https://threepipe.org/docs/classes/AssetManager.html#loadImported) can then be called to load the imported files after any processing. The viewer.load(), viewer.assetManager.addAsset() and viewer.assetManager.addAssetSingle() methods perform combination of import and loadImported.

3D models

The 3d models are added to viewer.scene.modelRoot on viewer.load unless some option is specified.

const objectGlb = await viewer.load<IObject3D>('https://example.com/file.glb')
const objectFbx = await viewer.load<IObject3D>('https://example.com/file.fbx')
const objectObj = await viewer.load<IObject3D>('https://example.com/file.obj') // .mtl referenced in obj is automatically loaded
// ... load any 3d model file as an object

Here, we are casting to IObject3D type to get the proper type and autocomplete for the object. IObject3D inherits Object3D from three.js and adds some additional properties.

For JavaScript, the type can be omitted.

const objectGlb = await viewer.load('https://example.com/file.glb')

When loading models, several options can be passed to automatically process the model first time, like autoScale, autoCenter, addToRoot etc. Check AddObjectOptions and ImportAddOptions for more details.

Materials

The materials downloaded as PMAT/BMAT/JSON etc from threepipe, webgi or the editor can be loaded and registered with the MaterialManager using the viewer.load method.

Custom material types can also be registered by plugins(like dmat for diamonds), which can also be loaded automatically using the viewer.load method.

const pMaterial = await viewer.load<PhysicalMaterial>('https://example.com/file.pmat')
const bMaterial = await viewer.load<UnlitMaterial>('https://example.com/file.bmat')
// ... load any material file as a material

Casting to PhysicalMaterial or UnlitMaterial is optional but recommended to get the proper type and autocomplete for the material.

To assign the material on any object, set it to object.material

// find a loaded mesh in the scene
const object = viewer.scene.getObjectByName('objectName');
// assign the material
object.material = pMaterial;

To copy the properties without changing the material reference, use material.copy() or material.setValues() methods.

object.material.copy(pMaterial);

// or use material manager to apply to multiple materials.
viewer.assetManager.materialManager.applyMaterial(pMaterial, 'METAL') // apply props to all materials/objects with the name METAL

TODO: add examples for material load and copy

Images/Textures

Images can be loaded using the viewer.load method. There is built-in support for loading all image formats supported by the browser (webp, png, jpeg, jpg, svg, ico, avif) and hdr, exr, hdr.png formats for all browsers. More formats like ktx2, ktx, etc. can be added using plugins.

const texture = await viewer.load<ITexture>('https://example.com/file.png')
// ... load any image file as a texture

Casting to ITexture is optional but recommended to get the proper type and autocomplete for the texture. It inherits from three.js Texture and adds some additional properties.

To assign the texture on any material, set it to material.map

// find a loaded mesh in the scene
const object = viewer.scene.getObjectByName('objectName');
const material = object.material as PhysicalMaterial;
// assign the texture
material.map = texture;
material.setDirty() // to let the viewer know that the material has changed and needs to re-render the scene. This will also trigger fade effect if FrameFadePlugin is added.

Check out the image load example to see it in action or to check the JS equivalent code: https://threepipe.org/examples/#image-load/

Zip files

.zip files are automatically unzipped and the files are sent to re-load recursively when loaded with viewer.load. Any level of zip hierarchy is flattened. Loading files like .gltf with references to assets inside the zip file, any relative references are also automatically resolved. This is supported for file types like gltf, glb, obj, etc which support references to external files and has root set to `true in IImporter.

const objectGltf = await viewer.load<IObject3D>('https://example.com/model.gltf.zip')

If we know that the zip file contains a single gltf with all the assets, we can cast the result to IObject3D type.

To load multiple assets from zip files like multiple textures or materials, use viewer.assetManager.addAsset method which returns a promise of array of loaded assets.

const textures = await viewer.assetManager.addAsset<ITexture[]>('https://example.com/textures.zip')
const materials = await viewer.assetManager.addAsset<IMaterial[]>('https://example.com/materials.zip')

The auto import of zip contents can be disabled to get the files and blobs in the zip

const zip = await viewer.load<any>('https://example.com/file.zip', {autoImportZipContents: false})

TODO - add example for loading zip files.

txt, json files

Text and JSON files can be loaded using the viewer.load method and return strings and objects respectively.

const text = await viewer.load<string>('https://example.com/file.txt')
const json = await viewer.load<any>('https://example.com/file.json')

Data URLs

Data URLs can be loaded using the viewer.load method. The correct mime-type is required to be set in the data URL for finding the correct importer.

const dataUrl = 'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA' // ... some data url
const texture = await viewer.load<ITexture>(dataUrl)

Local files, File and Blob

Local files can be loaded using the viewer.load method by passing a IAsset object with File or Blob object.

const file: File|Blob = fileObject // create a new file, blob or get from input element 
const text = await viewer.load<IObject>({
  // a path/name is required to determine the proper importer by extension. `file.name` can also be used if available
  path: 'file.glb', 
  file
})

The same can be done for any file type.

To load a Map of files(like when multiple files are dragged and dropped on the webpage) with internal references to other files, use viewer.assetManager.importer.importFiles method. Check the source for DropzonePlugin for an example.

Background, Environment maps

The background and environment maps can be set using the viewer.setBackgroundMap and viewer.setEnvironmentMap methods respectively. These accept both loaded textures from viewer.load and direct URLs. Files can be of any image format including hdr, exr.

await viewer.setEnvironmentMap('https://example.com/file.hdr')
await viewer.setBackgroundMap('https://example.com/file.png')

The same texture can be set to both by setting setBackground or setEnvironment to true in the options:

await viewer.setEnvironmentMap('https://example.com/file.hdr', {setBackground: true})

Check the HDR Load example to see it in action: https://threepipe.org/examples/#hdr-load/

SVG strings

SVG strings can be converted to data urls using the svgUrl string template function

const svgDataUrl = svgUrl`<svg viewBox="0 0 100 100" xmlns="http://www.w3.org/2000/svg"> ... </svg>`;
const texture = await viewer.load<ITexture>(dataUrl)

Custom file types

Custom file importers/loaders can be registered to the AssetImporter using the addImporter method.

class CustomLoader extends FileLoader implements ILoader{
    constructor(manager?: LoadingManager) {
        super(manager);
    }
    load(url: string, onLoad: (data: any) => void, onProgress?: (event: ProgressEvent) => void, onError?: (event: ErrorEvent) => void): Mesh {
      this.setResponseType('json')
      return super.load(url, (json: any)=>{
        const mat = new PhysicalMaterial(json)
        onLoad?.(mat)
      }, onProgress, onError)
    }
}

viewer.assetManager.importer.addImporter(new Importer(CustomLoader, ['ext'], ['mime/type'], false))

// load the file
const mat = await viewer.load<PhysicalMaterial>('https://example.com/file.ext')

Exporting files

Threepipe has support for exporting various asset type with AssetManager, as well as support to export viewer and plugin configuration, arbitrary objects etc using the serialization system.

viewer.export() is a high-level wrapper for exporting scene objects, materials, textures, render targets, viewer/scene configuration and plugin configurations.

AssetManager internally uses AssetExporter to export files. AssetExporter includes some basic exporters for glb, exr, textures, and materials and a system to register exporters for different file types with plugins or custom exporters.

Exporting 3D models

Export the root scene as glb

const blob = await viewer.exportScene({
  viewerConfig: true, // default = true. export all viewer and plugin configuration. if false only the model root object is exported.
})
// download the file
downloadBlob(blob, 'scene.glb')

Export a single object from the scene as glb

const object = viewer.scene.getObjectByName('objectName');
const glb: Blob = await viewer.export(object, {
  exportExt: 'glb', // default = glb for models
  embedUrlImages: true, // default = false. embed images in glb even when url is available.
})
// download the file
downloadBlob(glb, 'object.glb')

Check the example glb-export to see a demo.

Exporting Materials

Export a material

const material = viewer.assetManager.materialManager.findMaterialsByName('materialName')[0];
// or 
// const material = viewer.scene.getObjectByName('objectName').material;
const blob = await viewer.export(material)
// download the file
downloadBlob(blob, 'material.' + blob.ext)

Check the example pmat-material-export to see a demo.

Exporting Canvas Images

Canvas Screenshot/snapshot can be exported as png, jpeg or webp(if supported by the browser)

const blob = await viewer.getScreenshotBlob({mimeType: 'image/' + type, quality: 0.85})
// or to get data url:
// const dataUrl = await viewer.getScreenshotDataUrl({mimeType: 'image/' + type, quality: 0.85})
// download the file
downloadBlob(blob, 'screenshot.' + blob.ext)

Check the example image-snapshot-export to see a demo.

Exporting Textures

Textures can be exported to JSON using viewer.export or AssetExporter

const texture = await viewer.load('https://example.com/file.jpeg')
const blob = await viewer.export(texture)
downloadBlob(blob, texture.name + '.' + blob.ext)

Render target textures can be exported with viewer.renderManager.exportRenderTarget or viewer.export, read about Exporting Render Targets below.

TODO: add examples for texture export

Textures and Uint8 Data Textures can be exported as a data url or copied to a new canvas

// get a base64 data url
const dataUrl = textureToDataUrl(texture, 4096, false, 'image/png') // texture or data texture, max-size, flipY, mimeType
// or copy to a new canvas
const canvas = textureToCanvas(texture, 4096) // texture or data texture, max-size

Data Textures of type Half float and Float can be exported with viewer.export

const dataTex = await viewer.load('https://example.com/file.hdr')
const blob = await viewer.export(dataTexture, {exportExt: 'exr'})

Check the example hdr-to-exr to see a demo of HDR to EXR conversion.

TODO: add support to export unsigned byte textures as png, jpeg, webp

Exporting Images/Textures

Exporting Textures as Images with image of types ImageBitmap, HTMLImageElement, HTMLOrSVGImageElement, CanvasImageSource, HTMLCanvasElement, OffscreenCanvas can be exported to png data urls with imageBitmapToBase64 function.

const texture = await viewer.load('https://example.com/file.jpeg')

const dataUrl = await imageBitmapToBase64(texture.image, 'image/png', 0.85);

TODO: add support for texture export as images in AssetExporter

Exporting Render Targets

Unsigned byte render targets can be exported as png, jpeg or webp(if supported by the browser)

const depthPlugin = viewer.addPluginSync(DepthBufferPlugin, UnsignedByteType)
// wait for the first render
const blob = await viewer.export(depthPlugin.target!, {exportExt: 'png'})
if (blob) downloadBlob(blob, target.texture.name + '.' + blob.ext)

Half float and float render targets can be exported as exr

const depthPlugin = viewer.addPluginSync(DepthBufferPlugin, HalfFloatType)
// wait for the first render
const blob = await viewer.export(depthPlugin.target!, {exportExt: 'exr'})
if (blob) downloadBlob(blob, target.texture.name + '.' + blob.ext)

Note: exportExt is determined automatically if not specified.

Render pipeline

Threepipe includes a RenderManager for managing the composition pipeline, and provides helpers for rendering and render target management.

The RenderManager includes a EffectComposer from three.js for rendering passes and a WebGLRenderer for rendering, but the pass management and sorting is managed by the RenderManager itself.

The RenderManager inherits from RenderTargetManager which provides utilities for creating, tracking and destroying dedicated and temporary render targets.

Render Targets

Render targets can be created using the viewer.renderManager.createTarget and viewer.renderManager.createTargetCustom methods. These can then be disposed using the viewer.renderManager.disposeTarget method when not needed anymore.

Or to create temp targets for one time use viewer.renderManager.getTempTarget and viewer.renderManager.releaseTempTarget methods. can be used. All created render targets are tracked in the RenderManager, and are resized and disposed automatically when needed along with the viewer.

const newTarget = viewer.renderManager.createTarget({sizeMultiplier: 1})
// or
const newTarget2 = viewer.renderManager.createTarget({size: {
    width: 1024,
    height: 1024,
  },
  type: HalfFloatType
})
// or clone an existing target
const newTarget3 = viewer.renderManager.composerTarget.clone()
// for multi-sample render target
const newTarget4 = viewer.renderManager.createTarget({sizeMultiplier: 1, samples: 4})

// or create a custom target
const newTarget5 = viewer.renderManager.createTargetCustom(
    {width: 1024, height: 1024},
    {type: HalfFloatType},
    WebGLCubeRenderTarget
)

// dispose targets
viewer.renderManager.disposeTarget(newTarget)
viewer.renderManager.disposeTarget(newTarget2)
viewer.renderManager.disposeTarget(newTarget3)
viewer.renderManager.disposeTarget(newTarget4)
viewer.renderManager.disposeTarget(newTarget5)

// get a temporary target
const tempTarget = viewer.renderManager.getTempTarget({sizeMultiplier: 1})
// release the temporary target
viewer.renderManager.releaseTempTarget(tempTarget)

Note: Render targets created with a sizeMultiplier are automatically resized when the canvas is resized.

Passes

By default, the render pipeline includes 2 passes - RenderPass for rendering the scene hierarchy and ScreenPass for rendering the final output on the canvas.

More passes can be added and removed from the pipeline using the registerPass and unregisterPass methods.

The pipeline passes need to follow the interface of IPipelinePass and IPipelinePassPlugin. Which adds some important parameters over the three.js Pass, like pass id and support for defining where the pass should be added in the pipeline and it's dependants.

const pass = new GBufferRenderPass('customPass', viewer.renderManager.createTarget({sizeMultiplier: 1}))
pass.before = ['render'] // Add the pass before the render pass
pass.after = [] // Add the pass after these passes (none in this case)
pass.required = ['render'] // render pass is required to be in the pipeline for this
viewer.renderManager.registerPass(pass)

Note: See PipelinePassPlugin for an abstract plugin that provides the boilerplate to create a plugin that registers a custom pass in the pipeline. Check NormalBufferPlugin for an example of that.

Note: All effects in post-processing or material extension need not be a separate pass in the pipeline. Most effects can be achieved with either extending the scene object material shaders or the Screen Pass material shader using Material extension system

Material Extension

Threepipe includes a Material extension system along with a material manager. The material manager is used to register materials and material extensions.

The material extensions are used to extend any material in the scene, or any plugin/pass with additional uniforms, defines, shader snippets and provides hooks.

The material extensions are automatically applied to all materials in the scene that are compatible, when the extension is registered or when the material is added to the scene.

Threepipe includes several built-in materials like PhysicalMaterial, UnlitMaterial, ExtendedShaderMaterial, LegacyPhongMaterial, that include support for extending the material. Any three.js material can be made extendable, check the ShaderPass2 class for a simple example that adds support for material extension to three.js ShaderPass.

The material extensions must follow the MaterialExtension interface. Many plugins create their own material extensions either for the scene materials or shader passes(like the screen pass). Some plugins like DepthBufferPlugin also provides helper material extensions for other custom plugins to fetch value in the depth buffer.

A sample material extension

const extension: MaterialExtension = {
    shaderExtender: (shader)=> {
        // change the shader properties like shader.fragmentShader, etc
    },
    parsFragmentSnippet: ` // add some code before the main function in the fragment shader
    uniform sampler2D tTexture;
    uniform float opacity;
    `,
    extraUniforms: {
      tTexture: ()=>({value: getTexture()}),
      opacity: {value: 1}
      // add additional uniforms, these can be IUniform or functions that return IUniform
    },
    extraDefines: {
      ['DEPTH_PACKING']: BasicDepthPacking,
      ['SOME_DEFINE']: ()=>"1",
      // add additional defines, these can be values or functions that return values 
    },
    priority: 100, // priority when using multiple extensions on the same material
    isCompatible: (material) => material.isMeshBasicMaterial, // check if the material is compatible with this extension,
    computeCacheKey: (material) => material.uuid, // a custom cache key for the material extension. Shader is recompiled when this is changed
    onObjectRender: (object: Object3D, material: IMaterial) => {
      // called when some object is rendererd which has a material with this extension.
    },
    // uiConfig
    // check more properties and hooks in the MaterialExtension interface
}

// The extension can be registered to all the materials using the MaterialManager
viewer.assetManager.materialManager.registerMaterialExtension(extension)

// or register it on a single material (like the Screen Pass)
viewer.renderManager.screenPass.material.registerMaterialExtensions([extension])

UI Configuration

Most of the classes and plugins in Threepipe include uiconfig.js support and can be used to create configuration UIs, 3d configurators and even full-editors. The UIs are automatically generated based on the configuration object under .uiConfig property on all objects. These are of type UiObjectConfig. In some classes, the ui configs are also generated using typescript decorators.

The uiConfig is also added to all three.js objects and materials when they are added to the scene.

The UIs can be generated at runtime using any of the UI plugins like TweakpaneUIPlugin, BlueprintJsUiPlugin

An example showing how to create a UI for a material

const ui = viewer.addPluginSync(TweakpaneUiPlugin)

const object = viewer.scene.getObjectByName('objectName');
const material = object.material as PhysicalMaterial;

ui.appendChild(material.uiConfig)

See it in action: https://threepipe.org/examples/#material-uiconfig/

Check more examples showing Viewer UI, Scene UI, Object UI, Camera UI

TweakpaneEditorPlugin further uses the Tweakpane configuration panel along with various plugins to create an 3d editor.

Custom UI configuration can be created to generate custom UI for the editor or tweaking. This can be done by using typescript decorators or defining the UI in javascript as a UiObjectConfig object.

Here is a sample of extending the orbit controls class with decorators to automatically generate UI.

@uiPanelContainer('Orbit Controls')
export class OrbitControlsWithUi extends OrbitControls implements IUiConfigContainer {
  // for autocomplete
  uiConfig?: UiObjectConfig<void, 'panel'> 

  @uiToggle() enabled = true

  @uiToggle() dollyZoom = false
  @uiToggle() enableDamping = true
  @uiInput() dampingFactor = 0.08

  @uiToggle() autoRotate = false
  @uiInput() autoRotateSpeed = 2.0

  @uiToggle() enableZoom = true
  @uiInput() zoomSpeed = 0.15
  @uiInput() maxZoomSpeed = 0.20

  @uiToggle() enableRotate = true
  @uiInput() rotateSpeed = 2.0

  @uiToggle() enablePan = true
  @uiInput() panSpeed = 1.0

  @uiInput() autoPushTarget = false
  @uiInput() autoPullTarget = false
  @uiInput() minDistance = 0.35
  @uiInput() maxDistance = 1000

  @uiInput() minZoom = 0.01
  @uiInput() maxZoom = 1000

  @uiInput() minPolarAngle = 0
  @uiInput() maxPolarAngle = Math.PI

  @uiInput() minAzimuthAngle = -10000 // should be -Infinity but this breaks the UI
  @uiInput() maxAzimuthAngle = 10000

}

Check out the full source code: ./src/three/controls/OrbitControls3.ts for proper implementation

See it in action: https://threepipe.org/examples/#camera-uiconfig/ Open the Camera UI and click on the Orbit Controls panel.

There are many available decorators like uiToggle, uiSlider, uiInput, uiNumber, uiColor, uiImage. Check the complete list in the uiconfig.js documentation.

The UI configuration can also be created using json objects in both typescript and javascript

const viewer = new ThreeViewer({...})

const ui = viewer.addPluginSync(TweakpaneUiPlugin)

const state = {
    position: new Vector3(),
    scale: 1,
}

ui.appendChild({
  type: 'folder',
  label: 'Custom UI',
  children: [
    {
        type: 'vec3',
        label: 'Position',
        property: [state, 'position']
    },
    {
        type: 'slider',
        label: ()=>'Scale', // everything can be a function as well.
        property: [state, 'scale'],
        bounds: [1, 2],
        stepSize: 0.1,
    }
  ]
})

TODO: create example/codepen for this

Serialization

Easy serialization of all threepipe and most three.js objects are supported out of the box using the Asset Manager. Fine control over serialization is also supported using the ThreeSerialization class

Call ThreeSerialization.serialize on any object to serialize it. and ThreeSerialization.deserialize to deserialize the serialized object.

This is done by performing a nested serialization of all the properties of the object. It's possible to implement custom serializers for custom types and classes and is done for three.js primitives, objects and plugins in threepipe

To make a custom data class that is serializable, mark it using @serializable decorator and any properties using @serialize decorator.

@serializable('DataClass')
class DataClass{
    @serialize() prop1 = 1
    @serialize() prop2 = 'string'
    @serialize() prop3 = new Vector3()
    @serialize() prop4 = new PhysicalMaterial()
    @serialize() prop4 = {
        prop1: 1,
        prop2: 'string',
        prop3: new Vector3(),
        prop4: new PhysicalMaterial(),
    }
}

const data = new DataClass()
const serialized = ThreeSerialization.serialize(data)
const deserialized = ThreeSerialization.deserialize(serialized)

The classes without a @serializable decorator are serialized as plain objects. These can still include @serialize decorator to mark the properties are serializable but these classes cannot be deserialized into a new instance of the class. The ThreeViewer and plugins are an example of these. When deserialized they need an object to deserialize into. This ensures there is always just one instance. With this, the serialization system works like toJSON and fromJSON methods in three.js.

Check the plugin system below for more details on how to mark properties as serializable for plugins.

class CustomClass{
    @serialize() prop1 = 1
    @serialize() prop2 = 'string'
    @serialize() prop3 = new Vector3()
    @serialize() prop4 = new PhysicalMaterial()
}
const obj = new DataClass()
const serialized = ThreeSerialization.serialize(data)
// now to deserialize we need to pass in the object to deserialize into
ThreeSerialization.deserialize(serialized, obj)

Plugin System

Threepipe includes a plugin system for adding additional features to the viewer in a modular way. The plugins can be added synchronously or asynchronously using viewer.addPluginSync and viewer.addPlugin methods respectively.

It is recommended to create custom plugins for reusable features, as they provide built-in features for ui configuration, serialization, integration with editors etc and are easy to manage and tree-shake in the code.

Check out the list of plugins in the Plugin List section below.

To create new plugins, simply implement the IViewerPlugin interface or extend the AViewerPluginSync or AViewerPluginAsync classes. The only difference is that in async the onAdded and onRemove functions are async

Here is a sample plugin

@uiFolder("Sample Plugin") // This creates a folder in the Ui. (Supported by TweakpaneUiPlugin)
export class SamplePlugin extends AViewerPluginSync<"sample-1" | "sample-2"> {
  // These are the list of events that this plugin can dispatch.
  static readonly PluginType = "SamplePlugin"; // This is required for serialization and handling plugins. Also used in viewer.getPluginByType()

  @uiToggle() // This creates a checkbox in the Ui. (Supported by TweakpaneUiPlugin)
  @serialize() // Adds this property to the list of serializable. This is also used when serializing to glb in AssetExporter.
  enabled = true;

  // A plugin can have custom properties.

  @uiSlider("Some Number", [0, 100], 1) // Adds a slider to the Ui, with custom bounds and step size (Supported by TweakpaneUiPlugin)
  @serialize("someNumber")
  @onChange(SamplePlugin.prototype._updateParams) // this function will be called whenevr this value changes.
  val1 = 0;

  // A plugin can have custom properties.
  @uiInput("Some Text") // Adds a slider to the Ui, with custom bounds and step size (Supported by TweakpaneUiPlugin)
  @onChange(SamplePlugin.prototype._updateParams) // this function will be called whenevr this value changes.
  @serialize()
  val2 = "Hello";

  @uiButton("Print Counters") // Adds a button to the Ui. (Supported by TweakpaneUiPlugin)
  public printValues = () => {
    console.log(this.val1, this.val2);
    this.dispatchEvent({ type: "sample-1", detail: { sample: this.val1 } }); // This will dispatch an event.
  }

  constructor() {
    super();
    this._updateParams = this._updateParams.bind(this);
  }

  private _updateParams() {
    console.log("Parameters updated.");
    this.dispatchEvent({ type: "sample-2" }); // This will dispatch an event.
  }

  onAdded(v: ThreeViewer): void {
    super.onAdded(v);

    // Do some initialization here.
    this.val1 = 0;
    this.val2 = "Hello";

    v.addEventListener("preRender", this._preRender);
    v.addEventListener("postRender", this._postRender);
    v.addEventListener("preFrame", this._preFrame);
    v.addEventListener("postFrame", this._postFrame);

    this._viewer!.scene.addEventListener("addSceneObject", this._objectAdded); // this._viewer can also be used while this plugin is attached.
  }

  onRemove(v: ThreeViewer): void {
    // remove dispose objects

    v.removeEventListener("preRender", this._preRender);
    v.removeEventListener("postRender", this._postRender);
    v.removeEventListener("preFrame", this._preFrame);
    v.removeEventListener("postFrame", this._postFrame);

    this._viewer!.scene.removeEventListener("addSceneObject", this._objectAdded); // this._viewer can also be used while this plugin is attached.

    super.onRemove(v);
  }

  private _objectAdded = (ev: IEvent<any>) => {
    console.log("A new object, texture or material is added to the scene.", ev.object);
  };
  private _preFrame = (ev: IEvent<any>) => {
    // This function will be called before each frame. This is called even if the viewer is not dirty, so it's a good place to do viewer.setDirty()
  };
  private _preRender = (ev: IEvent<any>) => {
    // This is called before each frame is rendered, only when the viewer is dirty.
  };
  // postFrame and postRender work the same way as preFrame and preRender.
}

Notes:

• All plugins that are present in the dependencies array when the plugin is added to the viewer, are created and attached to the viewer in super.onAdded
• Custom events can be dispatched with this.dispatchEvent, and subscribed to with plugin.addEventListener. The event type must be described in the class signature for typescript autocomplete to work.
• Event listeners and other hooks can be added and removed in onAdded and onRemove functions for the viewer and other plugins.
• To the viewer render the next frame, viewer.setDirty() can be called, or set this.dirty = true in preFrame and reset in postFrame to stop the rendering. (Note that rendering may continue if some other plugin sets the viewer dirty like ProgressivePlugin or any of the animation plugins). Check isConverged in ProgressivePlugin to check if its the final frame.
• All Plugins which inherit from AViewerPlugin support serialisation. Create property serializeWithViewer = false to disable serialisation with the viewer in config and glb or toJSON: any = undefined to disable serialisation entirely
• plugin.toJSON() and plugin.fromJSON() or ThreeSerialization can be used to serialize and deserialize plugins. viewer.exportPluginConfig and viewer.importPluginConfig also exist for this.
• @serialize('label') decorator can be used to mark any public/private variable as serializable. label (optional) corresponds to the key in JSON.
• @serialize supports instances of ITexture, IMaterial, all primitive types, simple JS objects, three.js math classes(Vector2, Vector3, Matrix3...), and some more.
• uiDecorators can be used to mark properties and functions that will be shown in the Ui. The Ui shows up automatically when TweakpaneUiPlugin/BlueprintJsUiPlugin is added to the viewer. Plugins have special features in the UI for download preset and saving state.

Check various plugins in the source code for more examples.

Viewer API

ThreeViewer - is the main entry point to 3d rendering on the canvas.

• .renderManager: ViewerRenderManager & RenderManager & RenderTargetManager - Render manager for managing the rendering and composition pipeline, and provides helpers for rendering and render target management
  • .renderer: IWebGLRenderer - for rendering. Instance of three.js WebGLRenderer
  • .composer: EffectComposer2 - for rendering passes. Instance of three.js EffectComposer
  • .context: WebGLRenderingContext - WebGL rendering context
  • .renderPass: ExtendedRenderPass - Render pass for rendering the scene. Instance of three.js RenderPass with extra features
  • .screenPass: ScreenPass - Screen pass for rendering the final output. Instance of three.js ShaderPass with extra features.
• .scene: RootScene - Main scene used for rendering. Instance of three.js Scene
  • .mainCamera: PerspectiveCamera2 - Main camera currently being used for rendering. Instance of three.js PerspectiveCamera
• .assetManager: AssetManager - Asset manager for loading, managing and exporting assets
  • .importer: AssetImporter - for importing assets
  • .exporter: AssetExporter - for exporting assets
  • .materialManager: MaterialManager - for managing materials and material extensions
• .plugins: Record<string, IViewerPlugin> - Plugins added to the viewer
• .uiConfig: UiObjectConfig - UI confguration for the viewer. Used to automatically generate UIs for the viewer and plugins.

ThreeViewer

Source Code: src/viewer/ThreeViewer.ts

API Reference: ThreeViewer

ThreeViewer is the main entry point to the viewer. It provides all the API for managing the scene, camera, rendering, plugins, etc.

It is initialized with either a canvas element or a HTMLElement for the container. The canvas element is used for rendering, and the options are used to configure the viewer. If the canvas element is not provided, a new canvas element is created and appended to the container.

More options can be passed in the constructor to configure various built-in plugins and rendering features in the viewer.

Constructor

import {ThreeViewer, CameraViewPlugin} from 'threepipe'

// Create a viewer. All options except canvas/container are optional
const viewer = new ThreeViewer({
  canvas: document.getElementById('mcanvas') as HTMLCanvasElement,
  // or a container like: 
  // container: document.getElementById('mcontainer'),
  // container: document.body,

  // Set the render scale to render at device resolution and clamp to max 2.
  renderScale: 'auto',
  // or Set the render scale to render at device resolution
  // renderScale: window.devicePixelRatio,
  // modify the screen shader: See ScreenPass and ScreenPass.glsl for more details
  screenShader: `diffuseColor = diffuseColor * 2.0;`,
  
  // Add TonemapPlugin
  tonemap: true,
  // Use MSAA(anti-aliasing)
  msaa: false,
  // Use Uint8 RGBM HDR Render Pipeline. Provides better performance with post-processing. RenderManager Uses Half-float if set to false. 
  rgbm: true,
  // Use rendered gbuffer as depth-prepass / z-prepass.
  zPrepass: false,

  // Options for AssetManager
  assetManager: {
      // Use a custom CacheStorage
      storage: caches.open('threepipe-assetmanager'),
  },

  // Use DropzonePlugin to add support for file drag and drop
  // Enable and set properties
  dropzone: {
    // Set allowed extensions
    allowedExtensions: ['png', 'glb', 'gltf'],
    // Automatically add downloaded assets
    autoAdd: true
    // autoImport: true,
    // domElement: document.body,
    // addOptions: { ... }
    // importOptions: { ... }
  },
  // By default its false
  // dropzone: false,
  // To Enable without options
  // dropzone: true
  
  // Add some plugins after viewer creation.
  plugins: [CameraViewPlugin, new CustomPlugin()],
  
  // Shorthand to load files immediately after viewer initialization
  load: {
      src: 'https://example.com/file.glb',
      environment: 'https://example.com/file.hdr',
      background: 'https://example.com/file.png',
  },
  onLoad: (viewer) => {
      // Called when all the files are loaded
  },
})

Check the interface ThreeViewerOptions for all the options.

To dispose off the viewer and all its resources call viewer.dispose() method.

To dispose only the scene objects and not the complete viewer, use viewer.scene.disposeSceneModels()

Plugin Functions

import {ThreeViewer, TonemapPlugin, DepthBufferPlugin, NormalBufferPlugin} from 'threepipe'

const viewer = new ThreeViewer({...})

// Add a plugin
const plugin = viewer.addPluginSync(new TonemapPlugin())
// plugins can be added with just the class also
const plugin2 = viewer.addPluginSync(TonemapPlugin)

// Add multiple plugins at once
viewer.addPluginsSync([
  TonemapPlugin,
  new NormalBufferPlugin(),
  DepthBufferPlugin,
  //  ...
])

// Get a plugin
const plugin3 = viewer.getPlugin(TonemapPlugin)
        
// Get or add a plugin, when not sure if the plugin is already added
const plugin4 = viewer.getOrAddPluginSync(TonemapPlugin)

// Remove a plugin
viewer.removePluginSync(TonemapPlugin)

Note: all sync functions above have async counterparts like addPlugin, getOrAddPlugin, removePlugin that are used for async plugins. There are no async plugins built-in to threepipe yet.

Import/Export Functions

import {ThreeViewer} from 'threepipe'

const viewer = new ThreeViewer({...})

// Load a 3d model
const object = await viewer.load<IObject3D>('https://example.com/file.glb')

// Load a material
const material = await viewer.load<PhysicalMaterial>('https://example.com/file.pmat')

// Load an image
const texture = await viewer.load<ITexture>('https://example.com/file.png')

// Import a model without adding to the scene
const imported = await viewer.import('https://example.com/file.glb')

// Export the complete scene with viewer configuraion
const exportedScene = await viewer.exportScene({})

// Export an object
const exported = await viewer.export(object)

// Export a material
const exportedMaterial = await viewer.export(material)

// Export a texture
const exportedTexture = await viewer.export(texture)

// Export viewer and plugins configurations
const exportedConfig = await viewer.export(viewer)

// Export plugin configuration
const exportedPlugin = await viewer.exportPlugin(viewer.getPlugin(PluginClass))

// Set Background Image
await viewer.setBackgroundMap('https://example.com/file.png')

// Set Environment Map
await viewer.setEnvironmentMap('https://example.com/file.hdr')

// Add an imported object or a created three.js object to the scene
viewer.addSceneObject(imported)

viewer.load - Loads a single asset by path or IAsset object, and adds to the scene if its 3d object or loads it if it's a configuration It is the same as AssetManager.addAssetSingle. Use AssetManager.addAsset to load multiple assets from the same path like in case of zip bundles.

viewer.import - Load a single asset but does not add to the scene or load the configuration. It is the same as AssetManager.importer.importSingle. Use AssetManager.importer.import to import multiple assets from the same path like in case of zip bundles.

viewer.export - Exports an object, material, texture, render target or plugin configuration and returns a Blob. It is similar to AssetManager.exporter.exportObject but adds support for exporting plugin and self(viewer) configs.

viewer.exportScene - Exports the scene model root and all configurations into a bundled glb file and returns a blob.

viewer.exportPlugin - Exports a plugin configuration and returns a blob.

viewer.setBackgroundMap - Sets the background map to the given texture or url. Also sets it as environment map if setEnvironment is true in the options.

viewer.setEnvironmentMap - Sets the environment map to the given texture or url. Also sets it as background if setBackground is true in the options.

viewer.addSceneObject - Adds an imported object or a created three.js object to the scene model root. If an imported scene model root is passed, it will be loaded with viewer configuration, unless importConfig is false in the options.

Frame/Rendering Events

import {ThreeViewer} from 'threepipe'

const viewer = new ThreeViewer({...})

// Add a callback to be called before every frame, irrespective of whether enything is being rendered that frame
viewer.addEventListener('preFrame', (ev)=>{
    console.log(ev);
    // change something 
    viewer.setDirty() // let the viewer know to re-render the scene from this frame
})

// Add a callback to be called after every frame, irrespective of whether enything was rendered that frame
viewer.addEventListener('postFrame', (ev)=>{
    console.log(ev);
    // change something 
    viewer.setDirty() // let the viewer know to re-render the scene from next frame
})

// Add a callback to be called before every render, only if something is being rendered that frame
viewer.addEventListener('preRender', (ev)=>{
    // canvas is about to be rendered, or re-rendered for progressive rendering
    console.log(ev, viewer.renderManager.frameCount);
})

// Add a callback to be called after every render, only if something was rendered that frame
viewer.addEventListener('postRender', (ev)=>{
    // canvas is rendered, or re-rendered for progressive rendering
    console.log(ev);
})

// Listen to viewer.setDirty() calls 
viewer.addEventListener('update', (ev)=>{
    // viewer.setDirty() was called by some plugin