All Collections
frontline.io 3D Studio
Advanced processes
Adding Mixed Reality Object Detection
Adding Mixed Reality Object Detection
Updated over a week ago

With object detection, 3d objects can be placed on real elements from your physical environment by recognizing the outline shape.

For the detection model, there are a few requirements:

  • A Frontline scene must be created, if it hasn't yet, refer to Importing your 3d model

  • A detection object model - must be under 20k polygons, and made of under 10 parts. it can be the whole model or any part of it that is distinguishable for easy detection (for example a bright area with a defined shape and not a white cube)

  • The main model transforms (position, rotation, scale) should be exactly as the Root object in the scene.

Steps to add object detection:

1. After we have our detection model, right-click on it in the hierarchy window, go to LLS > create > object tracking manager. This creates multiple objects in our scene to help with object detection. These objects must remain untouched.

2. Now we need to define a visual frame position to set where object detection will start with:
2a. Choose a location that is logical to where the user will stand relative to the real-life element.
2b. Then select the Testing camera rotation object, under the object tracking manager. Click control shift F to snap it to your current view.
2c. after that, do the same to the testing camera distance object. you should be able to see the camera view in a small thumbnail.

3. After we have everything set, we need to build a file for Hololens, android, or any other available platform that supports mixed reality.

4. To test this, in your chosen platform, open the 3d item, select the AR mod, select the "Object Detection" option, and try to adjust the camera over the physical object in your room/environment.

The virtual object should snap and overlay on the physical one.

Troubleshooting:

Hololens build does not exist in frontline builder
​Cause: Hololens module is not installed in Unity.
Install it through Unity Hub under the name "Universal Windows Platform build Support", refer to installing Unity for more information.

After creating the Object Tracking Manager it is empty.

Solution: Check that the following are correct:

1. In the object called UIData, find the category called "Scene data", if the values there are empty or none, that means your scene is corrupt and you need to create it again

2. You have to create the tracking manager while selecting your detection model.

3. Remove any Object tracking manager and Placer objects in your scene before creating a new one.

I want to change the object detection model after setting it up

Solution: In order to change it, you need to first remove it entirely from your scene:

1. Remove from the scene the following objects: ObjectTrackingManager, and Placer.

2. Now import the new detection object to the scene, and create a new object tracking manager on it.

Object detection is offset
​cause 1: models are not in the center of the world, make sure that your main model object and Root object are in the center of the world, and have the same position, rotation, and scale values.
​cause 2: models under ObjectTrackingManager or Placer are not exactly aligned with the main model:

1. Turn on the mesh renderer on the object detection model under ObjectTrackingManager, check that it has the correct position with the main model, then disable the mesh renderer component again.

2. Do the same for the object detection model under the Placer object.

Did this answer your question?