There have been previous examples of integrating openHAB with a 3D rendering, but they have required significant coding and have relied on some of the more specialized OH UI interfaces such as HABPanel. While working on a completely unrelated project recently, it occurred to me that the library I was using, model-viewer, is a perfect candidate for bringing 3D capability into OH just using the custom widget system. So, in this tutorial I’m going to go over:
- The basics of getting a 3D model shown in a OH widget
- How to integrate other oh widget components with the 3D model for smart home control
- How to used advanced widget + javascript techniques to make the 3D model itself reactive to oh states and event.
A quick note about javascript and widgets
The custom widgets have their own built-in widget expressions. These expressions are based on javascript, but they do not actually access the browser javascript environment and only act in limited (and isolated) contexts. There are other topics in this forum that have broached the idea of actually accessing the full browser javascript environment so I will not go into all the pros and cons of this here. Until this point, however, there has not been, in my opinion, a compelling reason to take advantage of this capabilty. This topic, I believe, is a good reason to begin to showcase some of the ways to actually integrate oh widgets and full javascript scripts and modules (in fact, this will be one of the very first steps).
As always with javascript, do not run any code that you do not trust or know the origin of.
Adding a model-viewer widget
We’re going to need only two things for the first step of displaying a 3D model in a widget: 1) a model, of course, and 2) the model-viewer library.
I had some old svg floor plans of my house lying around, so I grabbed the main floor svg, quick threw it into Blender, extruded some walls, added a touch of color and a few example features and I had a demo model ready.
The model-viewer library requires glTF files (.gltf or .glb) and blender has an excellent gltf i/o plug-in. The single binary file format (.glb) might be a little easier for starters, but, as you will see later on, there is a reason for preferring the the separate file format (.gltf). The file (or files for .gltf) needs to be someplace that OH has access to for it to be loaded, so I just dropped mine in the OH_CONF/html folder.
The model-viewer library can be loaded remotely from a couple different hosting services. For our purposes it’s better to just have a self-hosted copy of the minified version, so I pointed a browser tab to this google hosted libraries site and saved the script (also to my OH_CONF/html folder) as model-viewer.min.js.
The widget
The basic widget is quite simple. For the root component of the widget I’ll use a basic div and set it to full width and height for demo purposes. The width and height, of course, can be adjusted to your own purposes; the div will come in handy in later steps. Inside the div we need to load the model-viewer javascript library. If this were html, the script tag would look like this (because the script is in the html folder and therefore accessed via OH’s \static url location):
<script src="/static/model-viewer.min.js" type="module"></script>
With the widget system we just use the tag as our component name and the attributes become config properties.
So, to load this external javascript library into our widget as a module we can just add:
- component: script
config:
src: /static/model-viewer.min.js
type: module
The last piece that we need is the model viewer itself. The library creates the custom tag model-viewer, and it only needs one attribute to display the model: src pointing to the model file.
Beyond the minimum, however, if we want to be able to interact with our model we should also add the camera-controls attribute. The id attribute will be useful in the next steps, though not requried if you prefer other methods of accessing the element in javascript. Lastly we want a nice big model view so we’ll use style to fill the whole parent div:
- component: model-viewer
config:
id: house-view
src: /static/Behrend_main_nt.glb
camera-controls: true
style:
width: 100%
height: 100%
All together the widget code is short and sweet:
uid: model_viewer_demo
tags: []
props:
parameters: []
parameterGroups: []
component: div
config:
style:
width: 100%
height: 100%
slots:
default:
- component: script
config:
src: /static/model-viewer.min.js
type: module
- component: model-viewer
config:
id: house-view
src: /static/Behrend_main_nt.glb
camera-controls: true
style:
width: 100%
height: 100%
That’s it, and the result is:
At this point, additional configuration of the model is easy. Any of the attributes listed in the model-viewer docs can be incorporated in the same manner by including them in the config object of the model-viewer component. For example, it doesn’t make much sense to be able to swing the model up and see the underside of the floor. To prevent this we can add limits to the camera movement; in this case, we want to limit the maximum pitch so we add:
max-camera-orbit: auto 77deg auto
to the model-viewer configuration.
The model-viewer is based on a streamlined version of the three.js library so it can handle many advanced rendering features (e.g., transparency and roughness in the window). The flat colors aren’t as nice as I’d like, so we’ll take advantage of gltf’s ability to include textures. A little more work in blender to add laminate floors, wood panels to the door, and textured knockdown to the walls and our widget model looks much better:
Here is where the difference between the single file glb format and the multi-file gltf format is important. The single file glb format will be fetching the texture images in a manner that violates OH’s content-security-policy so they will be blocked by the browser. It is possible to use a reverse proxy to modify the default OH CSP to allow OH to load the textures from the binary file, but this is not necessary because if you use the multi-file format and have the texture folder in your configuration directory with the other pieces of the gltf then OH can load the texture files without any issues.
Integrating Other Components
So far, this doesn’t do anything useful (even if it is fun). So, let’s start adding to our model. The power of the model-viewer library is that it makes it very easy (and widget compatible) to integrate html elements into the model. These elements will be drawn on top of the model, but, the library will handle moving and positioning these elements in sync with the model itself.
The system that model-viewer uses is its hotspot/annotation system. Let’s start with a simple example: I want to label some of the spaces in my model. To do this all I need to do is two steps:
- add a child component to the model-viewer that is my label
- make sure that child component is registered as a hotspot of the model-viewer
The child component is easy. I’ll start with a simple div again (the OH Label element will not work because it will not accept the additional required attributes). The component looks like this:
- component: div
config:
class:
- Hotspot
- indoor-label
data-position: 1.6907273451362617m 0m -0.8410376624211873m
data-normal: 1m 0m -1m
content: Living room
To register this component as a hotspot of the model-viewer it must have an attribute named slot with a value that starts with hotspot-. This is one place where we need a little trickery for a couple different technical reasons. This piece of trickery is our next encounter with adding browser level javascript to this widget. In this case we’ll create our own custom little script instead of loading a source library.
- component: script
config:
content: >
[...document.getElementsByClassName('Hotspot')].forEach((hs, idx) => {
hs.setAttribute('slot',`hotspot-${idx + 1}`)
})
This script collects an array of all the elements with the Hotspot class (which we added to our div) and adds an appropriate slot attribute to the element. Now our element is integrated into the model, but the library doesn’t know where in the model to put it. This is where the data-position and data-normal attributes come in. With a lot of extra work, it would be possible to get these values from either blender or your OH window, but the easiest way to do it is to take advantage of the excellent model tester/editor that model-viewer makes available at Model Editor . If you drag and drop your model file(s) into this editor, you can then use it’s Add Hotspot function and click on the location in the model where you want to add your component. The editor will add the new hotspot to its html snippet window and you can copy the position and normal values right from that window. The last few pieces of the above component are just a class to let me style all such labels so they are not just black text on dark flooring and the actual label itself as the content. Here’s the current full widget text and the result:
Widget code
uid: model_viewer_demo
tags: []
props:
parameters: []
parameterGroups: []
component: div
config:
style:
width: 100%
height: 100%
stylesheet: |
.indoor-label {
border-radius: 5px;
background: #90909090;
padding: 2px;
font-weight: bold;
}
slots:
default:
- component: script
config:
src: /static/model-viewer.min.js
type: module
- component: script
config:
content: >
[...document.getElementsByClassName('Hotspot')].forEach((hs, idx) => {
hs.setAttribute('slot',`hotspot-${idx + 1}`)
})
- component: model-viewer
config:
id: house-view
src: /static/Behrend_main.gltf
camera-controls: true
max-camera-orbit: auto 77deg auto
style:
width: 100%
height: 100%
slots:
default:
- component: div
config:
class:
- Hotspot
- indoor-label
data-position: 1.6907273451362617m 0m -0.8410376624211873m
data-normal: 1m 0m -1m
content: Living room
The beauty of this system is that it is not restricted to any single element. Hotspot children of the model-viewer can be any arbitrary element or complex element tree. This means that just as easily as we added div we can add OH specialized components as well. In my living room, on the wall next to the standing lamp is a switch that controls that lamp. Adding an oh-toggle to represent that switch follows the exact same steps as the div, and the component is simply this:
- component: oh-toggle
config:
class:
- Hotspot
data-position: 1.2739383812891471m 0.3081846817712068m -0.17051529192625736m
data-normal: 0m 0m -1m
style:
transform: rotate(-90deg) scale(75%)
item: Outlet_LivingRoomLamp_OnOff
And now there’s a toggle switch on the wall of the model where my real switch is:
You can’t see the effect on my lamp (we’ll come back to this problem later), but trust me it’s turning on and off.
Let’s add another example of an OH component and action. I’ve got a sensor out on my patio and I’d like to see the temperature and be able to open a window of all the sensor properties.
I’ll define two more hotspots: a patio label just like the living room label, and an oh-link with an expression for its text and the group action. I don’t want this component on the patio wall, however, so after I get the component’s position from the model-viewer editor, I’m going to manually adjust them a little so the link is floating in the patio area.
Widget code
uid: model_viewer_demo
tags: []
props:
parameters: []
parameterGroups: []
component: div
config:
style:
width: 100%
height: 100%
stylesheet: |
.indoor-label {
border-radius: 5px;
background: #90909090;
padding: 2px;
font-weight: bold;
}
slots:
default:
- component: script
config:
src: /static/model-viewer.min.js
type: module
- component: script
config:
content: >
[...document.getElementsByClassName('Hotspot')].forEach((hs, idx) => {
hs.setAttribute('slot',`hotspot-${idx + 1}`)
})
- component: model-viewer
config:
id: house-view
src: /static/Behrend_main.gltf
camera-controls: true
max-camera-orbit: auto 77deg auto
style:
width: 100%
height: 100%
slots:
default:
- component: div
config:
class:
- Hotspot
- indoor-label
data-position: 1.6907273451362617m 0m -0.8410376624211873m
data-normal: 1m 0m -1m
content: Living room
- component: oh-toggle
config:
class:
- Hotspot
- scaled-widget
data-position: 1.2739383812891471m 0.3081846817712068m -0.17051529192625736m
data-normal: 0m 0m -1m
style:
transform: rotate(-90deg) scale(75%)
item: Outlet_LivingRoomLamp_OnOff
- component: div
config:
class:
- Hotspot
data-position: -0.0618133434224899m 0m 1.9828180426074304m
data-normal: 0m 0m 1m
content: Patio
- component: oh-link
config:
class:
- Hotspot
data-position: 0.3870192117519836m 0.3260225588598695m 2m
data-normal: 0m 0m 1m
action: group
actionGroupPopupItem: Sensor_Patio
text: =@'Sensor_Patio_Temperature'
At this point you should be able to see that you can now add as many additional hotspots as you want turning this widget into a 3D model equivalent of the floorplan. If you don’t think this tutorial is already too long, then move on to the next section where we’ll start to add more complex custom features.
Using Javascript for Advanced Features
The model-viewer comes with a significant library of javascript methods for interacting with the model, so I’m going to use some of these to introduce how to integrate more js into the widget system. For most of the javascript I’m going to demonstrate here you can either continue to add inline scripts as we did above with adding the child elements to hotspot slots, or just develop your own external script and load it as the src of another script component. However, there are, as you will see, a couple of places where it is necessary to keep the javascript as an inline script to allow for integrating OH information.
Event listeners and calling javascript functions
Let’s start with the room labels. I want my model to automatically show me the ideal view of a room when I click on that label. If you’re familiar with javascript you know that is going to need a click event listener for the label div. You could include a loop in your custom script that adds the event listener to each element that needs it, but you can also define onEvent type events for an element directly through the appropriately named config property. For this task I’ll add onclick listeners that call a custom function. Here’s the custom function:
function camera2hotspot(e) {
mViewer = document.getElementById('house-view')
if (e.target.dataset.target) {
mViewer.cameraTarget = e.target.dataset.target
}
if (e.target.dataset.orbit) {
mViewer.cameraOrbit = e.target.dataset.orbit
}
const hsFov = e.target.getAttribute('field-of-view')
if (hsFov) {
mViewer.fieldOfView = hsFov
}
}
This function will check the label that was clicked on (e.target) for 3 pieces of data: the target, the orbit, and the field of view. For each of these data that exist, it will assign the value (either via model-viewer method, or directly changing the value) and those changes will automatically change the camera position to the desired view. Model-viewer itself will handle the interpolated camera track to the new location, so you don’t have to.
This means that we have to add not only the listener to the label div but also these new data. Again I recommend collecting these data from the model-viewer editor. This time, using the Save current as initial button will write the orbit and field of view data to the snippet window. You will have to refresh the target point to get those values from the edit boxes below the save button. Once you have those values the addition to the component is easy:
- component: div
config:
class:
- Hotspot
- indoor-label
data-position: 1.6907273451362617m 0m -0.8410376624211873m
data-normal: 1m 0m -1m
data-orbit: 115.9deg 38.3deg 8.405m
data-target: 1.62m 0m -0.68m
field-of-view: 13.48deg
onclick: camera2hotspot(event)
content: Living room
Integrating OH information in javascript
As mentioned above, the widget expressions occur in a context that is isolated. This means that the javascript that you run via the browser does not have access to things like the items object (and therefore item states). The place where these two different streams cross, however, is that we can use widget expressions to set the content of a script component.
For example, model-viewer gives us a method for adjusting the strength of light emission for a glowing material, such as a light bulb. If I want to make the light bulb in the standing lamp glow or not depending on the status of the OH items I need a two step process. First I’ll add another basic function which sets the emission value of a input material based on whether the input is 'ON' or not.
function setMaterialEmission(matName, newEmVal) {
mViewer = document.getElementById('house-view')
if (mViewer.loaded) {
mViewer.model.getMaterialByName(matName).setEmissiveStrength((newEmVal == 'ON') ? 100 : 1)
}
}
This function can go into whatever main javascript entity you are using: external js file or inline script.
The trick to integrate the OH information in this is to remember that the content property of a script component is also parsed by the expression parser. So, I create a new separate inline script component, and in this component I do two things: use a widget expression with a string template for the script content, and set the key property.
- component: script
config:
key: =Math.random() + items.Outlet_LivingRoomLamp_OnOff.state
content: =`setMaterialEmission('bulb',
'${items.Outlet_LivingRoomLamp_OnOff.state}')`
When this script component gets rendered into the html doc, the content will either be setMaterialEmission('bulb','ON') or setMaterialEmission('bulb','OFF'). As soon as it’s rendered that script will be run and the function is called with the parameters of the current item state. The problem is, that script will only run once when it is rendered; we also need it to run whenever the item state changes. This is where the key property comes in. I’ve discussed the key property in other posts, so just a quick reminder: this property causes the Vue renderer to re-render a component anytime it changes. So, now, any time the lamp item state changes, a “new” script with the correct function call is rendered and runs. Now my light bulb turns on an off with my item (and the lamp itself, of course).
Doors
A similar example of this same technique can be used to show the status of things that move such as doors and windows. I’ve actually created two front doors in the model, one open and one closed. The one that is closed is shown because its texture alpha value is set to 1, and the door that is open has an initial alpha value of 0 so it’s invisible. Just like changing the emission value of a material using our two step javascript we can simultaneously change the alpha values of these two versions of the door based on the OPEN/CLOSED status of the door sensor.
Step one - javascript function that changes the alpha value:
function setMaterialAlpha(matName, newAlpha) {
mViewer = document.getElementById('house-view')
mViewer.model.getMaterialByName(matName).pbrMetallicRoughness.setBaseColorFactor([1,1,1,newAlpha])
}
Step two - separate script component (with a key value) that sets it’s content based on an item state:
- component: script
config:
key: =Math.random() + items.Door_FrontDoor_Status.state
content: =(items.Door_FrontDoor_Status.state == 'OPEN')?(`
setMaterialAlpha('door.front.closed',0);
setMaterialAlpha('door.front.open',1); `):(`
setMaterialAlpha('door.front.open',0);
setMaterialAlpha('door.front.closed',1); `)
Now the model door opens and closes in sync with my door sensor.
Wrap up
This turned into a long one! Thanks for reading this far. There’s a lot of model-viewer functionality that I haven’t covered in tutorial, such as animations, variant mesh materials, and rending plug-ins. You can even swap out the more light weight renderer that model-viewer uses by default bring in a full three.js library if you want even more realistic effects such as lighting. Working with most of these other features, however, would follow more or less the same principles I’ve outlined here.
