Notifications with Video Attachments on iOS

Apps & Services openHAB Cloud
1

Hi there,

Today I installed the new iOS app and really wanted to try out the new features straight away.
Among other things, how to send a short video, as announced in the release notes for 4.2.
So far I have been using the ip cam binding to save a 5 sec video in tmpfs and then send it via Telegram after creating this video, but when I try to change this rule to cloud notification I fail miserably.

rule "kamera-3"
//Mail mit 5 Sekunden MP4 von Haustuer verschicken wenn bewegung
when
    Item Kamera_Haustuer_MP4_Recording changed to 0
then
    if( Mail_Cam_Eingang_Bewegung.state==ON ) {
    val videoFilePath = "file:///tmpfs/cam_front/ipcamera.mp4"
    val caption = "Bewegung an der Haustür"

    val telegramAction_1 = getActions("telegram", "telegram:telegramBot:Andreas")
    telegramAction_1.sendTelegramVideo(videoFilePath, caption)

    val telegramAction_2 = getActions("telegram", "telegram:telegramBot:Manu")
    telegramAction_2.sendTelegramVideo(videoFilePath, caption)
    Mail_Cam_Eingang_Bewegung.sendCommand(OFF)
    }
end

I was trying to place the path to the tmpfs file in the rule, but it only shows text:

sendNotification("xxx@icloud.com", "Bewegung an der Haustür", "motion", "Motion Tag", "Bewegung erkannt", "motion-id-1234", "///tmpfs/cam_front/ipcamera.mp4", null, null, null, null)

I also tried this:

sendNotification("xxx@icloud.com", "Bewegung an der Haustür", "motion", "Motion Tag", "Bewegung erkannt", "motion-id-1234", "http://192.168.1.206:8080/ipcamera/Haustuer/snapshots.mjpeg", null, null, null, null)

But this of course only works locally and gives me a live view but not the recorded video.

I cannot figure out how to fetch the last video recorded and saved to file:///tmpfs/cam_front/ipcamera.mp4, maybe somebody can assist me.

THX in advance.

BR Andreas

2

Typically web servers can only see files places in a very specific director or set of directories. This is a security feature. You don’t want anyone who can access the web server to be able to access all the files on your host machine.

For OH, that directory is $OH_CONF/html which can be accessed through http[s]://<path to OH instance>/static/<name of file>.

I’ve not experimented with video attachements on notifications but I’m almost certain that the actual video is not actually passed as part of that notification. Instead a URL is passed, and that URL needs to be accessible from the device receiving the notification.

In short, file://tmpfs/cam_front/ipcamera.mp4 is neither made accessible by being in the html folder nor is that URL a path to a file that exists on your phone.

1 Like
3

Yes, what you wrote makes perfect sense, I hadn’t thought about the web server thing.

As far as I can remember, tmpfs has advantages in that it is also ZRAM managed and therefore doesn’t put as much strain on the SD card.

Therefore, I would be reluctant to save the video files in another directory.

What would be interesting to know is why it works with the Telegram binding, since I’m accessing the videos from tmpfs, and whether this function can also be used somehow with the cloud notification.

4

The Telegram binding isn’t a web server. Bindings do not have the same limitations as the Jetty web server built into OH.

And I’m pretty certain that Telegram sends the actual video as part of the message, not just a link to it.

5

You probably know more about this than I do.

Yes, that’s right, with Telegram binding a video is sent instead of just a link.
But I want to achieve the same thing via the IOS notification, but I probably misunderstood the announcement about OH 4.2:

  • Media Attachments
    • Add images (or a short video on iOS) from an external URL or openHAB image item
6

Images and video (which i have not tried, but should work) need to be accessible via an external URL that your iOS device can access (think from a web browser), we send the URL as part of the message and the mobile app downloads the image/video data and attaches it to the message. OH Image items work similarly, but instead of downloading from an external URL, it talks to your openhab server to get the data. You can’t actually send media in a push notification as they have a max size of 4kb, its up to the mobile app to produce the media.

7

Ok that explains why this version only worked locally:

sendNotification("xxx@icloud.com", "Bewegung an der Haustür", "motion", "Motion Tag", "Bewegung erkannt", "motion-id-1234", "http://192.168.1.206:8080/ipcamera/Haustuer/snapshots.mjpeg", null, null, null, null)

But then why did this notification only open a web browser with a live video?

May I ask you to show me your rule for a notification with a image.

8

Because you set your link to a mjpeg as the onClickAction field, not as the mediaAttachmentUrl field

From openHAB Cloud Connector - System Integrations | openHAB

  • sendNotification(emailAddress, message, icon, tag, title, referenceId, onClickAction, mediaAttachmentUrl, actionButton1, actionButton2, actionButton3)

  • onClickAction: The action to be performed when the user clicks on the notification. Specified using the action syntax.

Also a mjpeg is a streaming format, not video file, so thats not going to work as an attachment, it would only support short video clips. I’m not sure how live video would be added to a push notification on iOS.

Here is your example, with the addition that the openhab logo from wikipedia would be attached to the message.

sendNotification("xxx@icloud.com", "Bewegung an der Haustür", "motion", "Motion Tag", "Bewegung erkannt", "motion-id-1234", "http://192.168.1.206:8080/ipcamera/Haustuer/snapshots.mjpeg", "https://upload.wikimedia.org/wikipedia/commons/thumb/0/07/OpenHAB_logo_2.svg/541px-OpenHAB_logo_2.svg.png", null, null, null)
9

Ah, thanks for the tip. Now I can send an image via an image item.

sendNotification("xxx@icloud.com", "Bewegung an der Haustür", "motion", null, "Bewegung erkannt", null, "ui:/basicui/app?w=00040009&sitemap=Andi", "item:Kamera_Haustuer_liveImg", null, null, null)

I guess I could no longer store the videos in tmpfs but in the HTML folder and add this folder to ZRAM by creating an entry in /etc/ztab, then it should be accessible, my videos are no longer then 5 sec, so it could work.

Another question I have concerns the onClickaction - the way I have currently integrated it, it takes me to the desired page, but the display is now like in a web browser and not like a classic sitemap in the iOS app. - Can this be adjusted?

10

The documentation shows many options for this, see Action Syntax, specifically see the ui:... actions .

11

Can you give me a little help here - I thought I had read the page thoroughly and understood it, but somehow I’m stuck.
As you’ve probably seen from the code, I’m still using the sitemap in the style of the Basic UI instead of the Main UI.

So I focused on the first part of the sentence:

  • ui:$path where $path is either /basicui/app?... for navigating sitemaps (using the native renderer) or /some/absolute/path for navigating (using the web view).

Cause it says using native render - but by doing so it looked like web view.
Of course I cloud also try the absolut path, but to be honest, I don’t know how to determine this…

12

this would load your sitemap called ‘main’

ui:/basicui/app?sitemap=main

you can use the BasicUI web client to find additional subpages page you want, then use the URL you see in your browser, so for example if i navigate my sitemap in the BasicUI in my web browser to this page :

https://openhab.local/basicui/app?w=0000&sitemap=main

I would use the relative path to open it from a notification

ui:/basicui/app?w=0000&sitemap=main

This will use the native renderer in the app, not the webview

13

First of all, thank you very much for the time you take to help me and also thank you very much for the very helpful further development of iOS notifications.
I can use these perfectly, the combination options are amazing, snapshot of a camera with the action buttons - brilliant.
For example, I always had a query sent to me via Telegram in the evening if the garage is still open and a separate message with a picture from my outside camera to see if all the cars are inside - now I can combine all of this in one message and the best thing is that it even works on the Apple Watch:

sendNotification("xxx@icloud.com", "Die Garage ist noch offen, soll sie geschlossen werden?", "garagedoor", null, "Da Hoam", null, null, "item:Cam_vor_Garage_liveImg", "Ja=command:Reply_Garagedoor:Ja", "Nein=command:Reply_Garagedoor:Nein", null)

Really a great improvement!!

But back to the point where I’m still stuck, I log in locally via web browser using the IP address to the webview of openhab and then go to the basic UI and select my sitemap:

sitemap Andi label="DA HOAM" {

And then I go to the subpage that I want to display - this gives me:

http://XXX.XXX.XXX.XXX:8080/basicui/app?w=00040009&sitemap=Andi

So I tried this code:

sendNotification("xxx@icloud.com", "Bewegung an der Haustür", "motion", null, "Bewegung erkannt", null, "ui:/basicui/app?w=00040009&sitemap=Andi", "item:Kamera_Haustuer_liveImg", null, null, null)

But still it is not natively rendered.

This is what it actually looks like when I click on the notification:

IMG_0642
IMG_06421178×2399 205 KB

This is how it should look like:

IMG_0643 2
IMG_0643 21178×1626 260 KB

Both are Screenshots from my iPhone, both are from the iOS openhab App, the first one when I click on the notification, the second one when I enter the App the normal way.

Do you have any idea where my mistake is?

THX in advance.

14

Looks like this might be a subtle bug i did not notice, the menu hamburger button on the top right should not be there when jumping into a sub-level of a sitemap. I’m not sure however why the layout of those buttons would be like that, or be different at all, i don’t see that in my test sitemaps, but i was probably not expecting any difference, so may have missed it. If you could post here, or private message me your sitemap code, i can try and replicate on my system.

Glad to hear this is (mostly) working for you.

15

Actually, your right its not native, i was cheating when i implemented this and its still using the webview, i totally forgot to change the code to render it natively. I’ll make sure thats in our next release in the coming days.

16

Thank you very much for taking care of it.

I guess I just need to update openhab as usual and the code should work - can you let me know when the update is available?

You won’t need my sitemap code anymore then? - otherwise, please let me know and I’ll send you a DM.

17

Hi,

I am also trying the new notification function on a separate OH instance because actually I am using telegram for sending pictures of my camera and I’d like to delete this.

But for me it is not working as expected. OH 4.3MS1 is running on a windows host and connected to OH cloud. My iphone is also connected with the same account.

Notifications were sent to my user and I can see them in the notification area of my iPhone but there is no push notification. Messages for OH are activated. I have deactivated and activated but without change.
Then I tried to use titles, icons or the picture url of my camera but nothing happens. Still a text message in my notification area without further details.

What is missing or is it just an Android feature and I still have to use telegram for sending pictures from my OH instance?

18

Hi Matthias,
first of all I also use an iPhone, so it is not an Android feature. I also went from telegram to iOS push notifications and apart from a few exceptions, everything now works perfectly. The exceptions refer to the action buttons and not to the image function.

But you must be aware that it is not just a change of the previous rule from telegram to notification.

my old telegram rule looked like this:

val telegramAction_1 = getActions("telegram","telegram:telegramBot:Andreas")
telegramAction_1.sendTelegramPhoto(Kamera_Haustuer_Image.state.toString, "🔔 Es klingelt.")

where the item is a String Kamera_Haustuer_Image which contains the url to an image channel

for the new rule you need to create an image item and save an image to it before executing an notification:
Image Kamera_Haustuer_liveImg

my new rule looks like this:

Kamera_Haustuer_ImgPoll.sendCommand(ON)
createTimer(now.plusNanos(700000000)) [|
Kamera_Haustuer_ImgPoll.sendCommand(OFF)
sendNotification("xxx@icloud.com", "🔔 Es klingelt.", "motion", null, "Da Hoam", null, null, "item:Kamera_Haustuer_liveImg", null, null, null)

But since you write that you already see messages under Notifications in the app, you may have already taken this into account?

So here are a few more thoughts from my side:

  • as I understand it, the notifications displayed in the app are not the list of messages that have been sent to this device but to this account
  • to test whether your device has been correctly recognized, you can log in to myopenhab.org and send test messages under devices
  • if you don’t see a device there or the test message doesn’t arrive, delete any existing devices and log in to the iOS app outside of your local WiFi so that it accesses via the cloud server and not via the local IP of your openhab server

then your device should be created and at least the test messages should arrive.

btw. I’m still on openhab 4.2.2 release build.

BR Andreas

19

Hi,

thanks for the provided steps. Will test it later on and maybe I can solve it then.

For the image thing → how do you store the image from an URL/file to an item? Do you need a rule-command for it or can you do it directly within the image definition with a static URL?

Thanks

20

See post #7 from Dan, there he mentions how to use a static url, in that case you do not need a item:

sendNotification("xxx@icloud.com", "Bewegung an der Haustür", "motion", "Motion Tag", "Bewegung erkannt", "motion-id-1234", "http://192.168.1.206:8080/ipcamera/Haustuer/snapshots.mjpeg", "https://upload.wikimedia.org/wikipedia/commons/thumb/0/07/OpenHAB_logo_2.svg/541px-OpenHAB_logo_2.svg.png", null, null, null)

I tested it for my self and it worked.

But in my case I want a actual image from my camera, for that I use the ip-cam binding, there you can link an image item to every camera and a poll switch, if activated it saves an actual image to the image item every x sec (according to config), that’s why I use an timer of 700 milisec to switch on and off the image-poll item, which is sufficient to me, so after the timer ends the rule grabs the image saved to the image item and sends it to my iPhone.
Works like a charm.