Hi there,
i would like to share my current speedtest logger and notification solution with you.
I started to log speedtest regularly when the internet connection in my old flat was going crazy.
The ISP (Unitymedia->Vodafone) wasn´t able to find any problems because the technician always checked the connection when i had no issues.
Most of the times it took days between calling the hotline, doing the speedtest while on the phone and a technician checking the connection at my flat.
I know someone that works at a subcontractor and is responsible for the network outside the houses.
Even with this connection we were not able to convince the higher ups of the company to repair parts of the network.
So i decided to go full apeshit, log speedtests every 15 minutes, store them into InfluxDB and create shiny charts for the ISP.
I made reports every month and send them to my ISP.
After two month someone had enough and allowed the technicians to do further analysis.
They found that one house around the corner was not directly connected to the network but with an old cable that comes from another house. This old (like 30 years old) cable wasn´t able to shield the core against things like radio waves or mobile phones.
They tried to correct the levels multiple times before finally sending someone that replaced the old cable with something new.
Now i´m living in a new flat with fibre-to-the-building, a personal DSLAM in the basement and a G.fast connection into my flat.
I don´t need the full logging anymore but still implemented the precautions again.
What my solution can do for you?
- Log every speedtest into InfluxDB
- Display the speedtest results in Grafana
- Get a notification when the Down-/Upload is under a predefined level
- Get the speedtest result image with this notification
List of components
- Speedtest binding by bhomeyer
- Speedtest CLI by Ookla tutorial
-
Rule to download an image and store it into an item of the type
Imageby mindstorms6
Requirements
- openHAB 3.1
- Running InfluxDB + Grafana
- speedtest binding by bhomeyer
- Optional: Running another openHAB 3.1 instance with Gigabit ethernet connection
My setup
- openHAB 3.1 Main
- Raspberry Pi 4 (4GB)
- Connected behind one switch
- Placed in the middle of the flat with a Z-Wave Stick installed
- openHAB 3.1 Speedtest
- HP Microserver Gen8 (E3-1265L V2, 16GB RAM, SSD + HDD, Active Cooling Mod)
- ESXi 6.7 U2
- Debian Buster 10.10
- Dedicated Ethernet port that is only used for this VM and directory connected to my Internet Accesspoint
Why use a speedtest instance?
Many openHAB users running their environment on raspberry Pi hardware.
Most of them are not capable to max out higher bandwidths.
| Connection | Model | Maximum Speed |
|---|---|---|
| LAN | 2 B | 94,8 Mbit/s |
| LAN | 3 B | 94,8 Mbit/s |
| LAN | 3 B+ | 224 Mbit/s |
| LAN | 4 B | 933 Mbit/s |
As you can see the Raspberry Pi 4 B is the only model capable to max out connections over 225 Mbit/s.
Getting started
Prepare a VM (optional)
I started with the installation of the speedtest instance on my ESXi.
- Download the latest Debian ISO matching your environment, i´m using the amd64 image
debian-10.10.0-amd64-xfce-CD-1.iso
- Upload the ISO file to the storage of your ESXi machine
- Prepare the VM
- I´m using 2 cores, 2GB RAM, 20GB SSD storage and the dedicated Ethernet port
- Start up the VM and make your way through
- Install → Choose your language → No GUI, only SSH → Bootloader → Restart
- Create a user named openhab in preparation for openhabian
- Wait for the restart and connect to your new VM via SSH
Prepare the VM for openhabian, install it and setup openHAB
- Switch to the superuser
su -, install sudoapt-get install sudoand add your openhab user to thesudoers-
nano /etc/sudoersand addopenhab ALL=(ALL) NOPASSWD:ALL -
Tutorial - I wasn´t able to use
visudoand had to add the openhab user manually
-
- Exit the superuser and try to execute
sudowith your openhab user - Switch to the superuser
su -and start to install openhabian- It´s already well documented here
- Start your new openHAB instance and go through the initial setup
- I like to use the openHAB Share that is currently not active after installing openhabian
sudo nano /etc/samba/smb.conf- Remove the # in front of all lines that belong to
[openHAB-share]
#=================== Custom Share Definitions ====================
[openHAB-share]
comment=openHAB combined folders
path=/srv
writeable=yes
public=no
create mask=0664
directory mask=0775
veto files = /Thumbs.db/.DS_Store/._.DS_Store/.apdisk/._*/
delete veto files = yes
Install the speedtest CLI by Ookla
Also see my other solution thread.
Based on the official install options from Ookla.
Install on Debian Buster
curl -s https://install.speedtest.net/app/cli/install.deb.sh | sudo bash
sudo apt-get install speedtest
## Accept the license agreement and GDPR as openhab
sudo -u openhab speedtest
## Test if you´re able to get JSON output
sudo -u openhab speedtest -f json
openhabian@openHAB:~ $ speedtest -f json
{"type":"result","timestamp":"2021-08-08T10:06:01Z","ping":{"jitter":0.032000000000000001,"latency":11.861000000000001},"download":{"bandwidth":39311633,"bytes":285433920,"elapsed":7305},"upload":{"bandwidth":6421818,"bytes":23171040,"elapsed":3600},"isp":"<ISP>","interface":{"internalIp":"<Your-internal-IP>","name":"eth0","macAddr":"<MAC-address>","isVpn":false,"externalIp":"<Your-external-IP>"},"server":{"id":<server-Id>,"name":"Spacken.net","location":"Hagen","country":"Germany","host":"speedtest.spacken.net","port":8080,"ip":"5.9.151.176"},"result":{"id":"<result-Id>","url":"<result-URL>"}}
Prepare the speedtest binding
- Download the latest release of the speedtest binding from Github
- 0.5 - 08.August 2021
- Connect to the openHAB Share and copy the new binding into
openhab-addons - Open your openHAB UI and login with the user you created earlier
- Open Settings → Things → Add → Speedtest binding → Add Manually
- Choose a UID, label and location if you want to
- Choose a refresh rate and enter the path
/usr/bin/speedtest- If you already have a preferred server you can enter the server id
Add the speedtest items
I´m using the text based config but you can also use the UI based config for items.
For more information please visit the readme on Github.
String spdServer {channel="speedtest:speedtest:binding:server"}
Number spdJitter {channel="speedtest:speedtest:binding:ping_jitter"}
Number spdPing {channel="speedtest:speedtest:binding:ping_latency"}
Number spdDown {channel="speedtest:speedtest:binding:download_bandwidth"}
Number spdDownBytes {channel="speedtest:speedtest:binding:download_bytes"}
Number spdDownTime {channel="speedtest:speedtest:binding:download_elapsed"}
Number spdUp {channel="speedtest:speedtest:binding:upload_bandwidth"}
Number spdUpBytes {channel="speedtest:speedtest:binding:upload_bytes"}
Number spdUpTime {channel="speedtest:speedtest:binding:upload_elapsed"}
String spdISP {channel="speedtest:speedtest:binding:isp"}
String spdIPin {channel="speedtest:speedtest:binding:interface_internalIp"}
String spdIPext {channel="speedtest:speedtest:binding:interface_externalIp"}
String spdURL {channel="speedtest:speedtest:binding:result_url"}
Switch spdTrigger {channel="speedtest:speedtest:binding:trigger_test"}
Yeah i know, using speedtest as the name of this instance wasn´t the best idea 
Add a small rule to reset the test trigger
The test trigger only fires the speedtest when switching to ON so we need to reset it after every use.
I´m sure there´s a way to do this in the new UI but i like it old fashioned.
Note: The integrated expire function could also solve this, i´m currently not aware of the correct syntax when used for items that already have a channel.
rule "Reset Speedtest Trigger"
when
Item spdTrigger changed to ON
then
Thread::sleep(2000)
spdTrigger.postUpdate(OFF)
end
Create an API token
To connect our main instance with the remote openHAB binding, we need to create an API token.
- Open the UI, login and click on the username in the bottom left corner
- Click on
Create new API token→ Enter the username and password of your openhab admin → Choose a name for the token → Click onCreate API token - Copy the API token and save it for later
Now we´re done with our speedtest instance. It doesn´t matter if you´re doing this with a VM like me or with a second raspberry Pi.
My raspberry Pi 4 isn´t able to keep up with the 300 Mbit/s because of the local traffic between other bridges (Homematic, Homebridge, Hue).
How to connect the main and speedtest instance
After getting the speedtest instance running and working we´re looking into the new remote openHAB binding.
The remote openHAB binding is only needed on the main instance that will receive something from the second instance.
Install the remote openHAB Binding
As we already prepared an API token we can directly install the remote openHAB Binding in the UI.
- Settings → Bindings → Add → remote openHAB Binding → Install
- wait for completion
- Settings → Things → Add → remote openHAB Binding → Scan
- You should see your second openHAB instance
- Depending on your environment you´ll see two things (IPv4 + IPv6)
- Click on the Thing you want to add → Click on
Show advanced- Token: Paste the token you already created on your speedtest instance
- Authenticate Anyway: Activate
Let openHAB talk…
Now it takes some time for your main instance to aquire information from the speedtest instance.
Reload the newly created Thing and switch to the Channels tab.
You should see every item on your second instance:
Create the speedtest items
Again, i´m using text based config for this.
It shouldn´t be a problem to create the custom items that are not linked to a channel.
Group gSpeedtest <"speedtest">
String spdSummary "[%s]" <"speedtest_summary"> (gPersist, gSpeedtest)
String spdServer "Server [%s]" {channel="remoteopenhab:server:speedtest:spdServer"}
Number spdJitter {channel="remoteopenhab:server:speedtest:spdJitter"}
Number spdPing "Ping [%.3f ms]" <"speedtest_ping"> (gPersist, gSpeedtest, gInflux) {channel="remoteopenhab:server:speedtest:spdPing"}
Number spdDown "Download [%.2f Mbit/s]" <"speedtest_download"> (gPersist, gSpeedtest, gInflux) {channel="remoteopenhab:server:speedtest:spdDown"}
Number spdDownBytes {channel="remoteopenhab:server:speedtest:spdDownBytes"}
Number spdDownTime {channel="remoteopenhab:server:speedtest:spdDownTime"}
Number spdDownStatus (gPersist)
Number spdUp "Upload [%.2f Mbit/s]" <"speedtest_upload"> (gPersist, gSpeedtest, gInflux) {channel="remoteopenhab:server:speedtest:spdUp"}
Number spdUpBytes {channel="remoteopenhab:server:speedtest:spdUpBytes"}
Number spdUpTime {channel="remoteopenhab:server:speedtest:spdUpTime"}
Number spdUpStatus (gPersist)
String spdISP {channel="remoteopenhab:server:speedtest:spdISP"}
String spdIPin {channel="remoteopenhab:server:speedtest:spdIPin"}
String spdIPext {channel="remoteopenhab:server:speedtest:spdIPext"}
String spdURL {channel="remoteopenhab:server:speedtest:spdURL"}
Image spdImage
Switch spdTrigger "Start now" <"speedtest_reload"> {channel="remoteopenhab:server:speedtest:spdTrigger"}
Switch spdCritMsg "Critical Message" <"speedtest"> (gPersist, gSpeedtest)
Switch spdWarnMsg "Reduced Message" <"speedtest"> (gPersist, gSpeedtest)
Switch spdLogMsg "Logging" <"speedtest"> (gPersist, gSpeedtest)
Download the result image and store it into an item
Let´s start with the rule that will download the speedtest result image and directly stores it into the spdImage item.
I played around with the HttpUtil but wasn´t able to get it working.
Good that mindstorms6 had the same challenge and found a solution.
rule "Speedtest Image Update"
when
Item spdURL changed
then
val speedImage = spdURL.state + ".png"
var userImageDataBytes = newByteArrayOfSize(0)
try {
// use the built in java URL class - pass it the url to download
val url = new URL(speedImage)
// create an output stream - we'll use it for building up our downloaded bytes
val byteStreamOutput = new ByteArrayOutputStream()
// open the url as a stream - aka - start getting stuff
val inputStream = url.openStream()
// n is a variable for tracking how much data we have read off the inputStream per loop (and how much we write to the output stream)
var n = 0
// buffer is another byte array. basically we're using it as a fixed size copy byte array
var buffer = newByteArrayOfSize(1024)
do {
// read from input stream (the data at the url) into buffer - and place how many bytes were read into n
n = inputStream.read(buffer)
if (n > 0) {
// if we read more than 0 bytes - copy them from buffer into our output stream
byteStreamOutput.write(buffer, 0, n)
}
} while (n > 0) // keep doing this until we don't have anything to read.
userImageDataBytes = byteStreamOutput.toByteArray() // assemble all the bytes we wrote into an actual byte array
} catch(Throwable t) {
logError(ruleId, "Es ist ein Problem aufgetreten: " + t.toString)
}
logDebug(ruleId, "Bild Daten erhalten")
// make a new RawType with the byte array and the mime type - the open hab type Image needs a raw type
// my images are all jpeg - so this mime type is right for me
val rawType = new RawType(userImageDataBytes, "image/jpeg")
// entry.getValue is the "ImageItem" from the map. In my case - this is BrelandImage
if (rawType.toString != spdImage.state.toString) { // if this raw type isn't the same as what's already there (the toString is a kind of accurate way to test - the raw type to string is bascially mime + size - which is a bad approximation for equality - but good enough for me
spdImage.postUpdate(rawType) // update it!
logDebug(ruleId, "Bild aktualisiert")
} else {
logDebug(ruleId, "Bilder waren identisch")
}
end
Process the speedtest data
Now comes the part where we process all the data we collected so far, do some calculation, logging and messaging
In preparation you should check all items and their values, to make sure none of them is NULL.
The three Msg items control parts of the rule.
spdWarnMsg → Will create a warning message if the speed is above the critical but under the normal speed
spdCritMsg → Will create a warning message if the speed is under the critical threshold
logMsg → Will create a report send to you via Telegram
The following thresholds come from my ISP.
var downOk = (down >= 251)
var downWarn = (down >= 163 && down <= 250)
var downCrit = (down <= 162)
var upOk = (up >= 43)
var upWarn = (up >= 26 && up <= 42)
var upCrit = (up <= 25)
ISPs in Germany are forced by the Bundesnetzagentur to tell the customer what bandwidth is avaible at maximum, normally and at minimum.
Here´s an example for my contract:
As i´m connected with FTTB and G.fast the VDSL values apply to me.
- Download:
- under 162 Mbit/s is critical
- between 163 and 250 Mbit/s is normal
- over 250 Mbit/s is ideal
- Upload:
- under 25 Mbit/s is critical
- between 26 and 42 Mbit/s is normal
- over 43 Mbit/s is ideal
Fill in the values that apply to your contract or that you want to check.
rule "Speedtest Threshold"
when
Item spdPing changed
then
timer = null
val telegramAction = getActions("telegram","telegram:telegramBot:bot")
val long bot1 = <Your-Telegram-ID> // Michael
// Options to control the warn, crit and log message
var warnMsg = (spdWarnMsg.state == ON)
var critMsg = (spdCritMsg.state == ON)
var logMsg = (spdLogMsg.state == ON)
// Build the strings for logging and messaging
val StringBuilder speedThreshold = new StringBuilder
val StringBuilder speedMessage = new StringBuilder
val critMsg1 = "Attention!\n"
val critMsg2 = "critical: "
val warnMsg1 = "Warning!\n"
val warnMsg2 = "lowered to "
val speedValue = " Mbits\n"
// Import results
var down = spdDown.state
var up = spdUp.state
var ping = spdPing.state
// Check results for threshold
var downOk = (down >= 251)
var downWarn = (down >= 163 && down <= 250)
var downCrit = (down <= 162)
var upOk = (up >= 43)
var upWarn = (up >= 26 && up <= 42)
var upCrit = (up <= 25)
// Reduce the values to full Mbits
down = down.toString.split("\\.").get(0)
up = up.toString.split("\\.").get(0)
ping = ping.toString.split("\\.").get(0)
// Update the Summary Item
val Summary = String::format("ᐁ %.3s Mbit/s ᐃ %.3s Mbit/s (%.3s ms)", down, up, ping)
spdSummary.postUpdate(Summary)
// Logging to check if the thresholds are working
if(logMsg)
{
speedThreshold.append("Results:\n")
speedThreshold.append("Download: " + down + speedValue + "\n")
speedThreshold.append("Download OK: " + downOk + "\n")
speedThreshold.append("Download Warning: " + downWarn + "\n")
speedThreshold.append("Download Critical: " + downCrit + "\n")
speedThreshold.append("Upload: " + up + speedValue + "\n")
speedThreshold.append("Upload OK: " + upOk + "\n")
speedThreshold.append("Upload Warning: " + upWarn + "\n")
speedThreshold.append("Upload Critical: " + upCrit + "\n")
logInfo(ruleId, speedThreshold.toString)
telegramAction.sendTelegram(bot1, speedThreshold.toString)
}
// Check for critital and warning threshold and build a message
if(downCrit || upCrit)
{
speedMessage.append(critMsg1)
}
else if(downWarn || upWarn)
{
speedMessage.append(warnMsg1)
}
if(downCrit)
{
speedMessage.append("Download " + critMsg2 + down + speedValue)
}
if(upCrit)
{
speedMessage.append("Upload " + critMsg2 + up + speedValue)
}
if(downWarn)
{
speedMessage.append("Download " + warnMsg2 + down + speedValue)
}
if(upWarn)
{
speedMessage.append("Upload " + warnMsg2 + up + speedValue)
}
if((downCrit || upCrit) && critMsg)
{
// Let´s wait 10 seconds to make sure the Image was updated
timer = createTimer(now.plusSeconds(10), [ |
telegramAction.sendTelegram(bot1, speedMessage.toString)
telegramAction.sendTelegramPhoto(bot1, spdImage.state.toFullString, "")
timer = null
])
}
else if((downWarn || upWarn) && warnMsg)
{
// Let´s wait 10 seconds to make sure the Image was updated
timer = createTimer(now.plusSeconds(10), [ |
telegramAction.sendTelegram(bot1, speedMessage.toString)
telegramAction.sendTelegramPhoto(bot1, spdImage.state.toFullString, "")
timer = null
])
}
// Update the status items for colored sitemap values
if(downOk)
{
spdDownStatus.postUpdate(1)
}
if(downWarn)
{
spdDownStatus.postUpdate(2)
}
if(downCrit)
{
spdDownStatus.postUpdate(3)
}
if(upOk)
{
spdUpStatus.postUpdate(1)
}
if(upWarn)
{
spdUpStatus.postUpdate(2)
}
if(upCrit)
{
spdUpStatus.postUpdate(3)
}
end
Sitemap settings
After processing all the values we´re able to display them in a sitemap.
Yes, i´m still using the basic-ui
Text item=SpeedtestSummary icon="speedtest"
{
Frame label="Results"
{
Text item=spdDown valuecolor=[SpeedtestResultDownStatus==3="red",SpeedtestResultDownStatus==2="orange",SpeedtestResultDownStatus==1="green"]
Text item=spdUp valuecolor=[SpeedtestResultUpStatus==3="red",SpeedtestResultUpStatus==2="orange",SpeedtestResultUpStatus==1="green"]
Text item=spdPing
}
Frame label="Control"
{
Switch item=spdTrigger mappings=[ON="Start"]
Switch item=spdWarnMsg mappings=[ON="active", OFF="inactive"]
Switch item=spdCritMsg mappings=[ON="active", OFF="inactive"]
Switch item=spdLogMsg mappings=[ON="active", OFF="inactive"]
}
Frame label="Statistics"
{
Image item=spdImage
}
}
Grafana charts
- I´m still working on this part as Grafana changed alot since my last implementation with oH 2.5
To do´s
- Find a way to control the execution based on an item state
- I could switch to a crown based execution and add one item that prevents a crown based execution
- Finalize the Grafana charts
- I´m still struggling to recreate the views i had because the Grafana UI changed alot
Conclusion
I hope some can use this tutorial to monitor their internet connection and maybe get your ISP to move.
I´m open for all suggestions and what could be done better.
kind regards
Michael
