Speedtest CLI by Ookla - Internet Up-/Downlink Measurement Integration

Tags: #<Tag:0x00007f5c9f8fe690> #<Tag:0x00007f5c9f8fe5c8> #<Tag:0x00007f5c9f8fe460>

Based on the solution from ThomDietrich i made a migration to the new official Speedtest CLI by Ookla.

Why use the Speedtest CLI by Ookla instead of the speedtest-cli by Sivel?

  • The speedtest-cli by Sivel uses very old technology from Ookla that is outdated
  • The maximum speed of the speedtest-cli is limited
  • The speedtest-cli is heavily based on the performance of the host system
  • The Speedtest CLI by Ookla exports clean JSON to make it machine readable

What functionality does the Speedtest CLI by Ookla offers?

  • Fully supported by Ookla with the latest speedtest technology for best results
  • Output of a timestamp instead of the own timestamp by openHAB
  • Output of Jitter and Ping
  • Output of Bandwidth, transferred bytes and elapsed time for Down-/Upload
  • Output of the used interface with Internal/External IP, MAC Address and interface name
  • Output of the used Server and location
  • Output of the result id and result URL

What functionality does the integration offers?

  • Execute a speedtest time based or directly
  • Show summarized results of the speedtest
  • Export the data to a database and visualize with Grafana
  • Start the speedtest from another oH instance or from a rule

Requirements

  • Uninstall the speedtest-cli if already installed
  • Install the Exec Binding
    • Exec Binding 2.5.2: You need to add the full path to your exec.whiteliste file!
  • Install the JSONPatch Transformation
  • Install the Speedtest CLI by Ookla
  • Test the speedtest before you continue
  • Build the rules and test

Linux

Uninstall the speedtest-cli from raspbian

I had some problems to completely uninstall the speedtest-cli and there´s no real guide on how to uninstall.

# If you installed the speedtest-cli with apt-get
sudo apt-get remove --auto-remove speedtest-cli
# If you installed the speedtest-cli with pip
sudo pip uninstall speedtest-cli

# Check for speedtest and speedtest-cli in /usr/local/bin/ and remove them
sudo rm /usr/local/bin/speedtest
sudo rm /usr/local/bin/speedtest-cli

# Check for speedtest or speedtest-cli parts in /usr/local/lib/python2.7/dist-packages and delete them
# Depending on the python version installed

# Just to be sure
apt-get update
reboot

Install the speedtest package

Based on the official install options from Ookla.

sudo apt-get install gnupg1 apt-transport-https dirmngr
export INSTALL_KEY=379CE192D401AB61
# Ubuntu versions supported: xenial, bionic
# Debian versions supported: jessie, stretch, buster
export DEB_DISTRO=$(lsb_release -sc)
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys $INSTALL_KEY
echo "deb https://ookla.bintray.com/debian ${DEB_DISTRO} main" | sudo tee  /etc/apt/sources.list.d/speedtest.list
sudo apt-get update
# Other non-official binaries will conflict with Speedtest CLI
sudo apt-get install speedtest

Windows

Download and install the speedtest.exe

Thanks to @KidSquid for the Windows details!

Please download the latest Speedtest CLI from the Ookla website:

Choose or create a new folder where your Speedtest CLI runs and place the downloaded speedtest.exe in this folder.
Example: D:/Tools/Speedtest/

Please run the speedtest from your commandline and use the user that runs openHAB before continuing to the next steps!
You may need to accept the GDPR and license when running speedtest for the first time.
This should be saved after the first execution.

Test the Speedtest CLI

Just type speedtest and watch the speedtest working.
Note: If you´re not using the openhab user, please use sudo -u openhab speedtest
You may need to accept the GDPR and license when running speedtest for the first time.
This should be saved after the first execution.

[13:02:28] openhab@oH-Speedtest:/$ speedtest

   Speedtest by Ookla

     Server: InoHost - Düsseldorf (id = 21467)
        ISP: Unitymedia
    Latency:     9.34 ms   (2.18 ms jitter)
   Download:   578.42 Mbps (data used: 621.6 MB)                               
     Upload:    52.74 Mbps (data used: 31.5 MB)                               
Packet Loss:     0.0%
 Result URL: https://www.speedtest.net/result/c/###

And start another test with the argument -f json or -f json-pretty to check if everything works fine.
The result should look like the following example and not start with {"type":"log","timestamp":"2020-03-12T16:02:56Z","message":"Error: [-2] Protocol error: Did not receive HELLO","level":"error"} as this would stop the rule from working.
Maybe you need to define a server with -s <ID>, to get a list of nearly server just use -L
I had to test some servers to find one that gives me reliable results.

[13:04:56] openhab@oH-Speedtest:/$ speedtest -f json-pretty
{
    "type": "result",
    "timestamp": "2020-03-12T12:05:14Z",
    "ping": {
        "jitter": 0.91000000000000003,
        "latency": 8.4429999999999996
    },
    "download": {
        "bandwidth": 114371030,
        "bytes": 1202757605,
        "elapsed": 10815
    },
    "upload": {
        "bandwidth": 6385055,
        "bytes": 25651164,
        "elapsed": 4026
    },
    "packetLoss": 0,
    "isp": "Unitymedia",
    "interface": {
        "internalIp": "###",
        "name": "###",
        "macAddr": "###",
        "isVpn": false,
        "externalIp": "###"
    },
    "server": {
        "id": 17392,
        "name": "myLoc managed IT AG",
        "location": "Dusseldorf",
        "country": "Germany",
        "host": "ookla.myloc.de",
        "port": 8080,
        "ip": "2001:4ba1:e001:103::1"
    },
    "result": {
        "id": "###",
        "url": "###"
    }
}

As you can see my speedtest changed the server between the two tests and that results in two different bandwiths for my download.
Another thing you can see is the different output format for the bandwith.
While the human-readable output shows the bandwidth as Mbps the JSON output just shows the bandwidth in bits. This output can´t be changed and we need to divide the result by 125000 to get the bandwidth in Mbps.

openHAB files

Icons


I made an icon set that fits better into my openHAB sitemaps and you´re free to use it: Download
Extract all icons to the icons folder $OPENHAB_CONF/icons/classic/

Items (speedtest.items)

Group gSpeedtest <"speedtest">
Group gSpeedChart
String SpeedtestCharts

String      SpeedtestSummary        "Speedtest [%s]"                                            <"speedtest_summary">   (gSpeedtest, gPersist)
Number      SpeedtestResultPing     "Ping [%.3f ms]"                                            <"speedtest_ping">      (gSpeedtest, gSpeedChart)
Number      SpeedtestResultDown     "Download [%.2f Mbit/s]"                                    <"speedtest_download">  (gSpeedtest, gSpeedChart)
Number      SpeedtestResultUp       "Upload [%.2f Mbit/s]"                                      <"speedtest_upload">    (gSpeedtest, gSpeedChart)
String      SpeedtestRunning        "Speedtest running ... [%s]"                                  <"speedtest_run">       (gSpeedtest)
Switch      SpeedtestRerun          "Manuell starten"                                           <"speedtest_reload">    (gSpeedtest)
DateTime    SpeedtestResultDate     "Last Run [%1$td.%1$tm.%1$tY, %1$tH:%1$tM]"   <"speedtest_date">      (gSpeedtest, gPersist)
String      SpeedtestResultError    "Error Message [%s]"                                        <"speedtest_error">     (gSpeedtest, gPersist)

String      SpeedtestResultImage    "Bild"

Sitemap (part of my default.sitemap)

			Text item=SpeedtestSummary icon="speedtest"
			{
				Frame label="Ergebnisse"
				{
					Text item=SpeedtestResultDown
					Text item=SpeedtestResultUp
					Text item=SpeedtestResultPing
				}
				Frame label="Steuerung"
				{
					Text item=SpeedtestResultDate
					Text item=SpeedtestRunning label="Speedtest [%s]" visibility=[SpeedtestRunning != "-"]
					Text item=SpeedtestResultError visibility=[SpeedtestRunning == "Fehler"]
					Switch item=SpeedtestRerun mappings=[ON="Start"]
				}
				Frame label="Statistik"
				{
					Image label="Letzte Messung" item=SpeedtestResultImage
                                        Webview url="/static/chart.html" height=12 icon="speedtest_summary"
				}
			}

I´m using InfluxDB+Grafana to visualize the results and have them stored to have some proofs for my ISP when the speed is far under the agreed bandwidth.

Linux rule file (speedtest.rules)

val String ruleId = "Speedtest"
val Number calc = 125000 // Converting from bits to Mbits

rule "Speedtest init"

when

    System started

then

    createTimer(now.plusSeconds(195))
    [|
        if(SpeedtestRerun.state == NULL)
        {
            SpeedtestRerun.postUpdate(OFF)
        }

        if(SpeedtestRunning.state == NULL)
        {
            SpeedtestRunning.postUpdate("-")
        }

        if(SpeedtestSummary.state == NULL || SpeedtestSummary.state == "")
        {
            SpeedtestSummary.postUpdate("⁉ (unbekannt)")
        }
    ]

end

rule "Speedtest"

when

    Time cron "0 0/15 * * * ?" or
    Item SpeedtestRerun changed from OFF to ON or
    Item SpeedtestRerun received command ON

then

    //logInfo(ruleId, "--> speedtest executed...")
    SpeedtestRunning.postUpdate("Messung läuft...")

    // execute the script, you may have to change the path depending on your system
    // Please use -f json and not -f json-pretty
    val speedtestExecute = "speedtest -f json"
    var speedtestCliOutput = executeCommandLine(speedtestExecute, 120*1000)

    // for debugging:
    // var String speedtestCliOutput = "Ping: 43.32 ms\nDownload: 21.64 Mbit/s\nUpload: 4.27 Mbit/s"
    //logInfo(ruleId, "--> speedtest output:\n" + speedtestCliOutput + "\n\n")
    SpeedtestRunning.postUpdate("Datenauswertung...")

    // starts off with a fairly simple error check, should be enough to catch all problems I can think of
    if (speedtestCliOutput.startsWith("{\"type\":\"result\",") && speedtestCliOutput.endsWith("}}"))
    {
        var ping = Float::parseFloat(transform("JSONPATH", "$.ping.latency", speedtestCliOutput))
        SpeedtestResultPing.postUpdate(ping)

        var float down = Float::parseFloat(transform("JSONPATH", "$.download.bandwidth", speedtestCliOutput))
        down = (down / calc)
        SpeedtestResultDown.postUpdate(down)

        var float up = Float::parseFloat(transform("JSONPATH", "$.upload.bandwidth", speedtestCliOutput))
        up = (up / calc)
        SpeedtestResultUp.postUpdate(up)

        var String url = transform("JSONPATH", "$.result.url", speedtestCliOutput)
        val img = url + ".png"
        SpeedtestResultImage.postUpdate(img)

        SpeedtestSummary.postUpdate(String::format("ᐁ  %.1f Mbit/s  ᐃ %.1f Mbit/s (%.0f ms)", down, up, ping))

        SpeedtestRunning.postUpdate("-")

        // update timestamp for last execution
        val DateTimeType ResultDate = DateTimeType.valueOf(transform("JSONPATH", "$.timestamp", speedtestCliOutput))
        SpeedtestResultDate.postUpdate(ResultDate)
    }
    else
    {
        SpeedtestResultPing.postUpdate(0)
        SpeedtestResultDown.postUpdate(0)
        SpeedtestResultUp.postUpdate(0)
        SpeedtestSummary.postUpdate("(unbekannt)")
        SpeedtestRunning.postUpdate("Fehler")

        logError(ruleId, "--> speedtest failed. Output:\n" + speedtestCliOutput + "\n\n")
    }

    SpeedtestRerun.postUpdate(OFF)

end

Windows rule file (speedtest.rules)

Please check the path on line 47 and change it to match your system.

val String ruleId = "Speedtest"
val Number calc = 125000 // Converting from bits to Mbits

rule "Speedtest init"

when

    System started

then

    createTimer(now.plusSeconds(195))
    [|
        if(SpeedtestRerun.state == NULL)
        {
            SpeedtestRerun.postUpdate(OFF)
        }

        if(SpeedtestRunning.state == NULL)
        {
            SpeedtestRunning.postUpdate("-")
        }

        if(SpeedtestSummary.state == NULL || SpeedtestSummary.state == "")
        {
            SpeedtestSummary.postUpdate("⁉ (unbekannt)")
        }
    ]

end

rule "Speedtest"

when

    Time cron "0 0/15 * * * ?" or
    Item SpeedtestRerun changed from OFF to ON or
    Item SpeedtestRerun received command ON

then

    //logInfo(ruleId, "--> speedtest executed...")
    SpeedtestRunning.postUpdate("Measurement in Progress...")

    // execute the script, you may have to change the path depending on your system
    // Please use -f json and not -f json-pretty
    val speedtestExecute = "D:/Tools/Speedtest/speedtest.exe -f json"
    var speedtestCliOutput = executeCommandLine(speedtestExecute, 120*1000)

    // for debugging:
    // var String speedtestCliOutput = "Ping: 43.32 ms\nDownload: 21.64 Mbit/s\nUpload: 4.27 Mbit/s"
    //logInfo(ruleId, "--> speedtest output:\n" + speedtestCliOutput + "\n\n")
    SpeedtestRunning.postUpdate("Data Analysis...")

    // starts off with a fairly simple error check, should be enough to catch all problems I can think of
    if (speedtestCliOutput.startsWith("{\"type\":\"result\",") && speedtestCliOutput.endsWith("}}"))
    {
        var ping = Float::parseFloat(transform("JSONPATH", "$.ping.latency", speedtestCliOutput))
        SpeedtestResultPing.postUpdate(ping)

        var float down = Float::parseFloat(transform("JSONPATH", "$.download.bandwidth", speedtestCliOutput))
        down = (down / calc)
        SpeedtestResultDown.postUpdate(down)

        var float up = Float::parseFloat(transform("JSONPATH", "$.upload.bandwidth", speedtestCliOutput))
        up = (up / calc)
        SpeedtestResultUp.postUpdate(up)

        var String url = transform("JSONPATH", "$.result.url", speedtestCliOutput)
        val img = url + ".png"
        SpeedtestResultImage.postUpdate(img)

        SpeedtestSummary.postUpdate(String::format("ᐁ  %.1f Mbit/s  ᐃ %.1f Mbit/s (%.0f ms)", down, up, ping))

        SpeedtestRunning.postUpdate("-")

        // update timestamp for last execution
        val DateTimeType ResultDate = DateTimeType.valueOf(transform("JSONPATH", "$.timestamp", speedtestCliOutput))
        SpeedtestResultDate.postUpdate(ResultDate)
    }
    else
    {
        SpeedtestResultPing.postUpdate(0)
        SpeedtestResultDown.postUpdate(0)
        SpeedtestResultUp.postUpdate(0)
        SpeedtestSummary.postUpdate("(unknown)")
        SpeedtestRunning.postUpdate("Error")

        logError(ruleId, "--> speedtest failed. Output:\n" + speedtestCliOutput + "\n\n")
    }

    SpeedtestRerun.postUpdate(OFF)

end

Using a different openHAB instance for the speedtest

As i´m using two different openHAB instances and build my rule to transfer the values to my main instance.
The main instance is running on a raspberry pi 4 and isn´t able to measure the full speed of my bandwidth, that´s why i created a virtual machine with raspbian running with an own Gigabit port.

My current rule:

Summary
val String ruleId = "Speedtest"
val Number calc = 125000 // Zur Umwandlung von Bits in Mbits

// Speedtest init placeholder

rule "Speedtest"

when

    //Time cron "0 0 5,13 * * ?" or
    Time cron "0 0/15 * * * ?" or
    Item SpeedtestRerun changed from OFF to ON or
    Item SpeedtestRerun received command ON

then

    //logInfo(ruleId, "--> speedtest executed...")
    SpeedtestRunning.postUpdate("Messung läuft...")
    sendHttpPutRequest("http://192.168.2.10:8080/rest/items/SpeedtestRunning/state", "text/plain", "Messung läuft...")

    // execute the script, you may have to change the path depending on your system
    val speedtestExecute = "speedtest -f json -s 28624"
    var speedtestCliOutput = executeCommandLine(speedtestExecute, 120*1000)

    // for debugging:
    // var String speedtestCliOutput = "Ping: 43.32 ms\nDownload: 21.64 Mbit/s\nUpload: 4.27 Mbit/s"
    //logInfo(ruleId, "--> speedtest output:\n" + speedtestCliOutput + "\n\n")

    SpeedtestRunning.postUpdate("Datenauswertung...")
    sendHttpPutRequest("http://192.168.2.10:8080/rest/items/SpeedtestRunning/state", "text/plain", "Datenauswertung...")

    // starts off with a fairly simple error check, should be enough to catch all problems I can think of
    if (speedtestCliOutput.startsWith("{\"type\":\"result\",") && speedtestCliOutput.endsWith("}}"))
    {
        var ping = Float::parseFloat(transform("JSONPATH", "$.ping.latency", speedtestCliOutput))
        //logInfo(ruleId, "Ping " + ping)
        SpeedtestResultPing.postUpdate(ping)
        sendHttpPutRequest("http://192.168.2.10:8080/rest/items/SpeedtestResultPing/state", "text/plain", ping.toString)

        var float down = Float::parseFloat(transform("JSONPATH", "$.download.bandwidth", speedtestCliOutput))
        down = (down / calc)
        //logInfo(ruleId, "Download " + down)
        SpeedtestResultDown.postUpdate(down)
        sendHttpPutRequest("http://192.168.2.10:8080/rest/items/SpeedtestResultDown/state", "text/plain", String::format("%s",down))

        var float up = Float::parseFloat(transform("JSONPATH", "$.upload.bandwidth", speedtestCliOutput))
        up = (up / calc)
        //logInfo(ruleId, "Upload " + up)
        SpeedtestResultUp.postUpdate(up)
        sendHttpPutRequest("http://192.168.2.10:8080/rest/items/SpeedtestResultUp/state", "text/plain", String::format("%s",up))

        var String url = transform("JSONPATH", "$.result.url", speedtestCliOutput)
        val img = url + ".png"
        sendHttpPutRequest("http://192.168.2.10:8080/rest/items/SpeedtestResultImage/state", "text/plain", String::format("%s",img))

        SpeedtestSummary.postUpdate(String::format("ᐁ  %.1f Mbit/s  ᐃ %.1f Mbit/s (%.0f ms)", down, up, ping))
        sendHttpPutRequest("http://192.168.2.10:8080/rest/items/SpeedtestSummary/state", "text/plain", String::format("ᐁ  %.1f Mbit/s  ᐃ %.1f Mbit/s (%.0f ms)", down, up, ping))

        SpeedtestRunning.postUpdate("-")
        sendHttpPutRequest("http://192.168.2.10:8080/rest/items/SpeedtestRunning/state", "text/plain", "-")

        // update timestamp for last execution
        val DateTimeType ResultDate = DateTimeType.valueOf(transform("JSONPATH", "$.timestamp", speedtestCliOutput))
        SpeedtestResultDate.postUpdate(ResultDate)
        sendHttpPutRequest("http://192.168.2.10:8080/rest/items/SpeedtestResultDate/state", "text/plain", ResultDate.toString)

        //logInfo(ruleId, "--> speedtest finished.")
    }
    else
    {
        SpeedtestResultPing.postUpdate(0)
        sendHttpPutRequest("http://192.168.2.10:8080/rest/items/SpeedtestResultPing/state", "text/plain", 0)
        SpeedtestResultDown.postUpdate(0)
        sendHttpPutRequest("http://192.168.2.10:8080/rest/items/SpeedtestResultDown/state", "text/plain", 0)
        SpeedtestResultUp.postUpdate(0)
        sendHttpPutRequest("http://192.168.2.10:8080/rest/items/SpeedtestResultUp/state", "text/plain", 0)
        SpeedtestSummary.postUpdate("(unbekannt)")
        sendHttpPutRequest("http://192.168.2.10:8080/rest/items/SpeedtestSummary/state", "text/plain", "Error")
        SpeedtestRunning.postUpdate("Fehler")
        sendHttpPutRequest("http://192.168.2.10:8080/rest/items/SpeedtestRunning/state", "text/plain", "Fehler")

        sendHttpPutRequest("http://192.168.2.10:8080/rest/items/SpeedtestResultError/state", "text/plain", speedtestCliOutput.toString)
        logError(ruleId, "--> speedtest failed. Output:\n" + speedtestCliOutput + "\n\n")
    }

    SpeedtestRerun.postUpdate(OFF)

end

Threshold checking

I´ve build another rule to check for user defined thresholds and send Telegram messages when the bandwidth falls under a warning or critical threshold.

The rule has three values to control the rule:
warnMsg This message will be send when the downWarn or upWarn threshold matched
critMsg This message will be send when the downCrit or upWarn threshold matched
logMsg This will build a logInfo with the threshold checks

Items

Number      SpeedtestResultDownStatus "Download Status"
Number      SpeedtestResultUpStatus "Upload Status"

Sitemap

Text item=SpeedtestResultDown valuecolor=[SpeedtestResultDownStatus==3="red",SpeedtestResultDownStatus==2="orange",SpeedtestResultDownStatus==1="green"]
Text item=SpeedtestResultUp valuecolor=[SpeedtestResultUpStatus==3="red",SpeedtestResultUpStatus==2="orange",SpeedtestResultUpStatus==1="green"]

Rule (attach it to the speedtest.rules from above)

val telegramAction = getActions("telegram","telegram:telegramBot:bot")
val long bot1 = <BotID>

rule "Speedtest Threshold"

when

    Item SpeedtestRunning changed to "-"

then

    // Options to control the warn, crit and log message
    var warnMsg = 0
    var critMsg = 1
    var logMsg = 0

    // Build the strings for logging and messaging
    val StringBuilder speedThreshold = new StringBuilder
    val StringBuilder speedMessage = new StringBuilder
    // Messsage will be build like: xMsg1 + Download/Upload + xMsg2 + +Download/Upload speed + speedValue
    // Attention! Download reduced to 100 Mbps
    val critMsg1 = "Attention!\n"
    val critMsg2 = "critical: "
    val warnMsg1 = "Warning!\n"
    val warnMsg2 = "reduced to "
    val speedValue = " Mbps\n"

    // Import results
    var down = SpeedtestResultDown.state
    var up = SpeedtestResultUp.state

    // Check results for threshold
    // Please insert your thresholds, i used the thresholds of my ISP
    var downOk = (down >= 851)
    var downWarn = (down >= 501 && down <= 850)
    var downCrit = (down <= 500)
    var upOk = (up >= 36)
    var upWarn = (up >= 16 && up <= 35)
    var upCrit = (up <= 15)
    // Reduce the values to full Mbits
    down = down.toString.split("\\.").get(0)
    up = up.toString.split("\\.").get(0)

    // Logging to check if the thresholds are working
    if(logMsg == 1)
    {
        speedThreshold.append("Results:\n")
        speedThreshold.append("Download OK: " + downOk + "\n")
        speedThreshold.append("Download Warning: " + downWarn + "\n")
        speedThreshold.append("Download Critical: " + downCrit + "\n")
        speedThreshold.append("Upload OK: " + upOk + "\n")
        speedThreshold.append("Upload Warning: " + upWarn + "\n")
        speedThreshold.append("Upload Critical: " + upCrit + "\n")

        logInfo(ruleId, speedThreshold.toString)
    }

    // Check for critital and warning threshold and build a message
    if(downCrit || upCrit)
    {
        speedMessage.append(critMsg1)
    }
    else if(downWarn || upWarn)
    {
        speedMessage.append(warnMsg1)
    }
    if(downCrit)
    {
        speedMessage.append("Download " + critMsg2 + down + speedValue)
    }
    if(upCrit)
    {
        speedMessage.append("Upload " + critMsg2 + up + speedValue)
    }
    if(downWarn)
    {
        speedMessage.append("Download " + warnMsg2 + down + speedValue)
    }
    if(upWarn)
    {
        speedMessage.append("Upload " + warnMsg2 + up + speedValue)
    }
    if((downCrit || upCrit) && critMsg == 1)
    {
        telegramAction.sendTelegram(bot1, speedMessage.toString)
    }
    else if((downWarn || upWarn) && warnMsg == 1)
    {
        telegramAction.sendTelegram(bot1, speedMessage.toString)
    }

    // Update the status items for colored sitemap values
    if(downOk)
    {
        SpeedtestResultDownStatus.postUpdate(1)
    }
    if(downWarn)
    {
        SpeedtestResultDownStatus.postUpdate(2)
    }
    if(downCrit)
    {
        SpeedtestResultDownStatus.postUpdate(3)
    }
    if(upOk)
    {
        SpeedtestResultUpStatus.postUpdate(1)
    }
    if(upWarn)
    {
        SpeedtestResultUpStatus.postUpdate(2)
    }
    if(upCrit)
    {
        SpeedtestResultUpStatus.postUpdate(3)
    }

end

My Environment

  • Raspberry Pi4 4GB
  • Raspbian GNU/Linux 10 (buster)
  • openHAB 2.5.2-1
  • Exec Binding 2.5.1
  • openjdk version “1.8.0_222”
  • OpenJDK Runtime Environment (Zulu8.40.0.178-CA-linux_aarch32hf) (build 1.8.0_222-b178)
  • OpenJDK Client VM (Zulu8.40.0.178-CA-linux_aarch32hf) (build 25.222-b178, mixed mode, Evaluation)

Conclusion

Why did i changed from the speedtest-cli by Sivel to the official Speedtest CLI by Ookla?
Because the speedtest-cli is using old technology that isn´t able to measure my available bandwith and because the tests weren´t consistent.

Using speedtest-cli

Using Speedtest CLI by Ookla

Next steps

  • I´m open for suggestions
  • Windows users to test how it works thanks KidSquid!
  • Use the timestamp from the json output

kind regards
Michael

9 Likes

Hi Michael,
thank you for detailed guide.
I had to modify speedtestExecute to accept license and gdpr

val speedtestExecute = “speedtest --accept-license --accept-gdpr -f json”

but I still get 2 json objects as result:

[17:02:55] openhabian@openhab:~$ speedtest --accept-license --accept-gdpr -f json
{"type":"log","timestamp":"2020-03-12T16:02:56Z","message":"Error: [-2] Protocol error: Did not receive HELLO","level":"error"}
{"type":"result","timestamp":"2020-03-12T16:03:17Z","ping":{"jitter":5.0119999999999996,"latency":14.154999999999999},"download":{"bandwidth":27342982,"bytes":302438216,"elapsed":11310},"upload":{"bandwidth":2503195,"bytes":10272648,"elapsed":4100},"packetLoss":0,"isp":"Unitymedia","interface":{"internalIp":"192.168.178.###","name":"eth0","macAddr":"####","isVpn":false,"externalIp":"####"},"server":{"id":25863,"name":"NOREST-TELECOM","location":"Hatten","country":"France","host":"speedtest.norest-telecom.fr","port":8080,"ip":"185.239.199.150"},"result":{"id":"####","url":"https://www.speedtest.net/result/c/######"}}

do you have a hint to get rid of this protocol error or parse 2 json objects?

Version: speedtest 1.0.0.2
Running openhab 2.5.0 release build

Thank you
/Sergej

Hi Sergej

did you started the speedtest before? When i did, i only had to accept the license + gdpr once and that decision was saved.

The {"type":"log", only appears if something went wrong, in this case "message":"Error: [-2] Protocol error: Did not receive HELLO",
I´ve to admit that i don´t know what this means.

Does this error always appears?

I added the -f json before the --accept Syntax. I did get no errors as you.

Jan

did you started the speedtest before? When i did, i only had to accept the license + gdpr once and that decision was saved.

starting speedtest from ssh session is different as start it from openhab rules I guess.

Does this error always appears?

Yes, also after changing to speedtest -f json --accept-license --accept-gdpr
/Sergej

Not for me.
What platform does your openhab instance run on?

rpi4 openhabian
I suppose I need to open some ports…

I´m also running an rpi4 with openhabian.
Strange, as i said. I only had to accept the license and gdpr once.

Ok guys, I understand now what is the issue.
If you start speedtest without selecting a server it grab any server from server list (type speedtest -L to get the list).
In my situation it took a server that is not compatible with the requests send by speedtest app.
[15:36:55] openhabian@openhab:~$ speedtest -L
Closest servers:

ID  Name                           Location             Country
==============================================================================
 14156  STARFACE GmbH                  Karlsruhe            Germany
 18613  TelemaxX Telekommunikation GmbH Karlsruhe            Germany
  5829  Bratschnitzel.de               Karlsruhe            Germany
 27612  Rabe.Systems                   Karlsruhe            Germany
 10709  bc-networks                    Ludwigsburg          Germany
 17811  bc-networks                    Remseck              Germany
 25863  NOREST-TELECOM                 Hatten               France
 16633  Stadtwerke Schorndorf GmbH     Schorndorf           Germany
 10291  TWL-KOM                        Ludwigshafen         Germany
 28818  Pfalzkom GMBH                  Ludwigshafen         Germany

So I selected special server from speedtest -L and now perform:
speedtest --accept-license --accept-gdpr -s 18613 -f json and get correct result without any log messages in between.

As Michael mentioned add --accept-license --accept-gdpr is needed only once for first time.

btw. the position of parameters like -f json is not important in linux app see getopt() function.

/Sergej

1 Like

Thanks for clarifying!
I directly choosed a server as only this server offers me full speed almost every time.

Hi Michael,
Would be awesome if you attach chart.html or link to “how to” for beginners :wink:
Again, many thanks for guide and sharing your work.

If you use rrd4j
instead of Webview use:
Chart item=gSpeedtest service="rrd4j" period=h refresh=15000
image
Note: only Number items are displayed in Chart if you add items to persistency group

/Sergej

It´s all based on the InfluxDB+Grafana guide i already linked :slight_smile:

Just a quick note to let you know I have this up and running on Windows…it was fairly easy…

I downloaded the appropriate version of the CLI for Windows

As of now I have removed the charting as I wanted to get the basic functionality working.

My Items:

Group gSpeedtest <"speedtest">

String      SpeedtestSummary        "Speedtest [%s]"                                            <"speedtest_summary">   (gSpeedtest)
Number      SpeedtestResultPing     "Ping [%.3f ms]"                                            <"speedtest_ping">      (gSpeedtest)
Number      SpeedtestResultDown     "Download [%.2f Mbit/s]"                                    <"speedtest_download">  (gSpeedtest)
Number      SpeedtestResultUp       "Upload [%.2f Mbit/s]"                                      <"speedtest_upload">    (gSpeedtest)
String      SpeedtestRunning        "Speedtest Running ... [%s]"                                <"speedtest_run">       (gSpeedtest)
Switch      SpeedtestRerun          "Start Manually"                                            <"speedtest_reload">    (gSpeedtest)
DateTime    SpeedtestResultDate     "Last Run [%1$td.%1$tm.%1$tY, %1$tH:%1$tM Uhr]"             <"speedtest_date">      (gSpeedtest)
String      SpeedtestResultError    "Error Message [%s]"                                        <"speedtest_error">     (gSpeedtest)

String      SpeedtestResultImage    "Image"

My Rules:

val String ruleId = "Speedtest"
val Number calc = 125000 // Converting from bits to Mbits

rule "Speedtest init"

when

    System started

then

    createTimer(now.plusSeconds(195))
    [|
        if(SpeedtestRerun.state == NULL)
        {
            SpeedtestRerun.postUpdate(OFF)
        }

        if(SpeedtestRunning.state == NULL)
        {
            SpeedtestRunning.postUpdate("-")
        }

        if(SpeedtestSummary.state == NULL || SpeedtestSummary.state == "")
        {
            SpeedtestSummary.postUpdate("⁉ (unbekannt)")
        }
    ]

end

rule "Speedtest"

when

    Time cron "0 0/15 * * * ?" or
    Item SpeedtestRerun changed from OFF to ON or
    Item SpeedtestRerun received command ON

then

    //logInfo(ruleId, "--> speedtest executed...")
    SpeedtestRunning.postUpdate("Measurement in Progress...")

    // execute the script, you may have to change the path depending on your system
    // Please use -f json and not -f json-pretty
    val speedtestExecute = "D:/Tools/Speedtest/speedtest.exe -f json"
    var speedtestCliOutput = executeCommandLine(speedtestExecute, 120*1000)

    // for debugging:
    // var String speedtestCliOutput = "Ping: 43.32 ms\nDownload: 21.64 Mbit/s\nUpload: 4.27 Mbit/s"
    //logInfo(ruleId, "--> speedtest output:\n" + speedtestCliOutput + "\n\n")
    SpeedtestRunning.postUpdate("Data Analysis...")

    // starts off with a fairly simple error check, should be enough to catch all problems I can think of
    if (speedtestCliOutput.startsWith("{\"type\":\"result\",") && speedtestCliOutput.endsWith("}}"))
    {
        var ping = Float::parseFloat(transform("JSONPATH", "$.ping.latency", speedtestCliOutput))
        SpeedtestResultPing.postUpdate(ping)

        var float down = Float::parseFloat(transform("JSONPATH", "$.download.bandwidth", speedtestCliOutput))
        down = (down / calc)
        SpeedtestResultDown.postUpdate(down)

        var float up = Float::parseFloat(transform("JSONPATH", "$.upload.bandwidth", speedtestCliOutput))
        up = (up / calc)
        SpeedtestResultUp.postUpdate(up)

        var String url = transform("JSONPATH", "$.result.url", speedtestCliOutput)
        val img = url + ".png"
        SpeedtestResultImage.postUpdate(img)

        SpeedtestSummary.postUpdate(String::format("ᐁ  %.1f Mbit/s  ᐃ %.1f Mbit/s (%.0f ms)", down, up, ping))

        SpeedtestRunning.postUpdate("-")

        // update timestamp for last execution
        val String ResultDate = "" + new DateTimeType()
        SpeedtestResultDate.postUpdate(ResultDate)
    }
    else
    {
        SpeedtestResultPing.postUpdate(0)
        SpeedtestResultDown.postUpdate(0)
        SpeedtestResultUp.postUpdate(0)
        SpeedtestSummary.postUpdate("(unknown)")
        SpeedtestRunning.postUpdate("Error")

        logError(ruleId, "--> speedtest failed. Output:\n" + speedtestCliOutput + "\n\n")
    }

    SpeedtestRerun.postUpdate(OFF)

end

On line 47 note how the path to the .exe is created…you also must add the entire path to the executable in the exec.whitelist file if you are running on 2.5.2

My rendered sitemap

Only things left to do are:

1, Change time and date display :heavy_check_mark:
2. Figure out why my Gigabit services is only showing 246Mb on the download. :heavy_check_mark:
3. Make HabPanel widget :heavy_check_mark:

Squid :squid:

1 Like

Try other servers and add -s <ID> to your rule when you found a reliable server.

For my region only one server (marked with >) is able to provide reliable bandwidth for Gigabit services.

[17:40:42] openhab@oH-Speedtest:~$ speedtest -L
Closest servers:

    ID  Name                           Location             Country
==============================================================================
   15431  Spacken.net                    Hagen                Germany
   17392  myLoc managed IT AG            Dusseldorf           Germany
   21467  InoHost                        Düsseldorf          Germany
 > 28624  Händle & Korte GmbH           Dusseldorf           Germany
   28716  ColocationIX 10G               Dusseldorf           Germany
   30906  Deutsche Telekom Technik GmbH  Dusseldorf           Germany
    6601  NetCologne                     Cologne              Germany
    6670  hotspot.koeln                  Cologne              Germany
   22274  Gemeindewerke Nümbrecht GmbH  Nümbrecht           Germany
   24887  Studenten Net Twente           Enschede             Netherlands

Could you please tell me where this directory and file should be? I can´t find it in my openhabian setup.
The docs are not a great help as there´s no full path listed.

Nevermind… i need to create it myself under /etc/openhab2/misc

@Bredmich

Question for you…I’m wanting to use the SpeedtestSummary data in a HabPanel Widget

Do to my current layout the data renders as follows:

image

How can I modify the String below to place line breaks between each of the three values so up, down, and ping are all on separate lines?

SpeedtestSummary.postUpdate(String::format("ᐁ  %.1f Mbit/s  ᐃ %.1f Mbit/s (%.0f ms)", down, up, ping))

You could try \n but i´m not sure if HABpanel will do it correctly.

SpeedtestSummary.postUpdate(String::format("ᐁ %.1f Mbit/s\nᐃ %.1f Mbit/s\n(%.0f ms)", down, up, ping))

E: The repository 'https://ookla.bintray.com/debian eoan Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

I assume i am doing something simple, wrong. Running openhab on ubuntu 19.10

I think you need to --allow-unauthenticated to your sudo apt-get update
As i´m not using Ubuntu i´m not sure if that´s the correct way.

As an FYI -

The /n would not work in HabPanel as you mentioned might be the case…

What I ended up doing was using the individual elements and making a widget from them.

Came out very nice and fits in with the overall design of my status board.
image

Thanks again for this great addition to OH!

Squid

Just to be sure, you used \n and not /n?