Hacking the Lidl Silvercrest ZigBee Gateway: A Step-by-Step Tutorial

What’s new
2021-12-05: initial version

Disclaimer
Follow this tutorial at your own risk: if you brick your hardware or burn down your house, I will not be responsible for any damage. I’m going to answer questions on a best effort basis, but please do not expect any support from my side.

What’s this all about?
AFAIK there is no cheap Ethernet to ZigBee gateway. All cheap ones are configured to phone home and cannot be used as a ZigBee Coordinator by the openHAB ZigBee Binding. By following this tutorial you can free your Lidl Silvercrest ZigBee gateway from ‘Lidl Home’ and make it compatible with the openHAB ZigBee binding.

Previous work used in this tutorial

Limitations
The canonical use case would be to connect the openHAB ZigBee Coordinator Thing to the Ethernet to ZigBee gateway via RFC 2217. But this doesn’t currently work - at least as far as the Silvercrest gateway is concerned; for a very detailed discussion see https://community.openhab.org/t/oh-3-2-0-m3-zigbee-binding-ember-m35x-coordinator-rfc2217-hacked-silvercrest-zigbee-gateway-efr32mg1b-256k-emberznet-6-7-8-0/ .
@chris is going to debug the problem, but there is no guarantee that it ever will be fixed. So the tutorial has to end here … - no, of course not, there is a workaround: you can use socat to create a local device that is connected to the gateway and can be used by the ZigBee Binding. There is a socat for Windows, but I couldn’t get it to work (in other words: if you run openHAB on Windows, as of now you are on your own).

Hardware requirements

  • Lidl Silvercrest Ethernet to ZigBee Gateway
  • Tools to open the case (e.g. screwdriver, scalpel, watchmaker’s knife)
  • Soldering iron
  • Pin strip
  • USB serial programmer (3.3 V)
  • cables to connect the USB serial programmer to the computer and to the pin strip soldered to the gateway

Software requirements

  • tested on openHABian 3.2.0-SNAPSHOT - Build #2608 / Linux 10 (buster)

Let’s get things done

  1. Pry it open.


  2. Remove the PCB.

  3. Solder the pin strip to the PCB.

  4. Connect the serial programmer to your computer and to the pin strip (for details see Hacking the Silvercrest (Lidl) Smart Home Gateway - Paul Banks DOT Org)

  5. Enter the Realtek bootloader by hitting <ESC> on the serial console as soon as power is applied to the gateway:
    <RealTek>

  6. Extract the root password:
    Enter:
    FLR 80000000 401802 16
    DW 80000000 4
    The output (your KEK - key-encryption-key) should look like:
    80000000: XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX
    Enter:
    FLR 80000000 402002 32
    DW 80000000 8
    The output (your encoded AUSKEY) should look like:
    80000000: XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX
    80000010: XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX
    On your computer:
    wget https://paulbanks.org/download/files/lidl-zigbee/lidl_auskey_decode.py
    python3 lidl_auskey_decode.py
    Paste KEK and encoded AUSKEY (without the ‘8…:’ prefix) to get your root password:
    Auskey: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    Root password: XXXXXXXX

  7. Test the root password:
    Reboot the gateway, login as root.
    Tuya Linux version 1.0
    Jan 1 00:00:45 login[121]: root login on 'console'
    #

  8. Install serial gateway
    On your computer:
    wget https://paulbanks.org/download/files/lidl-zigbee/serialgateway.bin
    cat serialgateway.bin | ssh -p2333 root@<gateway IP address> "cat >/tuya/serialgateway"

  9. Free the gateway
    Disable phoning home:
    On the gateway
    cp /tuya/tuya_start.sh /tuya/tuya_start.original.sh
    cat >/tuya/tuya_start.sh << 'EOF'
    #!/bin/sh
    while true; do
    pgrep -x serialgateway >/dev/null
    if [[ $? -ne 0 ]] ; then
    echo "Restarting SerialGateway: $(date)" >> /var/log/serialgateway.txt
    /tuya/serialgateway >> /var/log/serialgateway.txt &
    fi
    sleep 30
    done &
    EOF
    Optional:
    Disable brute force protection, shift SSHd from port 2333 to port 22:
    cp /tuya/ssh_monitor.sh /tuya/ssh_monitor.original.sh
    echo "#!/bin/sh" >/tuya/ssh_monitor.sh
    Reboot the gateway. Please note that dropbear is now running on port 22.

  10. Upgrade the Ember chip to 6.7.8.0
    On your computer
    wget https://github.com/grobasoz/zigbee-firmware/raw/master/EFR32%20Series%201/EFR32MG1B-256k/NCP/NCP_UHW_MG1B232_678_PA0-PA1-PB11_PA5-PA4.gbl
    wget https://github.com/Ordspilleren/lidl-gateway-freedom/raw/master/scripts/firmware_upgrade.sh
    chmod a+x firmware_upgrade.sh
    wget https://github.com/Ordspilleren/lidl-gateway-freedom/raw/master/scripts/sx
    Play safe and deactivate the serial gateway:
    On the gateway
    mv /tuya/serialgateway /tuya/serialgateway_save
    killall serialgateway
    On your computer
    ./firmware_upgrade.sh <gateway IP address> 22 V7 NCP_UHW_MG1B232_678_PA0-PA1-PB11_PA5-PA4.gbl
    Re-activate the serial gateway and reboot:
    On the gateway
    mv /tuya/serialgateway_save /tuya/serialgateway
    reboot

  11. Check the modifications
    Check that serialgateway is running:
    # ps -al | grep serial
    124 root 208 S /tuya/serialgateway
    139 root 1284 S grep serial
    Check that port 8888 is open:
    nmap <gateway IP address>
    Output
    PORT STATE SERVICE
    22/tcp open ssh
    8888/tcp open sun-answerbook

  12. Integrate the gateway into openHAB
    Stop openHAB
    sudo /bin/systemctl stop openhab.service
    Run socat:
    sudo socat -dd pty,link=/dev/ttyzbbridge1,raw,user-late=openhab,group-late=dialout tcp:<gateway IP address>:8888 &
    Restart openHAB:
    sudo /bin/systemctl start openhab.service
    Install ZigBee Binding.
    Add Ember Coordinator:
    Things → (+) → ZigBee Binding → Add manually → Ember Coordinator:
    Serial Port: /dev/ttyzbbridge1 - Create Thing → Save
    Check Ember Coordinator:
    On Karaf console
    openhab> zigbee firmware version
    Dongle firmware version is currently 6.7.8.0
    openhab> zigbee ncpversion
    Ember NCP version 6.7.8.0, EZSP version 8

  13. Add your ZigBee devices:
    Things → (+) → ZigBee Binding → Scan

  14. For the Adventurous. Be fearless. Be curious.
    EFR32-FW/TUYA_ZBGW at main · MattWestb/EFR32-FW · GitHub

8 Likes

This topic was automatically closed 41 days after the last reply. New replies are no longer allowed.