Nvidia Shield Binding - Successfully Decoded Protocol - Looking For Binding Coding Help

Good afternoon all,

After getting frustrated with the lag of the adb binding, and the lack of wanting to build an IPtoIR setup, I opted to see what I could find out about the Nvidia shield tv android app. I was able to successfully setup a TLS MITM with openssl and extract out the protocol exchange between the app and the device. I was also able to successfully intercept the PIN process and determine the authentication process as well as decode what the specific button clicks are for the remote. What I need, is some help building a binding around this. I’m not proficient enough at writing java anymore to do this anywhere in a timely or elegant manner.

As a primer, this is what I’ve determined:

  1. App connects to the shield on port 8987. This is a basic tls connection and was done using openssl s_connect in my testing. This is not an http connection like some of the android tv’s use.
  2. App sends an ID packet to the sheild (presumably to use as some sort of ID)
  3. Shield replies with it’s device name (the app uses this to populate the device name)
  4. The app and the shield exchange a 2 packet back and forth. This seems to stimulate the PIN request to pop up on the TV
  5. The app sends the PIN back embedded in some other data (I have no idea what that other data means yet)
  6. The shield replies with a 3 byte packet, not sure what this is yet.
  7. The shield replies with a 4204 byte payload (after tls decryption). This is broken up with a 50 byte block of data (unknown yet what this is, but it’s been almost identical on every attempt I’ve made so far), then a base64 encoded 2048 bit private rsa key (2442 bytes long), then a certificate (1708 bytes long), and finally a footer of 4 bytes. This certificate is 10 years long, starting on that day, and looks to be generated by the shield itself.

At this point, the PIN process has completed and you disconnect the current tls session. Using the data from that last packet, create a pem encoded key file and pem encoded certificate file. You will need to make sure that the data is base64 encoded. Going forward, it’s imperative that connections to the shield always use this key and certificate. It’s effectively a login for the client. If you miss once, it restarts the PIN process and invalidates the cert/key.

Once you’re here, you basically replay many of the same packets as above minus the key request packets. At this point, key presses are as simple as some basic 17 byte messages sent to the shield (I’ve recorded many). I’ve been able to store logs of the tls keys during the process and run PCAP to expose all of this. Full disclosure, I have not even attempted to decode the entire protocol, this is basically a replay of decrypted tls packets to achieve the effect. I have found one additional request where the app asks for the list of installed apps and the shield replies with a rather extensive set of URLs and metadata on the apps (and you can one button launch them). Further research may uncover things like device status.

So, does anyone want to sign up to work with me to develop a binding for the Nvidia Shield?

EDIT: I forgot to divide by 2 on all the bytes. Numbers above are hex digit lengths, not bytes.

2 Likes

Very interesting (and interested). But I’m a bit pressed for time atm. I already have a bit of a backlog of changes to other bindings that I haven’t been able to get to, and it looks like I’m gonna miss the release with those changes. :sob:

If no one else jumps in, perhaps we can try to do this on a slow burn.

I’m curious about a couple things…

  • It’s unclear how the packets are formatted. From the sounds of it, they’re not using JSON. Is that correct?

  • I’ve never actually used the app to control my Shield. Do you know how the app discovers the Shield on the network?

  • Do you know if there’s a difference between the Shield and the Shield Pro in terms of how it’s controlled?

I’m a person who knows what my own limits are. In respect to this, I’m absolutely happy to help write the functions, debug, etc., but I know I have never built a binding from the ground up, and I know I don’t know the bits and pieces of the way OH is setup internally to get the basics all together right. I’ve also never worked with the TLS libraries before so that will be a learning process to me. If we can get the structure setup properly, I can probably write a bunch of the inner guts. That said, I’m very proficient at the network side so I should be able to provide a good bit of input on how to actually interact with the device and troubleshoot issues. I’m more than happy to team up and work together if you can help me through those bits.

For your questions:

-You’re absolutely correct, this isn’t JSON or any other format that’s human readable (unfortunately). The packets all seem to start with 0x08. This was one of the decoded captures I did of the PIN exchange. This is specifically the TLS payload:

Phone:
0000   08 01 12 1a 08 01 12 10 73 61 6d 73 75 6e 67 20   ........samsung 
0010   53 4d 2d 47 39 39 38 55 18 01 28 fb ff 04         SM-G998U..(...

Shield:
0000   08 0a 12 21 08 e8 07 12 1c 08 01 12 14 5b 53 48   ...!.........[SH
0010   49 45 4c 44 5d 20 4c 69 76 69 6e 67 20 52 6f 6f   IELD] Living Roo
0020   6d 18 d7 fd 04 18 0a                              m......

Phone: <--- EDIT: This is now confirmed to be the PIN request initialization.
0000   08 0a 12 03 08 cd 08                              .......

Shield:
0000   08 0a 12 03 08 cf 08 18 0a                        .........

Phone: (I believe this to be some sort of nonce)
0000   08 00 28 96 d0 e7 f9 cc 30                        ..(.....0

Shield: (I believe this to be some sort of nonce)
0000   08 00 28 96 d0 e7 f9 cc 30                        ..(.....0

Phone:  <---This is the PIN exchange.  770111 was the PIN
0000   08 0a 12 1f 08 d1 08 12 1a 0a 06 37 37 30 31 31   ...........77011       
0010   31 12 10 36 64 65 64 64 64 61 32 66 39 36 66 35   1..6deddda2f96f5
0020   64 62 61                                          dba

Shield:
0000   20 b4 10                                           ..

Shield: <--- This was the huge packet which contained the new private and public keys.  The private key starts on the second line at 30 82.  All of the data before that seems to be almost identical every time I ran my tests.  There are maybe 4 total hex characters that change across the whole thing.

0000   08 0a 12 ad 10 08 b5 10 12 a7 10 08 01 12 07 53   ...............S
0010   75 63 63 65 73 73 1a c0 09 30 82 04 bc 02 01 00   uccess...0......
0020   30 0d 06 09 2a 86 48 86 f7 0d 01 01 01 05 00 04   0...*.H.........
0030   82 04 a6 30 82 04 a2 02 01 00 02 82 01 01 00 9f   ...0............
0040   67 a1 f0 9f 96 03 16 4b 59 a5 25 ba 52 32 2e f5   g......KY.%.R2..
0050   2f 46 94 57 c0 a7 23 e4 a5 2e 46 e7 a0 cc 71 bb   /F.W..#...F...q.
0060   6f ce 91 7a ba 9f c6 81 2f 77 7a 5b 8f 2c 29 c2   o..z..../wz[.,).
0070   8c a6 f1 74 33 0b e3 5e 12 a0 5a 65 bc e3 6d 3b   ...t3..^..Ze..m;
0080   b2 12 fc d6 54 c9 63 d3 8c 50 13 a1 cb e5 25 1f   ....T.c..P....%.
0090   99 2b 49 30 7e 0c fe 70 be b9 4c 04 31 a4 cd fd   .+I0~..p..L.1...
00a0   23 c6 e0 bc 02 85 63 2c 05 07 45 b1 1a 94 cb 2f   #.....c,..E..../
00b0   be 87 f2 13 a9 3b 02 99 42 9f 54 8f 34 61 96 d9   .....;..B.T.4a..
TRUNCATED
0800   a0 6a a8 36 d3 32 3b ac b4 6c fb 1b 24 6e 5f 64   .j.6.2;..l..$n_d
0810   f0 bc f9 78 86 0a 89 bc 7d b3 0d 1d b6 a6 59 68   ...x....}.....Yh
0820   f9 fc 13 6e 16 74 ea 91 36 07 dd f9 00 d1 ef 0c   ...n.t..6.......
0830   4f 8f 18 0a                                       O...

"18 0a" seems to be some sort of end of packet delimiter.  The Shield sends it almost at the end of every packet it sends.

These are a few button pushes (I don't remember which, I wasn't making a point to record specifics yet):
0000   08 e9 07 12 0c 08 14 10 01 20 0a 28 02 32 02 96   ......... .(.2..
0010   02                                                .
0000   08 e9 07 12 0c 08 14 10 01 20 0a 28 01 32 02 96   ......... .(.2..
0010   02                                                .
0000   08 e9 07 12 0c 08 14 10 01 20 0a 28 01 32 02 d8   ......... .(.2..
0010   02                                                .
0000   08 e9 07 12 0c 08 14 10 01 20 0a 28 02 32 02 d8   ......... .(.2..
0010   02                                                .
0000   08 e9 07 12 0c 08 14 10 01 20 0a 28 01 32 02 d4   ......... .(.2..
0010   01                                                .


There is a UPNP message sent by the shield for discovery. We can absolutely use this for device autodescovery. JuPnP should be cataloging this so we just need to pull it out of the already provided list.

EDIT: I dug through the PCAP files I had, there is also an MDNS query that can be sent out for them to respond to. Several actually for different things. These were in the clear so we should be able to easily query for this.

I have no idea the difference between the two. That said, given that they are all Android based, I would hope and assume that while there could be some differences that our code could be reusable for the majority. Maybe just a different interpreter, tcp port, or keycode. I’ve got the MITM working really well, and it’s really easy to setup so as long as we have someone who has the device in question I would assume that we could easily integrate all of the different shield models.

If I’m understanding this correctly enough, as long as we properly establish the TLS connection to the device (using the correct keys and such), and can maintain and interact with that easily enough, then this is more or less similar to a replay attack where we just send

Do you have a Shield or a Shield Pro?

I have a Pro running the latest version. 9.1.1, I think.

I have a Shield P2571 running Shield Android TV SW Version 9.1.1(33.2.0.157) which I believe is Android 11

EDIT: I also have a Shield P2897 running the same 9.1.1 version.

And I don’t think it matters but mine is running a custom launcher…

I’m bone stock. I do have a second shield, but I’m 99% sure it’s the same. The only thing I know that’s different is that one doesn’t have the IR port like this one does (they took it out in newer models).

EDIT: I lied, I have both a PRO and a non-PRO 2017.

EDIT2: I tested the MITM on both, seems to be the same. As far as I can tell the only difference between the two is the IR port, SD card slot, and HDD size. Otherwise, same software.

I went through and created a branch on git for this. I also ran the bundle creation script to build the basic framework.

Sitting thinking about this overnight gave me an idea. @BobA is this similar to how the lutron binding works? Its been a minute since I’ve seen the raw feed but if I remember correctly its more or less a proprietary string sent back and forth between OH and the lutron controller. I want to go so far as to say that maybe it also had a TLS layer now as well? Am I close or just completely out of left field?

My thought, if it is similar, is to attempt to reuse some of that code (assuming @BobA doesn’t mind) and modify it to send the nvidia protocol as needed.

Hi @morph166955. Feel free to copy code from the Lutron and Alarmdecoder bindings. I probably copied it from somewhere else. :slight_smile: In the Lutron binding, the leapbridge thing opens and maintains a TLS connection with the hub, but it is a bit complicated. The Alarmdecoder binding’s ipbridge thing may be a simpler example to use.

Awesome! I’ll parse through both and grab the relevant pieces. This nvidia binding doesnt seem complicated, just needs a specific order of operations. Thanks!

To update everyone, I’ve made some limited progress. Code as posted is able to establish a basic tls connection to the shield using a cert/key that I derived previously. Next step is to write the protocol interpreter and start to talk to it. Once I do that, I should be able to begin the PIN code process and then ultimately the keypress process.

2022-12-11 21:05:16.051 [TRACE] [ng.shieldtv.internal.ShieldTVHandler] - Initializing keystore
2022-12-11 21:05:16.254 [TRACE] [ng.shieldtv.internal.ShieldTVHandler] - Initializing SSL Context
2022-12-11 21:05:16.305 [DEBUG] [ng.shieldtv.internal.ShieldTVHandler] - Opening SSL connection to 10.255.0.175:8987
2022-12-11 21:05:16.373 [DEBUG] [ng.shieldtv.internal.ShieldTVHandler] - Assuming server certificate is valid
2022-12-11 21:05:16.374 [TRACE] [ng.shieldtv.internal.ShieldTVHandler] - Subject DN: C=US, O=Nvidia, OU=Nvidia, CN=nvbeyonder/darcy/darcy/SHIELD Android TV
2022-12-11 21:05:16.375 [TRACE] [ng.shieldtv.internal.ShieldTVHandler] - Issuer DN: C=US, O=Nvidia, OU=Nvidia, CN=nvbeyonder/darcy/darcy/SHIELD Android TV
2022-12-11 21:05:16.376 [TRACE] [ng.shieldtv.internal.ShieldTVHandler] - Serial number: 1564616000658
2022-12-11 21:05:16.421 [DEBUG] [ng.shieldtv.internal.ShieldTVHandler] - Message reader thread started
2022-12-11 21:05:16.421 [DEBUG] [ng.shieldtv.internal.ShieldTVHandler] - Starting keepalive job with interval 5
2022-12-11 21:05:16.421 [DEBUG] [ng.shieldtv.internal.ShieldTVHandler] - Command sender thread started

Thing and Item configuration are basically trival at this point:

String ShieldTV_KEYPRESS "KEYPRESS [%s]" { channel = "shieldtv:shieldtv:theater:keypress" }
String ShiledTV_PINCODE  "PINCODE [%s]" { channel = "shieldtv:shieldtv:theater:pincode" }

Thing shieldtv:shieldtv:theater [ ipAddress="x.x.x.x", keystore="/home/openhab/shieldtv.theater.keystore", keystorePassword="secret" ]

A significant amount of the code is pulled in from the Lutron Leap binding and adapted to work for this.

At what point should I create a PR so this can be tracked? I think that for “version 1” I’m going to strictly focus on basic device connectivity (the PIN process) and basic keypress to keep it simple. I’ll save auto discovery, app control, and device info for a follow-on PR.

Thoughts?

I’ve made some significant progress today. I was able to create a new channel called RAW so I could experiment a little easier with the protocol. I was able to connect to the shield and manually work through the PIN process by sending the raw hex that I’ve been able to decode. I have confirmed the strings that begin the process as well as the proper way to reply with the PIN. I have some more work to do in terms of having this not be a bunch of raw hex, but given the success here I see no reason that this won’t work at this point. In theory, I could probably have sent some key presses if I tried to test that.

EDIT: Draft PR created - [shieldtv][WIP] Initial Contribution - NVIDIA ShieldTV by morph166955 · Pull Request #13934 · openhab/openhab-addons · GitHub

Going to move away from this as the binding is now posted in the community