It’s complicated but yes. The esp8266 is VERY powerful and as such can get very complicated.
By default, most use (opinion alert?) the AT command set. Basically you send AT commands via serial and listen for responses. You set your arduino code to talk to and handle serial from the esp8266. The biggest issue is you must talk to the esp over serial. That means you have to create a software serial on most arduino other than mega. Once you get them talking over the software serial port it’s not too bad but in my case, I relized I needed a mega worth of io pins (as software serial sucked up yet another 2) to do my project and moved on.
As I mentioned, I plan to circle back but life has been very hectic lately and I haven’t had time for fun projects.
To actually ANSWER your question, I still would recommend MQTT but you could use http bindings as well. I think MQTT makes more sense and gives you more options. You can go two routes, either program an esp8266 or variation to talk directly to OH via mqtt or connect the esp8266 to an arduino and have the arduino do the thinking and use the esp as a “serial modem” if you will. I think it depends on how many devices you need to connect to the esp. The standard 8 pin esp is limited but the breakouts like huzzah present more io pins.
I would google around a bit for MQTT esp8266 projects. Like: https://home-assistant.io/blog/2015/10/11/measure-temperature-with-esp8266-and-report-to-mqtt/
There is one on the forum that doesn’t use MQTT: DIY: Cheap wifi-based temperature/humidity sensor based on ESP8266 & DHT22
This blog series is fantastic for learning mqtt and arduino among other things: https://openhardwarecoza.wordpress.com/?s=mqtt