HabAPP fail to use async_http.get_client_session()

I have all my complex rules working on HabAPP.

I need to implement session handling in order to send http/json commands to IMEON inverter, but I am struggling to understand how this works.

I am extending the example from habapp article on asyncio to leverage session, but I gather session cookies are not saved. What am I get wrong?

import asyncio
import logging

import HABApp


class AsyncRule2(HABApp.Rule):

    def __init__(self):
        super().__init__()
        self.log = logging.getLogger(f"Rule.{self.__class__.__name__}")

        self.run.soon(self.async_func)

    async def async_func(self):
        await asyncio.sleep(2)
        s = self.async_http.get_client_session()
        async with s.get('http://httpbin.org/get') as resp:
            self.log.debug(resp)
            self.log.debug(await resp.text())
            self.log.info(f"session  headers: {s.headers}")
            self.log.info(f"session cookies: {s.cookie_jar}")
            cookies = s.cookie_jar
            self.log.debug(f"length {len(cookies)}")
            for cookie in cookies:
                self.log.info(f'Cookie: {cookie.key}={cookie.value}')


AsyncRule2()

What do you mean by that? Also get_client_session will return the same session so there is no need to save it.

Well, I expected to have the sesseion to save cookies.

in my case with imeon inverter, I create session, login to IMEON with session.post() request succesfully, but when in next step I try to read IMEON values with session.get() it responds with login sceen (which means session cookies were not present).

So I thought I with this example I am checking if session cookies are saved. but s.cookie_jar is empty.

Do I get it wrong?

I think you have to call update_cookies to set the cookie after successfully logging in.

The client session should have a cookie_jar and things should work out of the box.
Maybe there is some other issue? Can you reproduce the issue with a simple asnycio script?

async def test():
   async with aiohttp.ClientSession() as session:
        await session.post(...)
        await session.get(...)

asyncio.run(test())

You are right, it seems aiohttp.ClientSession() does not correctly handle cookies in my case (I tried on several python environments).

I succeeded in storing and holding cookies manually, so I will test with habapp implementation.

So I tested with manual cookie handling with aiohttp and this code works.

But how I do implement in with habapp?

import asyncio
import aiohttp
import json
from http.cookies import SimpleCookie
from yarl import URL

async def test():
    payload = {
        'do_login': 'true',
        'email': "****",
        'passwd': "****"
    }
    async with aiohttp.ClientSession() as session:
        r = await session.post('http://*.*.*.*/login', data=payload)
        
        # Print response and headers to verify cookies are being set
        print(f"result of call: {r}")
        response_json = await r.json()
        print(response_json)
        print(f"Response headers: {r.headers}")
        
        # Manually parse the Set-Cookie headers
        set_cookie_headers = r.headers.getall('Set-Cookie', [])
        cookie_jar = SimpleCookie()
        for header in set_cookie_headers:
            cookie_jar.load(header)
        
        # Debug: Print parsed cookies
        print("Parsed cookies from Set-Cookie headers:")
        cookies = {}
        for key, morsel in cookie_jar.items():
            cookies[key] = morsel.value
            print(f"cookie: {key}, value: {morsel.value}")
        
        # Perform an authorized GET request with the manually set cookies
        async with session.get('http://*.*.*.*/data', cookies=cookies, timeout=10) as r:
            r.raise_for_status()
            print(f"IMEON READ data status: {r.status}")

            if 'application/json' in r.headers.get('Content-Type', ''):
                json_data = await r.json()
                print(json_data)

            if r.status != 200:
                print("!200")

            else:
                response_text = await r.text()
                print(f"Unexpected response content type: {r.headers.get('Content-Type')}")
                print(f"Response content: {response_text}")

asyncio.run(test())

await session.post

becomes

async with self.async_http.post

and get accordingly

I modified the code according to your suggestion:

import asyncio
import logging
from http.cookies import SimpleCookie
from yarl import URL

import HABApp


class AsyncRule2(HABApp.Rule):

    def __init__(self):
        super().__init__()
        self.log = logging.getLogger(f"Rule.{self.__class__.__name__}")

        self.run.soon(self.async_func)

    async def async_func(self):
        payload = {
            'do_login': 'true',
            'email': "****",
            'passwd': "****"
        }
        async with self.async_http.get_client_session() as session:
            r = await session.post('http://10.0.20.201/login', data=payload)

            # self.log.debug response and headers to verify cookies are being set
            self.log.debug(f"result of call: {r}")
            response_json = await r.json()
            self.log.debug(response_json)
            self.log.debug(f"Response headers: {r.headers}")

            # Manually parse the Set-Cookie headers
            set_cookie_headers = r.headers.getall('Set-Cookie', [])
            cookie_jar = SimpleCookie()
            for header in set_cookie_headers:
                cookie_jar.load(header)

            # Debug: self.log.debug parsed cookies
            self.log.debug("Parsed cookies from Set-Cookie headers:")
            cookies = {}
            for key, morsel in cookie_jar.items():
                cookies[key] = morsel.value
                self.log.debug(f"cookie: {key}, value: {morsel.value}")

            # Perform an authorized GET request with the manually set cookies
            async with session.get('http://10.0.20.201/data', cookies=cookies, timeout=10) as r:
                r.raise_for_status()
                self.log.debug(f"IMEON READ data status: {r.status}")

                if 'application/json' in r.headers.get('Content-Type', ''):
                    json_data = await r.json()
                    self.log.debug(json_data)

                if r.status != 200:
                    self.log.debug("!200")

                else:
                    response_text = await r.text()
                    self.log.debug(f"Unexpected response content type: {r.headers.get('Content-Type')}")
                    self.log.debug(f"Response content: {response_text}")


AsyncRule2()

This is a logging error output:

2024-07-16 18:41:33.129 [INFO ] [HABApp.Rules                        ] - Added rule "AsyncRule2" from rules/tetsas2.py

2024-07-16 18:41:33.146 [ERROR] [HABApp.Worker                       ] - Error in AsyncRule2.async_func: Session is closed

2024-07-16 18:41:33.146 [ERROR] [HABApp.Worker                       ] - File "/etc/openhab/habapp/rules/tetsas2.py", line 24 in async_func

2024-07-16 18:41:33.146 [ERROR] [HABApp.Worker                       ] - --------------------------------------------------------------------------------

2024-07-16 18:41:33.146 [ERROR] [HABApp.Worker                       ] -      17 | async def async_func(self):

2024-07-16 18:41:33.146 [ERROR] [HABApp.Worker                       ] -      18 |     payload = {

2024-07-16 18:41:33.146 [ERROR] [HABApp.Worker                       ] -      19 |         'do_login': 'true',

2024-07-16 18:41:33.146 [ERROR] [HABApp.Worker                       ] -      22 |     }

2024-07-16 18:41:33.146 [ERROR] [HABApp.Worker                       ] -      23 |     async with self.async_http.get_client_session() as session:

2024-07-16 18:41:33.146 [ERROR] [HABApp.Worker                       ] - -->  24 |         r = await session.post('http://10.0.20.201/login', data=payload)

2024-07-16 18:41:33.146 [ERROR] [HABApp.Worker                       ] -      26 |         # self.log.debug response and headers to verify cookies are being set

2024-07-16 18:41:33.146 [ERROR] [HABApp.Worker                       ] -    ------------------------------------------------------------

2024-07-16 18:41:33.146 [ERROR] [HABApp.Worker                       ] -      self = <AsyncRule2>

2024-07-16 18:41:33.147 [ERROR] [HABApp.Worker                       ] -      self.log = <Logger Rule.AsyncRule2 (DEBUG)>

2024-07-16 18:41:33.147 [ERROR] [HABApp.Worker                       ] -      session = <aiohttp.client.ClientSession object at 0x7f8589bb4b90>

2024-07-16 18:41:33.147 [ERROR] [HABApp.Worker                       ] -    ------------------------------------------------------------

2024-07-16 18:41:33.147 [ERROR] [HABApp.Worker                       ] - 

2024-07-16 18:41:33.147 [ERROR] [HABApp.Worker                       ] - --------------------------------------------------------------------------------

2024-07-16 18:41:33.147 [ERROR] [HABApp.Worker                       ] - Traceback (most recent call last):

2024-07-16 18:41:33.147 [ERROR] [HABApp.Worker                       ] -   File "/opt/habapp/lib/python3.11/site-packages/HABApp/core/internals/wrapped_function/wrapped_async.py", line 31, in async_run

2024-07-16 18:41:33.147 [ERROR] [HABApp.Worker                       ] -     await self.func(*args, **kwargs)

2024-07-16 18:41:33.147 [ERROR] [HABApp.Worker                       ] -   File "/etc/openhab/habapp/rules/tetsas2.py", line 24, in async_func

2024-07-16 18:41:33.147 [ERROR] [HABApp.Worker                       ] -     r = await session.post('http://10.0.20.201/login', data=payload)

2024-07-16 18:41:33.147 [ERROR] [HABApp.Worker                       ] -         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

2024-07-16 18:41:33.147 [ERROR] [HABApp.Worker                       ] -   File "/opt/habapp/lib/python3.11/site-packages/aiohttp/client.py", line 425, in _request

2024-07-16 18:41:33.147 [ERROR] [HABApp.Worker                       ] -     raise RuntimeError("Session is closed")

2024-07-16 18:41:33.147 [ERROR] [HABApp.Worker                       ] - RuntimeError: Session is closed

You have to remove the session context manager, otherwise the session will close.
Sorry if that was not clear.

async with self.async_http.get_client_session() as session:

has to become

session = self.async_http.get_client_session()

You have to restart HABApp to create a new session and make it work again after you modified the rule.

Thanks, in my case after restarting habapp the code started working.

When I moved

session = self.async_http.get_client_session()

to init block

self.session =  self.async_http.get_client_session()

it does not need to restart habapp to work properly.

So all in all, mo code now works perfectly with cookies extraction and writing code. I do not not know if malfunction of automatic ClientSession cookies management implementation was specific to my circumstances.

@Spaceman_Spiff , thanks a lot for you help, it was most useful. And HabAPP for me is super useful.

I would have worked without moving to the init block.
The context manager (with ... as session) closed the existing session that’s why you had to do the restart once (so you have an open session which you can use again).

When in doubt you can also create the rule with the async function from your test script.
You will then create a session each time which is not optimal, but tbh for some home automation stuff which runs once a couple of minutes it doesn’t really matter.