Version 0.5.4

Hardening (#30 )
* set build-argument for environment * hardening remove dangerous commands * add hardening scripts for base and final image
2023-11-22 22:26:10 +01:00 · 2023-11-22 21:57:42 +01:00 · 2023-11-22 18:45:16 +01:00 · 2023-11-22 18:45:03 +01:00 · 2023-11-22 18:33:56 +01:00 · 2023-11-21 22:31:46 +01:00
16 changed files with 535 additions and 80 deletions
--- a/.gitignore
+++ b/.gitignore
@@ -1,5 +1,6 @@
 __pycache__
 .pytest_cache
+bin/**
 mosquitto/**
 homeassistant/**
 tsun_proxy/**
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,6 +7,33 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

 ## [Unreleased]

+## [0.5.4] - 2023-11-22
+
+- hardening remove dangerous commands from busybox
+- add OTA start message counter
+- add message handler for over the air updates
+- add unit tests for ota messages
+- add unit test for int64 data type
+- cleanup msg_get_time_handler
+- remove python packages setuptools, wheel, pip from final image to reduce the attack surface
+ 
+## [0.5.3] - 2023-11-12
+
+- remove apk packet manager from the final image
+- send contact info every time a client connection is established
+- use TSUN timestamp instead of local time, as TSUN also expects Central European Summer Time in winter
+    
+## [0.5.2] - 2023-11-09
+
+- add int64 data type to info parser
+- allow multiple calls to Message.close()
+- check for race cond. on closing and establishing client connections
+ 
+## [0.5.1] - 2023-11-05
+
+- fixes f-string by limes007
+- add description for dns settings by limes007
+
 ## [0.5.0] - 2023-11-04

 - fix issue [#21](https://github.com/s-allius/tsun-gen3-proxy/issues/21)
--- a/README.md
+++ b/README.md
@@ -131,6 +131,23 @@ suggested_area = 'balcony'    # Optional, suggested installation area for home-a

 ```

+## DNS Settings
+
+### Loop the proxy into the connection
+To include the proxy in the connection between the inverter and the TSUN Cloud, you must adapt the DNS record of *logger.talent-monitoring.com* within the network that your inverter uses. You need a mapping from logger.talent-monitoring.com to the IP address of the host running the Docker engine.
+
+This can be done, for example, by adding a local DNS record to the Pi-hole if you are using it.
+
+### DNS Rebind Protection
+If you are using a router as local DNS server, the router may have DNS rebind protection that needs to be adjusted. For security reasons, DNS rebind protection blocks DNS queries that refer to an IP address on the local network.
+
+If you are using a FRITZ!Box, you can do this in the Network Settings tab under Home Network / Network. Add logger.talent-monitoring.com as a hostname exception in DNS rebind protection.
+
+### DNS server of proxy
+The proxy itself must use a different DNS server to connect to the TSUN Cloud. If you use the DNS server with the adapted record, you will end up in an endless loop as soon as the proxy tries to send data to the TSUN Cloud.
+
+As described above, set a DNS sever in the Docker command or Docker compose file.
+
 ## License

 This project is licensed under the [BSD 3-clause License](https://opensource.org/licenses/BSD-3-Clause).
--- a/app/.dockerignore
+++ b/app/.dockerignore
@@ -1,4 +1,5 @@
 tests/
 **/__pycache__
 *.pyc
-.DS_Store
+.DS_Store
+build.sh
--- a/app/Dockerfile
+++ b/app/Dockerfile
@@ -7,20 +7,21 @@ ARG GID=1000
 FROM python:3.12-alpine AS base
 USER root

-RUN apk update && \
-    apk upgrade
-RUN apk add --no-cache su-exec	
+COPY --chmod=0700 ./hardening_base.sh .
+RUN apk upgrade --no-cache && \
+    apk add --no-cache su-exec	&& \
+    ./hardening_base.sh && \
+    rm ./hardening_base.sh

 #
 # second stage for building wheels packages
 FROM base as builder

-RUN apk add --no-cache build-base && \
-    python -m pip install --no-cache-dir -U pip wheel
-
 # copy the dependencies file to the root dir and install requirements
 COPY ./requirements.txt /root/
-RUN python -OO -m pip wheel --no-cache-dir --wheel-dir=/root/wheels -r /root/requirements.txt
+RUN apk add --no-cache build-base && \
+    python -m pip install --no-cache-dir -U pip wheel && \
+    python -OO -m pip wheel --no-cache-dir --wheel-dir=/root/wheels -r /root/requirements.txt


 #
@@ -31,26 +32,32 @@ ARG VERSION
 ARG UID
 ARG GID
 ARG LOG_LVL
+ARG environment

 ENV VERSION=$VERSION
 ENV SERVICE_NAME=$SERVICE_NAME
 ENV UID=$UID
 ENV GID=$GID 
 ENV LOG_LVL=$LOG_LVL
+ENV HOME=/home/$SERVICE_NAME

    
 # set the working directory in the container
 WORKDIR /home/$SERVICE_NAME

-# update PATH environment variable
-ENV HOME=/home/$SERVICE_NAME
-
 VOLUME ["/home/$SERVICE_NAME/log", "/home/$SERVICE_NAME/config"]

-# install the requirements from the wheels packages from the builder stage
+# install the requirements from the wheels packages from the builder stage 
+# and unistall python packages and alpine package manger to reduce attack surface
 COPY --from=builder /root/wheels /root/wheels
+COPY --chmod=0700 ./hardening_final.sh .
 RUN python -m pip install --no-cache --no-index /root/wheels/* && \
-    rm -rf /root/wheels
+    rm -rf /root/wheels && \
+    python -m pip uninstall --yes setuptools wheel pip && \
+    apk --purge del apk-tools && \
+    ./hardening_final.sh && \
+    rm ./hardening_final.sh
+

 # copy the content of the local src and config directory to the working directory
 COPY --chmod=0700 entrypoint.sh /root/entrypoint.sh
--- a/app/build.sh
+++ b/app/build.sh
@@ -22,11 +22,11 @@ fi

 echo version: $VERSION  build-date: $BUILD_DATE   image: $IMAGE
 if [[ $1 == dev ]];then
-docker build --build-arg "VERSION=${VERSION}" --build-arg "LOG_LVL=DEBUG" --label "org.label-schema.build-date=${BUILD_DATE}" --label "org.opencontainers.image.version=${VERSION}" -t ${IMAGE}:latest app
+docker build --build-arg "VERSION=${VERSION}" --build-arg environment=dev --build-arg "LOG_LVL=DEBUG" --label "org.label-schema.build-date=${BUILD_DATE}" --label "org.opencontainers.image.version=${VERSION}" -t ${IMAGE}:latest app
 elif [[ $1 == rc ]];then
-docker build --build-arg "VERSION=${VERSION}" --label "org.label-schema.build-date=${BUILD_DATE}" --label "org.opencontainers.image.version=${VERSION}" -t ${IMAGE}:latest app
+docker build --build-arg "VERSION=${VERSION}" --build-arg environment=production --label "org.label-schema.build-date=${BUILD_DATE}" --label "org.opencontainers.image.version=${VERSION}" -t ${IMAGE}:latest app
 elif [[ $1 == rel ]];then
-docker build --no-cache --build-arg "VERSION=${VERSION}" --label "org.label-schema.build-date=${BUILD_DATE}" --label "org.opencontainers.image.version=${VERSION}" -t ${IMAGE}:latest -t ${IMAGE}:${MAJOR} -t ${IMAGE}:${VERSION} app
+docker build --no-cache --build-arg "VERSION=${VERSION}" --build-arg environment=production --label "org.label-schema.build-date=${BUILD_DATE}" --label "org.opencontainers.image.version=${VERSION}" -t ${IMAGE}:latest -t ${IMAGE}:${MAJOR} -t ${IMAGE}:${VERSION} app
 docker push ghcr.io/s-allius/tsun-gen3-proxy:latest
 docker push ghcr.io/s-allius/tsun-gen3-proxy:${MAJOR}
 docker push ghcr.io/s-allius/tsun-gen3-proxy:${VERSION}
--- a/app/entrypoint.sh
+++ b/app/entrypoint.sh
@@ -11,10 +11,12 @@ if [ "$user" = '0' ]; then
    mkdir -p /home/$SERVICE_NAME/log /home/$SERVICE_NAME/config
    
    if ! id $SERVICE_NAME &> /dev/null; then
+        echo "# create user"
        addgroup --gid $GID $SERVICE_NAME 2> /dev/null
        adduser -G $SERVICE_NAME -s /bin/false -D -H -g "" -u $UID $SERVICE_NAME
+    	chown -R $SERVICE_NAME:$SERVICE_NAME /home/$SERVICE_NAME || true
+        rm -fr /usr/sbin/addgroup /usr/sbin/adduser /bin/chown
    fi    
-	chown -R $SERVICE_NAME:$SERVICE_NAME /home/$SERVICE_NAME || true
    echo "######################################################"
    echo "#"

--- a/app/hardening_base.sh
+++ b/app/hardening_base.sh
@@ -0,0 +1,19 @@
+#!/bin/sh
+
+rm -fr /var/spool/cron
+rm -fr /etc/crontabs
+rm -fr /etc/periodic
+
+# Remove every user and group but root
+sed -i -r '/^(root)/!d' /etc/group
+sed -i -r '/^(root)/!d' /etc/passwd    
+
+# Remove init scripts since we do not use them.
+rm -fr /etc/inittab
+
+# Remove kernel tunables since we do not need them.
+rm -fr /etc/sysctl*
+rm -fr /etc/modprobe.d
+
+# Remove fstab since we do not need it.
+rm -f /etc/fstab
--- a/app/hardening_final.sh
+++ b/app/hardening_final.sh
@@ -0,0 +1,22 @@
+#!/bin/sh
+# For production images delete all uneeded admin commands and remove dangerous commands.
+# addgroup, adduser and chmod will be removed in entrypoint.sh during first start
+# su-exec will be needed for ever restart of the cotainer
+if [ "$environment" = "production" ] ; then \
+  find /sbin /usr/sbin  ! -type d \
+  -a ! -name addgroup \
+  -a ! -name adduser \
+  -a ! -name nologin \
+  -a ! -name su-exec \
+   -delete; \
+  find /bin /usr/bin -xdev \( \
+  -name chgrp -o \
+  -name chmod -o \
+  -name hexdump -o \
+  -name ln -o \
+  -name od -o \
+  -name strings -o \
+  -name su -o \
+  -name wget -o \
+  \) -delete  \
+; fi
--- a/app/src/async_stream.py
+++ b/app/src/async_stream.py
@@ -9,18 +9,22 @@ logger = logging.getLogger('conn')

 class AsyncStream(Message):

-    def __init__(self, reader, writer, addr, remote_stream, server_side: bool
-                 ) -> None:
-        super().__init__(server_side)
+    def __init__(self, reader, writer, addr, remote_stream, server_side: bool,
+                 id_str=b'') -> None:
+        super().__init__(server_side, id_str)
        self.reader = reader
        self.writer = writer
        self.remoteStream = remote_stream
        self.addr = addr
+        self.r_addr = ''
+        self.l_addr = ''

    '''
    Our puplic methods
    '''
-    async def loop(self) -> None:
+    async def loop(self):
+        self.r_addr = self.writer.get_extra_info('peername')
+        self.l_addr = self.writer.get_extra_info('sockname')

        while True:
            try:
@@ -35,26 +39,27 @@ class AsyncStream(Message):
                    ConnectionAbortedError,
                    BrokenPipeError,
                    RuntimeError) as error:
-                logger.warning(f'In loop for {self.addr}: {error}')
+                logger.warning(f'In loop for l{self.l_addr} | '
+                               f'r{self.r_addr}: {error}')
                self.close()
-                return
+                return self
            except Exception:
                logger.error(
                    f"Exception for {self.addr}:\n"
                    f"{traceback.format_exc()}")
                self.close()
-                return
+                return self

    def disc(self) -> None:
-        logger.debug(f'in AsyncStream.disc() {self.addr}')
+        logger.debug(f'in AsyncStream.disc() l{self.l_addr} | r{self.r_addr}')
        self.writer.close()

    def close(self):
-        logger.debug(f'in AsyncStream.close() {self.addr}')
+        logger.debug(f'in AsyncStream.close() l{self.l_addr} | r{self.r_addr}')
        self.writer.close()
        super().close()         # call close handler in the parent class

-#        logger.info (f'AsyncStream refs: {gc.get_referrers(self)}')
+#        logger.info(f'AsyncStream refs: {gc.get_referrers(self)}')

    '''
    Our private methods
@@ -79,6 +84,10 @@ class AsyncStream(Message):
        if self._forward_buffer:
            if not self.remoteStream:
                await self.async_create_remote()
+                if self.remoteStream:
+                    self.remoteStream._init_new_client_conn(self.contact_name,
+                                                            self.contact_mail)
+                    await self.remoteStream.__async_write()

            if self.remoteStream:
                hex_dump_memory(logging.INFO,
@@ -96,4 +105,4 @@ class AsyncStream(Message):
        pass

    def __del__(self):
-        logging.debug(f"AsyncStream.__del__  {self.addr}")
+        logging.debug(f"AsyncStream.__del__  l{self.l_addr} | r{self.r_addr}")
--- a/app/src/infos.py
+++ b/app/src/infos.py
@@ -61,6 +61,7 @@ class Infos:
        0xffffff03:  {'name': ['proxy', 'Invalid_Data_Type'],  'singleton': True,   'ha': {'dev': 'proxy', 'comp': 'sensor', 'dev_cla': None, 'stat_cla': None, 'id': 'inv_data_type_', 'fmt': '| int', 'name': 'Invalid Data Type',    'icon': 'mdi:counter', 'ent_cat': 'diagnostic'}},  # noqa: E501
        0xffffff04:  {'name': ['proxy', 'Internal_Error'],     'singleton': True,   'ha': {'dev': 'proxy', 'comp': 'sensor', 'dev_cla': None, 'stat_cla': None, 'id': 'intern_err_',    'fmt': '| int', 'name': 'Internal Error',       'icon': 'mdi:counter', 'ent_cat': 'diagnostic', 'en': False}},  # noqa: E501
        0xffffff05:  {'name': ['proxy', 'Unknown_Ctrl'],       'singleton': True,   'ha': {'dev': 'proxy', 'comp': 'sensor', 'dev_cla': None, 'stat_cla': None, 'id': 'unknown_ctrl_',  'fmt': '| int', 'name': 'Unknown Control Type', 'icon': 'mdi:counter', 'ent_cat': 'diagnostic'}},  # noqa: E501
+        0xffffff06:  {'name': ['proxy', 'OTA_Start_Msg'],      'singleton': True,   'ha': {'dev': 'proxy', 'comp': 'sensor', 'dev_cla': None, 'stat_cla': None, 'id': 'ota_start_cmd_', 'fmt': '| int', 'name': 'OTA Start Cmd',        'icon': 'mdi:counter', 'ent_cat': 'diagnostic'}},  # noqa: E501
        # 0xffffff03:  {'name':['proxy', 'Voltage'],                        'level': logging.DEBUG, 'unit': 'V',    'ha':{'dev':'proxy', 'dev_cla': 'voltage',     'stat_cla': 'measurement', 'id':'proxy_volt_',  'fmt':'| float','name': 'Grid Voltage'}},  # noqa: E501

        # events
@@ -262,7 +263,7 @@ class Infos:
                                          f"{via}")

                    for key in ('mdl', 'mf', 'sw', 'hw'):      # add optional
-                        # values fpr 'modell', 'manufaturer', 'sw version' and
+                        # values fpr 'modell', 'manufacturer', 'sw version' and
                        # 'hw version'
                        if key in device:
                            data = self.dev_value(device[key])
@@ -341,6 +342,9 @@ class Infos:
            elif data_type == 0x46:  # 'F' -> float32
                result = round(struct.unpack_from('!f', buf, ind)[0], 2)
                ind += 4
+            elif data_type == 0x4c:  # 'L' -> int64
+                result = struct.unpack_from('!q', buf, ind)[0]
+                ind += 8
            else:
                self.inc_counter('Invalid_Data_Type')
                logging.error(f"Infos.parse: data_type: {data_type}"
@@ -374,6 +378,6 @@ class Infos:
                name = str(f'info-id.0x{info_id:x}')

            self.tracer.log(level, f'{name} : {result}{unit}'
-                            '  update: {update}')
+                            f'  update: {update}')

            i += 1
--- a/app/src/inverter.py
+++ b/app/src/inverter.py
@@ -107,7 +107,7 @@ class Inverter(AsyncStream):
        self.inc_counter('Inverter_Cnt')
        await self.loop()
        self.dec_counter('Inverter_Cnt')
-        logging.info(f'Server loop stopped for {addr}')
+        logging.info(f'Server loop stopped for r{self.r_addr}')

        # if the server connection closes, we also have to disconnect
        # the connection to te TSUN cloud
@@ -121,15 +121,22 @@ class Inverter(AsyncStream):

    async def client_loop(self, addr):
        '''Loop for receiving messages from the TSUN cloud (client-side)'''
-        await self.remoteStream.loop()
-        logging.info(f'Client loop stopped for {addr}')
+        clientStream = await self.remoteStream.loop()
+        logging.info(f'Client loop stopped for l{clientStream.l_addr}')

        # if the client connection closes, we don't touch the server
        # connection. Instead we erase the client connection stream,
        # thus on the next received packet from the inverter, we can
        # establish a new connection to the TSUN cloud
-        self.remoteStream.remoteStream = None    # erase backlink to inverter
-        self.remoteStream = None                 # than erase client connection
+
+        # erase backlink to inverter
+        clientStream.remoteStream = None
+
+        if self.remoteStream == clientStream:
+            # logging.debug(f'Client l{clientStream.l_addr} refs:'
+            #               f' {gc.get_referrers(clientStream)}')
+            # than erase client connection
+            self.remoteStream = None

    async def async_create_remote(self) -> None:
        '''Establish a client connection to the TSUN cloud'''
@@ -142,7 +149,8 @@ class Inverter(AsyncStream):
            logging.info(f'Connected to {addr}')
            connect = asyncio.open_connection(host, port)
            reader, writer = await connect
-            self.remoteStream = AsyncStream(reader, writer, addr, self, False)
+            self.remoteStream = AsyncStream(reader, writer, addr, self,
+                                            False, self.id_str)
            asyncio.create_task(self.client_loop(addr))

        except ConnectionRefusedError as error:
@@ -197,7 +205,7 @@ class Inverter(AsyncStream):
                                    f"/{node_id}{id}/config", data_json)

    def close(self) -> None:
-        logging.debug(f'Inverter.close() {self.addr}')
+        logging.debug(f'Inverter.close() l{self.l_addr} | r{self.r_addr}')
        super().close()         # call close handler in the parent class
 #        logger.debug (f'Inverter refs: {gc.get_referrers(self)}')

--- a/app/src/messages.py
+++ b/app/src/messages.py
@@ -55,8 +55,8 @@ class Control:
    def is_ind(self) -> bool:
        return (self.ctrl == 0x91)

-    # def is_req(self) -> bool:
-    #    return not (self.ctrl & 0x08)
+    def is_req(self) -> bool:
+        return (self.ctrl == 0x70)

    def is_resp(self) -> bool:
        return (self.ctrl == 0x99)
@@ -74,7 +74,7 @@ class Message(metaclass=IterRegistry):
    _registry = []
    new_stat_data = {}

-    def __init__(self, server_side: bool):
+    def __init__(self, server_side: bool, id_str=b''):
        self._registry.append(weakref.ref(self))
        self.server_side = server_side
        self.header_valid = False
@@ -83,6 +83,9 @@ class Message(metaclass=IterRegistry):
        self.unique_id = 0
        self.node_id = ''
        self.sug_area = ''
+        self.id_str = id_str
+        self.contact_name = b''
+        self.contact_mail = b''
        self._recv_buffer = b''
        self._send_buffer = bytearray(0)
        self._forward_buffer = bytearray(0)
@@ -90,6 +93,7 @@ class Message(metaclass=IterRegistry):
        self.new_data = {}
        self.switch = {
            0x00: self.msg_contact_info,
+            0x13: self.msg_ota_update,
            0x22: self.msg_get_time,
            0x71: self.msg_collector_data,
            0x04: self.msg_inverter_data,
@@ -110,7 +114,7 @@ class Message(metaclass=IterRegistry):
        # we have refernces to methods of this class in self.switch
        # so we have to erase self.switch, otherwise this instance can't be
        # deallocated by the garbage collector ==> we get a memory leak
-        del self.switch
+        self.switch.clear()

    def inc_counter(self, counter: str) -> None:
        self.db.inc_counter(counter)
@@ -175,6 +179,16 @@ class Message(metaclass=IterRegistry):
                        f' Ctl: {int(self.ctrl):#02x} Msg: {fnc.__name__!r}')
        return

+    def _init_new_client_conn(self, contact_name, contact_mail) -> None:
+        logger.info(f'name: {contact_name} mail: {contact_mail}')
+        self.msg_id = 0
+        self.__build_header(0x91)
+        self._send_buffer += struct.pack(f'!{len(contact_name)+1}p'
+                                         f'{len(contact_mail)+1}p',
+                                         contact_name, contact_mail)
+
+        self.__finish_send_msg()
+
    '''
    Our private methods
    '''
@@ -194,7 +208,7 @@ class Message(metaclass=IterRegistry):
            type += 'S'
        return switch.get(type, '???')

-    def __timestamp(self):
+    def _timestamp(self):   # pragma: no cover
        if False:
            # utc as epoche
            ts = time.time()
@@ -271,30 +285,58 @@ class Message(metaclass=IterRegistry):
            self.__build_header(0x99)
            self._send_buffer += b'\x01'
            self.__finish_send_msg()
+            self.__process_contact_info()
+            # don't forward this contact info here, we will build one
+            # when the remote connection is established
+            return
        elif self.ctrl.is_resp():
            return  # ignore received response from tsun
        else:
            self.inc_counter('Unknown_Ctrl')
+            self.forward(self._recv_buffer, self.header_len+self.data_len)

-        self.forward(self._recv_buffer, self.header_len+self.data_len)
+    def __process_contact_info(self):
+        result = struct.unpack_from('!B', self._recv_buffer, self.header_len)
+        name_len = result[0]
+
+        result = struct.unpack_from(f'!{name_len+1}pB', self._recv_buffer,
+                                    self.header_len)
+        self.contact_name = result[0]
+        mail_len = result[1]
+        logger.info(f'name: {self.contact_name}')
+
+        result = struct.unpack_from(f'!{mail_len+1}p', self._recv_buffer,
+                                    self.header_len+name_len+1)
+        self.contact_mail = result[0]
+        logger.info(f'mail: {self.contact_mail}')

    def msg_get_time(self):
-        if self.ctrl.is_ind():
-            ts = self.__timestamp()
-            logger.debug(f'time: {ts:08x}')
-
-            self.__build_header(0x99)
-            self._send_buffer += struct.pack('!q', ts)
-            self.__finish_send_msg()
-
-        elif self.ctrl.is_resp():
-            result = struct.unpack_from('!q', self._recv_buffer,
-                                        self.header_len)
-            logger.debug(f'tsun-time: {result[0]:08x}')
-            return  # ignore received response from tsun
+        tsun = Config.get('tsun')
+        if tsun['enabled']:
+            if self.ctrl.is_resp():
+                ts = self._timestamp()
+                result = struct.unpack_from('!q', self._recv_buffer,
+                                            self.header_len)
+                logger.debug(f'tsun-time: {result[0]:08x}'
+                             f'  proxy-time: {ts:08x}')
+            elif not self.ctrl.is_ind():
+                self.inc_counter('Unknown_Ctrl')
+            self.forward(self._recv_buffer, self.header_len+self.data_len)
        else:
-            self.inc_counter('Unknown_Ctrl')
-        self.forward(self._recv_buffer, self.header_len+self.data_len)
+            if self.ctrl.is_ind():
+                ts = self._timestamp()
+                logger.debug(f'time: {ts:08x}')
+
+                self.__build_header(0x99)
+                self._send_buffer += struct.pack('!q', ts)
+                self.__finish_send_msg()
+
+            elif self.ctrl.is_resp():
+                result = struct.unpack_from('!q', self._recv_buffer,
+                                            self.header_len)
+                logger.debug(f'tsun-time: {result[0]:08x}')
+            else:
+                self.inc_counter('Unknown_Ctrl')

    def parse_msg_header(self):
        result = struct.unpack_from('!lB', self._recv_buffer, self.header_len)
@@ -350,6 +392,15 @@ class Message(metaclass=IterRegistry):
            if update:
                self.new_data[key] = True

+    def msg_ota_update(self):
+        if self.ctrl.is_req():
+            self.inc_counter('OTA_Start_Msg')
+        elif self.ctrl.is_ind():
+            pass
+        else:
+            self.inc_counter('Unknown_Ctrl')
+        self.forward(self._recv_buffer, self.header_len+self.data_len)
+
    def msg_unknown(self):
        logger.warning(f"Unknow Msg: ID:{self.msg_id}")
        self.inc_counter('Unknown_Msg')
--- a/app/tests/test_infos.py
+++ b/app/tests/test_infos.py
@@ -12,16 +12,96 @@ def ContrDataSeq(): # Get Time Request message
    msg +=  b'\x49\x00\x00\x00\x02\x00\x0d\x04\x08\x49\x00\x00\x00\x00\x00\x07\xa1\x84\x49\x00\x00\x00\x01\x00\x0c\x50\x59\x49\x00\x00\x00\x4c\x00\x0d\x1f\x60\x49\x00\x00\x00\x00'
    return msg

+@pytest.fixture
+def Contr2DataSeq(): # Get Time Request message
+    msg  =  b'\x00\x00\x00\x39\x00\x09\x2b\xa8\x54\x10\x52'
+    msg +=  b'\x53\x57\x5f\x34\x30\x30\x5f\x56\x31\x2e\x30\x30\x2e\x32\x30\x00'
+    msg +=  b'\x09\x27\xc0\x54\x06\x52\x61\x79\x6d\x6f\x6e\x00\x09\x2f\x90\x54'
+    msg +=  b'\x0b\x52\x53\x57\x2d\x31\x2d\x31\x30\x30\x30\x31\x00\x09\x5a\x88'
+    msg +=  b'\x54\x0f\x74\x2e\x72\x61\x79\x6d\x6f\x6e\x69\x6f\x74\x2e\x63\x6f'
+    msg +=  b'\x6d\x00\x09\x5a\xec\x54\x1c\x6c\x6f\x67\x67\x65\x72\x2e\x74\x61'
+    msg +=  b'\x6c\x65\x6e\x74\x2d\x6d\x6f\x6e\x69\x74\x6f\x72\x69\x6e\x67\x2e'
+    msg +=  b'\x63\x6f\x6d\x00\x0d\x2f\x00\x54\x10\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\x00\x0d\x32\xe8\x54\x10\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x00'
+    msg +=  b'\x0d\x36\xd0\x54\x10\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\x00\x0d\x3a\xb8\x54\x10\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x00\x0d\x3e\xa0\x54'
+    msg +=  b'\x10\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\x00\x0d\x42\x88\x54\x10\xff\xff\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\x00\x0d\x46\x70\x54\x10\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x00\x0d\x4a'
+    msg +=  b'\x58\x54\x10\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\x00\x0d\x4e\x40\x54\x10\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\x00\x0d\x52\x28\x54\x10\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x00'
+    msg +=  b'\x0d\x56\x10\x54\x10\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\x00\x0d\x59\xf8\x54\x10\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x00\x0d\x5d\xe0\x54'
+    msg +=  b'\x10\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\x00\x0d\x61\xc8\x54\x10\xff\xff\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\x00\x0d\x65\xb0\x54\x10\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x00\x0d\x69'
+    msg +=  b'\x98\x54\x10\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\x00\x0d\x6d\x80\x54\x10\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\x00\x0d\x71\x68\x54\x10\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x00'
+    msg +=  b'\x0d\x75\x50\x54\x10\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\x00\x0d\x79\x38\x54\x10\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x00\x0d\x7d\x20\x54'
+    msg +=  b'\x10\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\x00\x0d\x81\x08\x54\x10\xff\xff\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\x00\x0d\x84\xf0\x54\x10\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x00\x0d\x88'
+    msg +=  b'\xd8\x54\x10\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\x00\x0d\x8c\xc0\x54\x10\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\x00\x0d\x90\xa8\x54\x10\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x00'
+    msg +=  b'\x0d\x94\x90\x54\x10\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\x00\x0d\x98\x78\x54\x10\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x00\x0d\x9c\x60\x54'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff'
+    msg +=  b'\x00\x0d\x00\x20\x49\x00\x00\x00\x01\x00\x0c\x35\x00\x49\x00\x00'
+    msg +=  b'\x00\x10\x00\x0c\x96\xa8\x49\x00\x00\x01\x4e\x00\x0c\x7f\x38\x49'
+    msg +=  b'\x00\x00\x00\x01\x00\x0c\xfc\x38\x49\x00\x00\x00\x01\x00\x0c\xf8'
+    msg +=  b'\x50\x49\x00\x00\x01\x2c\x00\x0c\x63\xe0\x49\x00\x00\x00\x00\x00'
+    msg +=  b'\x0c\x67\xc8\x49\x00\x00\x00\x00\x00\x0c\x50\x58\x49\x00\x00\x00'
+    msg +=  b'\x01\x00\x09\x5e\x70\x49\x00\x00\x13\x8d\x00\x09\x5e\xd4\x49\x00'
+    msg +=  b'\x00\x13\x8d\x00\x09\x5b\x50\x49\x00\x00\x00\x02\x00\x0d\x04\x08'
+    msg +=  b'\x49\x00\x00\x00\x00\x00\x07\xa1\x84\x49\x00\x00\x00\x01\x00\x0c'
+    msg +=  b'\x50\x59\x49\x00\x00\x00\x33\x00\x0d\x1f\x60\x49\x00\x00\x00\x00'
+    msg +=  b'\x00\x0d\x23\x48\x49\xff\xff\xff\xff\x00\x0d\x27\x30\x49\xff\xff'
+    msg +=  b'\xff\xff\x00\x0d\x2b\x18\x4c\x00\x00\x00\x00\x00\x00\xff\xff\x00'
+    msg +=  b'\x0c\xa2\x60\x49\x00\x00\x00\x00\x00\x0d\xa0\x48\x49\x00\x00\x00'
+    msg +=  b'\x00\x00\x0d\xa4\x30\x49\x00\x00\x00\x00\x00\x0d\xa8\x18\x49\x00'
+    msg +=  b'\x00\x00\x00'
+    return msg
+
@pytest.fixture
 def InvDataSeq(): # Data indication from the controller
    msg =   b'\x00\x00\x00\x06\x00\x00\x00\x0a\x54\x08\x4d\x69\x63\x72\x6f\x69\x6e\x76\x00\x00\x00\x14\x54\x04\x54\x53\x55\x4e\x00\x00\x00\x1E\x54\x07\x56\x35\x2e\x30\x2e\x31\x31\x00\x00\x00\x28'
-    msg +=  b'\x54\x10\x54\x31\x37\x45\x37\x33\x30\x37\x30\x32\x31\x44\x30\x30\x36\x41\x00\x00\x00\x32\x54\x0a\x54\x53\x4f\x4c\x2d\x4d\x53\x36\x30\x30\x00\x00\x00\x3c\x54\x05\x41\x2c\x42\x2c\x43'
+    msg +=  b'\x54\x10T170000000000001\x00\x00\x00\x32\x54\x0a\x54\x53\x4f\x4c\x2d\x4d\x53\x36\x30\x30\x00\x00\x00\x3c\x54\x05\x41\x2c\x42\x2c\x43'
    return msg

@pytest.fixture
 def InvalidDataSeq(): # Data indication from the controller
    msg =   b'\x00\x00\x00\x06\x00\x00\x00\x0a\x54\x08\x4d\x69\x63\x72\x6f\x69\x6e\x76\x00\x00\x00\x14\x64\x04\x54\x53\x55\x4e\x00\x00\x00\x1E\x54\x07\x56\x35\x2e\x30\x2e\x31\x31\x00\x00\x00\x28'
-    msg +=  b'\x54\x10\x54\x31\x37\x45\x37\x33\x30\x37\x30\x32\x31\x44\x30\x30\x36\x41\x00\x00\x00\x32\x54\x0a\x54\x53\x4f\x4c\x2d\x4d\x53\x36\x30\x30\x00\x00\x00\x3c\x54\x05\x41\x2c\x42\x2c\x43'
+    msg +=  b'\x54\x10T170000000000001\x00\x00\x00\x32\x54\x0a\x54\x53\x4f\x4c\x2d\x4d\x53\x36\x30\x30\x00\x00\x00\x3c\x54\x05\x41\x2c\x42\x2c\x43'
    return msg

@pytest.fixture
@@ -103,13 +183,21 @@ def test_parse_control(ContrDataSeq):
    assert json.dumps(i.db) == json.dumps(
 {"collector": {"Collector_Fw_Version": "RSW_400_V1.00.06", "Chip_Type": "Raymon", "Chip_Model": "RSW-1-10001", "Trace_URL": "t.raymoniot.com", "Logger_URL": "logger.talent-monitoring.com", "No_Inputs": 2}, "controller": {"Signal_Strength": 100, "Power_On_Time": 29, "Data_Up_Interval": 300}})        

+def test_parse_control2(Contr2DataSeq):
+    i = Infos()
+    for key, result in i.parse (Contr2DataSeq):
+        pass
+
+    assert json.dumps(i.db) == json.dumps(
+{"collector": {"Collector_Fw_Version": "RSW_400_V1.00.20", "Chip_Type": "Raymon", "Chip_Model": "RSW-1-10001", "Trace_URL": "t.raymoniot.com", "Logger_URL": "logger.talent-monitoring.com", "No_Inputs": 2}, "controller": {"Signal_Strength": 16, "Power_On_Time": 334, "Data_Up_Interval": 300}})
+
 def test_parse_inverter(InvDataSeq):
    i = Infos()
    for key, result in i.parse (InvDataSeq):
        pass

    assert json.dumps(i.db) == json.dumps(
-{"inverter": {"Product_Name": "Microinv", "Manufacturer": "TSUN", "Version": "V5.0.11", "Serial_Number": "T17E7307021D006A", "Equipment_Model": "TSOL-MS600"}})
+{"inverter": {"Product_Name": "Microinv", "Manufacturer": "TSUN", "Version": "V5.0.11", "Serial_Number": "T170000000000001", "Equipment_Model": "TSOL-MS600"}})

 def test_parse_cont_and_invert(ContrDataSeq, InvDataSeq):
    i = Infos()
@@ -122,7 +210,7 @@ def test_parse_cont_and_invert(ContrDataSeq, InvDataSeq):
    assert json.dumps(i.db) == json.dumps(
    {
 "collector": {"Collector_Fw_Version": "RSW_400_V1.00.06", "Chip_Type": "Raymon", "Chip_Model": "RSW-1-10001", "Trace_URL": "t.raymoniot.com", "Logger_URL": "logger.talent-monitoring.com", "No_Inputs": 2}, "controller": {"Signal_Strength": 100, "Power_On_Time": 29, "Data_Up_Interval": 300},
-"inverter": {"Product_Name": "Microinv", "Manufacturer": "TSUN", "Version": "V5.0.11", "Serial_Number": "T17E7307021D006A", "Equipment_Model": "TSOL-MS600"}})
+"inverter": {"Product_Name": "Microinv", "Manufacturer": "TSUN", "Version": "V5.0.11", "Serial_Number": "T170000000000001", "Equipment_Model": "TSOL-MS600"}})


 def test_build_ha_conf1(ContrDataSeq):
@@ -320,13 +408,13 @@ def test_statistic_counter():
    assert val == None or val == 0

    i.static_init()                # initialize counter
-    assert json.dumps(i.stat) == json.dumps({"proxy": {"Inverter_Cnt": 0, "Unknown_SNR": 0, "Unknown_Msg": 0, "Invalid_Data_Type": 0, "Internal_Error": 0,"Unknown_Ctrl": 0}})
+    assert json.dumps(i.stat) == json.dumps({"proxy": {"Inverter_Cnt": 0, "Unknown_SNR": 0, "Unknown_Msg": 0, "Invalid_Data_Type": 0, "Internal_Error": 0,"Unknown_Ctrl": 0, "OTA_Start_Msg": 0}})
                                            
    val = i.dev_value(0xffffff00)  # valid and initiliazed addr
    assert val == 0

    i.inc_counter('Inverter_Cnt')
-    assert json.dumps(i.stat) == json.dumps({"proxy": {"Inverter_Cnt": 1, "Unknown_SNR": 0, "Unknown_Msg": 0, "Invalid_Data_Type": 0, "Internal_Error": 0,"Unknown_Ctrl": 0}})
+    assert json.dumps(i.stat) == json.dumps({"proxy": {"Inverter_Cnt": 1, "Unknown_SNR": 0, "Unknown_Msg": 0, "Invalid_Data_Type": 0, "Internal_Error": 0,"Unknown_Ctrl": 0, "OTA_Start_Msg": 0}})
    val = i.dev_value(0xffffff00)
    assert val == 1

--- a/app/tests/test_messages.py
+++ b/app/tests/test_messages.py
@@ -42,6 +42,8 @@ class MemoryStream(Message):
            pass     
        return copied_bytes
    
+    def _timestamp(self):
+        return 1700260990000
    
    def _Message__flush_recv_msg(self) -> None:
        super()._Message__flush_recv_msg()
@@ -67,6 +69,10 @@ def Msg2ContactInfo(): # two Contact Info messages
 def MsgContactResp(): # Contact Response message
    return b'\x00\x00\x00\x14\x10R170000000000001\x99\x00\x01'

+@pytest.fixture
+def MsgContactResp2(): # Contact Response message
+    return b'\x00\x00\x00\x14\x10R170000000000002\x99\x00\x01'
+
@pytest.fixture
 def MsgContactInvalid(): # Contact Response message
    return b'\x00\x00\x00\x14\x10R170000000000001\x93\x00\x01'
@@ -108,7 +114,7 @@ def MsgInverterInd(): # Data indication from the controller
    msg  =  b'\x00\x00\x00\x8b\x10R170000000000001\x91\x04\x01\x90\x00\x01\x10R170000000000001'
    msg +=  b'\x01\x00\x00\x01\x89\xc6\x63\x61\x08'
    msg +=  b'\x00\x00\x00\x06\x00\x00\x00\x0a\x54\x08\x4d\x69\x63\x72\x6f\x69\x6e\x76\x00\x00\x00\x14\x54\x04\x54\x53\x55\x4e\x00\x00\x00\x1E\x54\x07\x56\x35\x2e\x30\x2e\x31\x31\x00\x00\x00\x28'
-    msg +=  b'\x54\x10\x54\x31\x37\x45\x37\x33\x30\x37\x30\x32\x31\x44\x30\x30\x36\x41\x00\x00\x00\x32\x54\x0a\x54\x53\x4f\x4c\x2d\x4d\x53\x36\x30\x30\x00\x00\x00\x3c\x54\x05\x41\x2c\x42\x2c\x43'
+    msg +=  b'\x54\x10T170000000000001\x00\x00\x00\x32\x54\x0a\x54\x53\x4f\x4c\x2d\x4d\x53\x36\x30\x30\x00\x00\x00\x3c\x54\x05\x41\x2c\x42\x2c\x43'
    return msg

@pytest.fixture
@@ -123,15 +129,6 @@ def MsgInverterInvalid(): # Get Time Request message
 def MsgUnknown(): # Get Time Request message
    return b'\x00\x00\x00\x17\x10R170000000000001\x91\x17\x01\x02\x03\x04'

-@pytest.fixture
-def MsgGetTime(): # Get Time Request message
-    return b'\x00\x00\x00\x13\x10R170000000000001\x91\x22'           
-
-
-@pytest.fixture
-def MsgTimeResp(): # Get Time Resonse message
-    return b'\x00\x00\x00\x1b\x10R170000000000001\x99\x22\x00\x00\x01\x89\xc6\x63\x4d\x80'
-
@pytest.fixture
 def ConfigTsunAllowAll():
    Config.config = {'tsun':{'enabled': True}, 'inverters':{'allow_all':True}}
@@ -144,6 +141,36 @@ def ConfigNoTsunInv1():
 def ConfigTsunInv1():
    Config.config = {'tsun':{'enabled': True},'inverters':{'R170000000000001':{'node_id':'inv1','suggested_area':'roof'}}}

+@pytest.fixture
+def MsgOtaReq(): # Over the air update rewuest from tsun cloud
+    msg  =  b'\x00\x00\x01\x16\x10R170000000000001\x70\x13\x01\x02\x76\x35\x70\x68\x74\x74\x70'
+    msg +=  b'\x3a\x2f\x2f\x77\x77\x77\x2e\x74\x61\x6c\x65\x6e\x74\x2d\x6d\x6f'
+    msg +=  b'\x6e\x69\x74\x6f\x72\x69\x6e\x67\x2e\x63\x6f\x6d\x3a\x39\x30\x30'
+    msg +=  b'\x32\x2f\x70\x72\x6f\x64\x2d\x61\x70\x69\x2f\x72\x6f\x6d\x2f\x75'
+    msg +=  b'\x70\x64\x61\x74\x65\x2f\x64\x6f\x77\x6e\x6c\x6f\x61\x64\x3f\x76'
+    msg +=  b'\x65\x72\x3d\x56\x31\x2e\x30\x30\x2e\x31\x37\x26\x6e\x61\x6d\x65'
+    msg +=  b'\x3d\x47\x33\x2d\x57\x69\x46\x69\x2b\x2d\x56\x31\x2e\x30\x30\x2e'
+    msg +=  b'\x31\x37\x2d\x4f\x54\x41\x26\x65\x78\x74\x3d\x30\x60\x68\x74\x74'
+    msg +=  b'\x70\x3a\x2f\x2f\x77\x77\x77\x2e\x74\x61\x6c\x65\x6e\x74\x2d\x6d'
+    msg +=  b'\x6f\x6e\x69\x74\x6f\x72\x69\x6e\x67\x2e\x63\x6f\x6d\x3a\x39\x30'
+    msg +=  b'\x30\x32\x2f\x70\x72\x6f\x64\x2d\x61\x70\x69\x2f\x72\x6f\x6d\x2f'
+    msg +=  b'\x75\x70\x64\x61\x74\x65\x2f\x63\x61\x6c\x6c\x62\x61\x63\x6b\x3f'
+    msg +=  b'\x71\x69\x64\x3d\x31\x35\x30\x33\x36\x32\x26\x72\x69\x64\x3d\x32'
+    msg +=  b'\x32\x39\x26\x64\x69\x64\x3d\x31\x33\x34\x32\x32\x35\x20\x36\x35'
+    msg +=  b'\x66\x30\x64\x37\x34\x34\x62\x66\x33\x39\x61\x62\x38\x32\x34\x64'
+    msg +=  b'\x32\x38\x62\x38\x34\x64\x31\x39\x65\x64\x33\x31\x31\x63\x06\x34'
+    msg +=  b'\x36\x38\x36\x33\x33\x01\x31\x01\x30\x00'  
+    return msg    
+
+@pytest.fixture
+def MsgOtaAck(): # Over the air update rewuest from tsun cloud
+    return b'\x00\x00\x00\x14\x10R170000000000001\x91\x13\x01'
+
+@pytest.fixture
+def MsgOtaInvalid(): # Get Time Request message
+    return b'\x00\x00\x00\x14\x10R170000000000001\x99\x13\x01'
+
+
 def test_read_message(MsgContactInfo):
    m = MemoryStream(MsgContactInfo, (0,))
    m.read()         # read complete msg, and dispatch msg
@@ -246,7 +273,7 @@ def test_read_message_in_chunks2(MsgContactInfo):
    assert not m.header_valid  # must be invalid, since msg was handled and buffer flushed
    m.close()

-def test_read_two_messages(ConfigTsunAllowAll, Msg2ContactInfo):
+def test_read_two_messages(ConfigTsunAllowAll, Msg2ContactInfo,MsgContactResp,MsgContactResp2):
    ConfigTsunAllowAll
    m = MemoryStream(Msg2ContactInfo, (0,))
    m.db.stat['proxy']['Unknown_Ctrl'] = 0
@@ -259,9 +286,15 @@ def test_read_two_messages(ConfigTsunAllowAll, Msg2ContactInfo):
    assert m.msg_id==0 
    assert m.header_len==23
    assert m.data_len==25
-    assert m._forward_buffer==b'\x00\x00\x00,\x10R170000000000001\x91\x00\x08solarhub\x0fsolarhub@123456'
+    assert m._forward_buffer==b''
+    assert m._send_buffer==MsgContactResp
    assert m.db.stat['proxy']['Unknown_Ctrl'] == 0

+    m._send_buffer = bytearray(0) # clear send buffer for next test
+    m._init_new_client_conn(b'solarhub', b'solarhub@123456')
+    assert m._send_buffer==b'\x00\x00\x00,\x10R170000000000001\x91\x00\x08solarhub\x0fsolarhub@123456'
+
+    m._send_buffer = bytearray(0) # clear send buffer for next test
    m.read()         # read complete msg, and dispatch msg
    assert not m.header_valid  # must be invalid, since msg was handled and buffer flushed
    assert m.msg_count == 2
@@ -271,8 +304,13 @@ def test_read_two_messages(ConfigTsunAllowAll, Msg2ContactInfo):
    assert m.msg_id==0 
    assert m.header_len==23
    assert m.data_len==25
-    assert m._forward_buffer==b'\x00\x00\x00,\x10R170000000000002\x91\x00\x08solarhub\x0fsolarhub@123456'
+    assert m._forward_buffer==b''
+    assert m._send_buffer==MsgContactResp2
    assert m.db.stat['proxy']['Unknown_Ctrl'] == 0
+
+    m._send_buffer = bytearray(0) # clear send buffer for next test
+    m._init_new_client_conn(b'solarhub', b'solarhub@123456')
+    assert m._send_buffer==b'\x00\x00\x00,\x10R170000000000002\x91\x00\x08solarhub\x0fsolarhub@123456'
    m.close()

 def test_msg_contact_resp(ConfigTsunInv1, MsgContactResp):
@@ -289,6 +327,7 @@ def test_msg_contact_resp(ConfigTsunInv1, MsgContactResp):
    assert m.header_len==23
    assert m.data_len==1
    assert m._forward_buffer==b''
+    assert m._send_buffer==b''
    assert m.db.stat['proxy']['Unknown_Ctrl'] == 0
    m.close()

@@ -324,6 +363,25 @@ def test_msg_get_time(ConfigTsunInv1, MsgGetTime):
    assert m.header_len==23
    assert m.data_len==0
    assert m._forward_buffer==MsgGetTime
+    assert m._send_buffer==b''
+    assert m.db.stat['proxy']['Unknown_Ctrl'] == 0
+    m.close()
+
+def test_msg_get_time_autark(ConfigNoTsunInv1, MsgGetTime):
+    ConfigNoTsunInv1
+    m = MemoryStream(MsgGetTime, (0,))
+    m.db.stat['proxy']['Unknown_Ctrl'] = 0
+    m.read()         # read complete msg, and dispatch msg
+    assert not m.header_valid  # must be invalid, since msg was handled and buffer flushed
+    assert m.msg_count == 1
+    assert m.id_str == b"R170000000000001" 
+    assert m.unique_id == 'R170000000000001'
+    assert int(m.ctrl)==145
+    assert m.msg_id==34
+    assert m.header_len==23
+    assert m.data_len==0
+    assert m._forward_buffer==b''
+    assert m._send_buffer==b'\x00\x00\x00\x1b\x10R170000000000001\x99"\x00\x00\x01\x8b\xdfs\xcc0'
    assert m.db.stat['proxy']['Unknown_Ctrl'] == 0
    m.close()

@@ -340,7 +398,26 @@ def test_msg_time_resp(ConfigTsunInv1, MsgTimeResp):
    assert m.msg_id==34
    assert m.header_len==23
    assert m.data_len==8
+    assert m._forward_buffer==MsgTimeResp
+    assert m._send_buffer==b''
+    assert m.db.stat['proxy']['Unknown_Ctrl'] == 0
+    m.close()
+
+def test_msg_time_resp_autark(ConfigNoTsunInv1, MsgTimeResp):
+    ConfigNoTsunInv1
+    m = MemoryStream(MsgTimeResp, (0,), False)
+    m.db.stat['proxy']['Unknown_Ctrl'] = 0
+    m.read()         # read complete msg, and dispatch msg
+    assert not m.header_valid  # must be invalid, since msg was handled and buffer flushed
+    assert m.msg_count == 1
+    assert m.id_str == b"R170000000000001" 
+    assert m.unique_id == 'R170000000000001'
+    assert int(m.ctrl)==153
+    assert m.msg_id==34
+    assert m.header_len==23
+    assert m.data_len==8
    assert m._forward_buffer==b''
+    assert m._send_buffer==b''
    assert m.db.stat['proxy']['Unknown_Ctrl'] == 0
    m.close()

@@ -362,6 +439,23 @@ def test_msg_time_invalid(ConfigTsunInv1, MsgTimeInvalid):
    assert m.db.stat['proxy']['Unknown_Ctrl'] == 1
    m.close()

+def test_msg_time_invalid_autark(ConfigNoTsunInv1, MsgTimeInvalid):
+    ConfigNoTsunInv1
+    m = MemoryStream(MsgTimeInvalid, (0,), False)
+    m.db.stat['proxy']['Unknown_Ctrl'] = 0
+    m.read()         # read complete msg, and dispatch msg
+    assert not m.header_valid  # must be invalid, since msg was handled and buffer flushed
+    assert m.msg_count == 1
+    assert m.id_str == b"R170000000000001" 
+    assert m.unique_id == 'R170000000000001'
+    assert int(m.ctrl)==148
+    assert m.msg_id==34
+    assert m.header_len==23
+    assert m.data_len==0
+    assert m._forward_buffer==b''
+    assert m._send_buffer==b''
+    assert m.db.stat['proxy']['Unknown_Ctrl'] == 1
+    m.close()

 def test_msg_cntrl_ind(ConfigTsunInv1, MsgControllerInd, MsgControllerAck):
    ConfigTsunInv1
@@ -474,6 +568,68 @@ def test_msg_inv_invalid(ConfigTsunInv1, MsgInverterInvalid):
    assert m.db.stat['proxy']['Unknown_Ctrl'] == 1
    m.close()

+def test_msg_ota_req(ConfigTsunInv1, MsgOtaReq):
+    ConfigTsunInv1
+    m = MemoryStream(MsgOtaReq, (0,), False)
+    m.db.stat['proxy']['Unknown_Ctrl'] = 0
+    m.db.stat['proxy']['OTA_Start_Msg'] = 0
+    m.read()         # read complete msg, and dispatch msg
+    assert not m.header_valid  # must be invalid, since msg was handled and buffer flushed
+    assert m.msg_count == 1
+    assert m.id_str == b"R170000000000001" 
+    assert m.unique_id == 'R170000000000001'
+    assert int(m.ctrl)==112
+    assert m.msg_id==19
+    assert m.header_len==23
+    assert m.data_len==259
+    assert m._forward_buffer==MsgOtaReq
+    assert m._send_buffer==b''
+    assert m.db.stat['proxy']['Unknown_Ctrl'] == 0
+    assert m.db.stat['proxy']['OTA_Start_Msg'] == 1
+    m.close()
+
+def test_msg_ota_ack(ConfigTsunInv1, MsgOtaAck):
+    ConfigTsunInv1
+    tracer.setLevel(logging.ERROR)
+
+    m = MemoryStream(MsgOtaAck, (0,), False)
+    m.db.stat['proxy']['Unknown_Ctrl'] = 0
+    m.db.stat['proxy']['OTA_Start_Msg'] = 0
+    m.read()         # read complete msg, and dispatch msg
+    assert not m.header_valid  # must be invalid, since msg was handled and buffer flushed
+    assert m.msg_count == 1
+    assert m.id_str == b"R170000000000001" 
+    assert m.unique_id == 'R170000000000001'
+    assert int(m.ctrl)==145
+    assert m.msg_id==19
+    assert m.header_len==23
+    assert m.data_len==1
+    assert m._forward_buffer==MsgOtaAck
+    assert m._send_buffer==b''
+    assert m.db.stat['proxy']['Unknown_Ctrl'] == 0
+    assert m.db.stat['proxy']['OTA_Start_Msg'] == 0
+    m.close()
+
+def test_msg_ota_invalid(ConfigTsunInv1, MsgOtaInvalid):
+    ConfigTsunInv1
+    m = MemoryStream(MsgOtaInvalid, (0,), False)
+    m.db.stat['proxy']['Unknown_Ctrl'] = 0
+    m.db.stat['proxy']['OTA_Start_Msg'] = 0
+    m.read()         # read complete msg, and dispatch msg
+    assert not m.header_valid  # must be invalid, since msg was handled and buffer flushed
+    assert m.msg_count == 1
+    assert m.id_str == b"R170000000000001" 
+    assert m.unique_id == 'R170000000000001'
+    assert int(m.ctrl)==153
+    assert m.msg_id==19
+    assert m.header_len==23
+    assert m.data_len==1
+    assert m._forward_buffer==MsgOtaInvalid
+    assert m._send_buffer==b''
+    assert m.db.stat['proxy']['Unknown_Ctrl'] == 1
+    assert m.db.stat['proxy']['OTA_Start_Msg'] == 0
+    m.close()
+
 def test_msg_unknown(ConfigTsunInv1, MsgUnknown):
    ConfigTsunInv1
    m = MemoryStream(MsgUnknown, (0,), False)
--- a/system_tests/test_tcp_socket.py
+++ b/system_tests/test_tcp_socket.py
@@ -92,8 +92,29 @@ def MsgInverterInd(): # Data indication from the inverter
    msg +=  b'\x31\x38\x46\x42\x08\x00\x00\x00\x00\x31\x9c\x53\x00\x05\x00\x00\x32\x00\x53\x04\x00\x00\x00\x32\x64\x53\x00\x01\x00\x00\x32\xc8\x53\x13\x9c\x00\x00\x33\x2c\x53\x0f'
    msg +=  b'\xa0\x00\x00\x33\x90\x53\x00\x4f\x00\x00\x33\xf4\x53\x00\x66\x00\x00\x34\x58\x53\x03\xe8\x00\x00\x34\xbc\x53\x04\x00\x00\x00\x35\x20\x53\x00\x00\x00\x00\x35\x84\x53'
    msg +=  b'\x00\x00\x00\x00\x35\xe8\x53\x00\x00\x00\x00\x36\x4c\x53\x00\x00\x00\x01\x38\x80\x53\x00\x02\x00\x01\x38\x81\x53\x00\x01\x00\x01\x38\x82\x53\x00\x01\x00\x01\x38\x83'
-    msg +=  b'\x53\x00\x00' 
-    
+    msg +=  b'\x53\x00\x00'     
+    return msg
+
+@pytest.fixture
+def MsgOtaUpdateReq(): # Over the air update request from talent cloud
+    msg  = b'\x00\x00\x01\x16\x10'+ get_sn() + b'\x70\x13\x01\x02\x76\x35'
+    msg += b'\x70\x68\x74\x74\x70'
+    msg += b'\x3a\x2f\x2f\x77\x77\x77\x2e\x74\x61\x6c\x65\x6e\x74\x2d\x6d\x6f'
+    msg += b'\x6e\x69\x74\x6f\x72\x69\x6e\x67\x2e\x63\x6f\x6d\x3a\x39\x30\x30'
+    msg += b'\x32\x2f\x70\x72\x6f\x64\x2d\x61\x70\x69\x2f\x72\x6f\x6d\x2f\x75'
+    msg += b'\x70\x64\x61\x74\x65\x2f\x64\x6f\x77\x6e\x6c\x6f\x61\x64\x3f\x76'
+    msg += b'\x65\x72\x3d\x56\x31\x2e\x30\x30\x2e\x31\x37\x26\x6e\x61\x6d\x65'
+    msg += b'\x3d\x47\x33\x2d\x57\x69\x46\x69\x2b\x2d\x56\x31\x2e\x30\x30\x2e'
+    msg += b'\x31\x37\x2d\x4f\x54\x41\x26\x65\x78\x74\x3d\x30\x60\x68\x74\x74'
+    msg += b'\x70\x3a\x2f\x2f\x77\x77\x77\x2e\x74\x61\x6c\x65\x6e\x74\x2d\x6d'
+    msg += b'\x6f\x6e\x69\x74\x6f\x72\x69\x6e\x67\x2e\x63\x6f\x6d\x3a\x39\x30'
+    msg += b'\x30\x32\x2f\x70\x72\x6f\x64\x2d\x61\x70\x69\x2f\x72\x6f\x6d\x2f'
+    msg += b'\x75\x70\x64\x61\x74\x65\x2f\x63\x61\x6c\x6c\x62\x61\x63\x6b\x3f'
+    msg += b'\x71\x69\x64\x3d\x31\x35\x30\x33\x36\x32\x26\x72\x69\x64\x3d\x32'
+    msg += b'\x32\x39\x26\x64\x69\x64\x3d\x31\x33\x34\x32\x32\x35\x20\x36\x35'
+    msg += b'\x66\x30\x64\x37\x34\x34\x62\x66\x33\x39\x61\x62\x38\x32\x34\x64'
+    msg += b'\x32\x38\x62\x38\x34\x64\x31\x39\x65\x64\x33\x31\x31\x63\x06\x34'
+    msg += b'\x36\x38\x36\x33\x33\x01\x31\x01\x30\x00'
    return msg


@@ -137,6 +158,7 @@ def test_send_contact_info1(ClientConnection, MsgContactInfo, MsgContactResp):
        pass
    assert data == MsgContactResp

+
 def test_send_contact_info2(ClientConnection, MsgContactInfo2, MsgContactInfo, MsgContactResp):
    s = ClientConnection  
    try:
@@ -154,7 +176,20 @@ def test_send_contact_info2(ClientConnection, MsgContactInfo2, MsgContactInfo, M
        pass
    assert data == MsgContactResp

-
+def test_send_contact_info3(ClientConnection, MsgContactInfo, MsgContactResp, MsgTimeStampReq):
+    s = ClientConnection
+    try:
+        s.sendall(MsgContactInfo)
+        data = s.recv(1024)
+    except TimeoutError:
+        pass
+    assert data == MsgContactResp
+    try:
+        s.sendall(MsgTimeStampReq)
+        data = s.recv(1024)
+    except TimeoutError:
+        pass
+ 

 def test_send_contact_resp(ClientConnection, MsgContactResp):
    s = ClientConnection  
@@ -197,3 +232,11 @@ def test_send_inv_data(ClientConnection, MsgTimeStampReq, MsgTimeStampResp, MsgI
        data = s.recv(1024)
    except TimeoutError:
        pass
+
+def test_ota_req(ClientConnection, MsgOtaUpdateReq):
+    s = ClientConnection  
+    try:
+        s.sendall(MsgOtaUpdateReq)
+        data = s.recv(1024)
+    except TimeoutError:
+        pass
Author	SHA1	Message	Date
Stefan Allius	bd7c4ae822	Version 0.5.4	2023-11-22 22:26:10 +01:00
Stefan Allius	e2873ffce7	Hardening (#30 ) * set build-argument for environment * hardening remove dangerous commands * add hardening scripts for base and final image	2023-11-22 21:57:42 +01:00
Stefan Allius	f10207b5ba	Merge branch 'main' of https://github.com/s-allius/tsun-gen3-proxy into main	2023-11-22 18:45:16 +01:00
Stefan Allius	aeb2a82df1	ignore bin directory	2023-11-22 18:45:03 +01:00
Stefan Allius	3b75c45344	OTA update (#29 ) * add pv module configuration * add OTA start message counter * add OTA start message counter * fix test_statistic_counter	2023-11-22 18:33:56 +01:00
Stefan Allius	9edfa40054	- add unit tests for ota messages	2023-11-21 22:31:46 +01:00
Stefan Allius	0a566a3df2	- add message handler for over the air updates	2023-11-21 22:29:59 +01:00
Stefan Allius	3e7eba9998	improve test coverage	2023-11-17 23:59:34 +01:00
Stefan Allius	00ddcc138f	add tests for int64 datatype in controller msg	2023-11-17 23:21:34 +01:00
Stefan Allius	0db2c3945d	cleanup msg_get_time handler	2023-11-17 23:20:03 +01:00
Stefan Allius	690c66a13a	hardening docker image remove the python packages setuptools, wheel and pip from final image to reduce the attack surface	2023-11-13 20:47:14 +01:00
Stefan Allius	a47ebb1511	fix messgae unit tests	2023-11-13 00:01:26 +01:00
Stefan Allius	4b7431ede9	Merge pull request #28 from s-allius/s-allius/issue26 Version 0.5.3	2023-11-12 20:25:00 +01:00
Stefan Allius	c3430f509e	Version 0.5.3	2023-11-12 15:23:43 +01:00
Stefan Allius	51b046c351	Version 0.5.3	2023-11-12 15:22:41 +01:00
Stefan Allius	32a669d0d1	Merge pull request #27 from s-allius/s-allius/issue26 S allius/issue26	2023-11-12 15:19:48 +01:00
Stefan Allius	4d9f00221c	fix the palnt offline problem in tsun cloud - use TSUN timestamp instead of local time, as TSUN also expects Central European Summer Time in winter	2023-11-12 15:15:30 +01:00
Stefan Allius	27c723b0c8	init contact_mail and contact_name	2023-11-12 01:06:24 +01:00
Stefan Allius	4bd59b91b3	send contact info every time a client connection is established	2023-11-11 23:49:06 +01:00
Stefan Allius	3a3c6142b8	ignore build.sh	2023-11-09 20:43:46 +01:00
Stefan Allius	5d36397f2f	remover apk from the final image	2023-11-09 20:17:19 +01:00
Stefan Allius	bb39567d05	Version 0.5.2	2023-11-09 20:05:56 +01:00
Stefan Allius	b6431f8448	improve client conn disconection - check for race cond. on closing and establishing client connections - improve connection trace	2023-11-09 20:03:09 +01:00
Stefan Allius	714dd92f35	allow multiple calls to Message.close()	2023-11-08 18:57:56 +01:00
Stefan Allius	02861f70af	- add int64 data type to info parser	2023-11-07 00:19:48 +01:00
Stefan Allius	942e17d7c3	Version 0.5.1	2023-11-05 00:57:10 +01:00
Stefan Allius	37f7052811	Merge pull request #23 from limes007/dns_desc add description for DNS settings	2023-11-05 00:14:30 +01:00
Stefan Allius	05e446dc74	Merge pull request #24 from limes007/main fix f-string	2023-11-05 00:09:51 +01:00
limes007	647ef157d4	fix f-string	2023-11-04 23:29:53 +01:00
limes007	9ae391b46d	add description for DNS settings	2023-11-04 23:28:20 +01:00