version 0.5.5

* - fix issue 33

  The TSUN Cloud now responds to contact_info and get_time messages with
  an empty display message and not with a response message as before.
  We tried to parse data from the empty message, which led to an
  exception

* Add test with empty conn_ind from inverter

CHANGELOG.md

@@ -7,6 +7,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.5.5] - 2023-12-31
+
+- Fixed [#33](https://github.com/s-allius/tsun-gen3-proxy/issues/33)
+- Fixed detection of the connected inputs/MPPTs
+- Preparation for overwriting received data
+- home assistant improvements:
+  - Add unit 'W' to the `Rated Power` value for home assistant
+  - `Collect_Interval`, `Connect_Count` and `Data_Up_Interval` as diagnostic value and not as graph
+  - Add data acquisition interval
+  - Add number of connections
+  - Add communication type
+  - Add 'Internal SW Exception' counter
+
 ## [0.5.4] - 2023-11-22
 
 - hardening remove dangerous commands from busybox
@@ -16,19 +29,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - add unit test for int64 data type
 - cleanup msg_get_time_handler
 - remove python packages setuptools, wheel, pip from final image to reduce the attack surface
- 
+
 ## [0.5.3] - 2023-11-12
 
 - remove apk packet manager from the final image
 - send contact info every time a client connection is established
 - use TSUN timestamp instead of local time, as TSUN also expects Central European Summer Time in winter
-    
+
 ## [0.5.2] - 2023-11-09
 
 - add int64 data type to info parser
 - allow multiple calls to Message.close()
 - check for race cond. on closing and establishing client connections
- 
+
 ## [0.5.1] - 2023-11-05
 
 - fixes f-string by limes007
@@ -54,7 +67,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - count definition errors in our internal tables
 - increase test coverage of the Infos class to 100%
 - avoid resetting the daily generation counters even if the inverter sends zero values at sunset
-   
+
 ## [0.4.1] - 2023-10-20
 
 - fix issue [#18](https://github.com/s-allius/tsun-gen3-proxy/issues/18)
@@ -80,13 +93,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 - optimize and reduce logging
 - switch to pathon 3.12
-- classify some values for diagnostics 
+- classify some values for diagnostics
   
 ## [0.2.0] - 2023-10-07
 
 This version halves the size of the Docker image and reduces the attack surface for security vulnerabilities, by omitting unneeded code. The feature set is exactly the same as the previous release version 0.1.0.
 
-### Changes
+### Changes in 0.2.0
 
 - move from slim-bookworm to an alpine base image
 - install python requirements with pip wheel
@@ -119,31 +132,31 @@ This version halves the size of the Docker image and reduces the attack surface
   
 ❗Due to the change from one device to multiple devices in the Home Assistant, the previous MQTT device should be deleted in the Home Assistant after the update to pre-release '0.0.4'. Afterwards, the proxy must be restarted again to ensure that the sub-devices are created completely.
 
-### Added
+### Added in 0.0.4
 
 - Register multiple devices at home-assistant instead of one for all measurements.
   Now we register: a Controller, the inverter and up to 4 input devices to home-assistant.
   
 ## [0.0.3] - 2023-09-28
 
-### Added
+### Added in 0.0.3
 
 - Fixes Running Proxy with host UID and GUID #2
 
 ## [0.0.2] - 2023-09-27
 
-### Added
+### Added in 0.0.2
 
 - Dockerfile opencontainer labels
 - Send voltage and current of inputs to mqtt
 
 ## [0.0.1] - 2023-09-25
 
-### Added
+### Added in 0.0.1
 
 - Logger for inverter packets
 - SIGTERM handler for fast docker restarts
-- Proxy as non-root docker application 
+- Proxy as non-root docker application
 - Unit- and system tests
 - Home asssistant auto configuration
 - Self-sufficient island operation without internet

README.md

@@ -40,7 +40,8 @@ If you use a Pi-hole, you can also store the host entry in the Pi-hole.
 
 ## Features
 
-- supports TSOL MS300, MS350, MS400, MS600, MS700 and MS800 inverters from TSUN
+- supports TSUN G3 inverters: TSOL MS-300, MS-350, MS-400, MS-600, MS-700 and MS-800
+- support for TSUN G3 Plus inverters is in preperation (e.g. MS-2000)
 - `MQTT` support
 - `Home-Assistant` auto-discovery support
 - Self-sufficient island operation without internet
@@ -120,6 +121,7 @@ inverters.allow_all = false   # True: allow inverters, even if we have no invert
 # inverter mapping, maps a `serial_no* to a `node_id` and defines an optional `suggested_area` for `home-assistant`
 #
 # for each inverter add a block starting with [inverters."<16-digit serial numbeer>"]
+
 [inverters."R17xxxxxxxxxxxx1"]
 node_id = 'inv1'              # Optional, MQTT replacement for inverters serial number  
 suggested_area = 'roof'       # Optional, suggested installation area for home-assistant
@@ -148,6 +150,33 @@ The proxy itself must use a different DNS server to connect to the TSUN Cloud. I
 
 As described above, set a DNS sever in the Docker command or Docker compose file.
 
+### Over The Air (OTA) firmware update
+Even if the proxy is connected between the inverter and the TSUN Cloud, an OTA update is supported. To do this, the inverter must be able to reach the website http://www.talent-monitoring.com:9002/ in order to download images from there.
+
+It must be ensured that this address is not mapped to the proxy!
+
+## Compatibility
+In the following table you will find an overview of which inverter model has been tested for compatibility with which firmware version.
+A combination with a red question mark should work, but I have not checked it in detail.
+
+Micro Inverter Model | Fw. 1.00.06 | Fw. 1.00.17 | Fw. 1.00.20| Fw. 1.1.00.0B
+:---|:---:|:---:|:---:|:---:|
+G3 micro inverters (single MPPT):<br>MS-300, MS-350, MS-400| ❓ | ❓ | ❓ |➖
+G3 micro inverters (dual MPPT):<br>MS-600, MS-700, MS-800| ✔️ | ✔️ | ✔️ |➖
+G3 PLUS micro inverters:<br>MS-1600, MS-1800, MS-2000| ➖  |➖ | ➖ | 🚧 
+balcony micro inverters:<br>MS-400-D, MS-800-D, MS-2000-D| ❓ | ❓ | ❓| ❓ 
+
+```
+Legend
+➖: Firmware not available for this devices
+✔️: proxy support testet
+❓: proxy support possible but not testet
+🚧: Proxy support in preparation
+```
+❗The new inverters of the G3Plus generation (e.g. MS-2000) use a completely different protocol for data transmission to the TSUN server. I already have such an inverter in operation and am working on the integration for the proxy version 0.6. The serial numbers of these inverters start with `Y17E` instead of `R17E`
+
+If you have one of these combinations with a red question mark, it would be very nice if you could send me a proxy trace so that I can carry out the detailed checks and adjust the device and system tests. [Ask here how to send a trace](https://github.com/s-allius/tsun-gen3-proxy/discussions/categories/traces-for-compatibility-check)
+
 ## License
 
 This project is licensed under the [BSD 3-clause License](https://opensource.org/licenses/BSD-3-Clause).
@@ -157,7 +186,6 @@ Note the aiomqtt library used is based on the paho-mqtt library, which has a dua
 - One use of "COPYRIGHT OWNER" (EDL) instead of "COPYRIGHT HOLDER" (BSD)
 - One use of "Eclipse Foundation, Inc." (EDL) instead of "copyright holder" (BSD)
 
-
 ## Versioning
 
 This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). Breaking changes will only occur in major `X.0.0` releases.

app/src/async_stream.py

@@ -44,6 +44,7 @@ class AsyncStream(Message):
                 self.close()
                 return self
             except Exception:
+                self.inc_counter('SW_Exception')
                 logger.error(
                     f"Exception for {self.addr}:\n"
                     f"{traceback.format_exc()}")

app/src/infos.py

@@ -33,11 +33,13 @@ class Infos:
         'controller': {'via': 'proxy',      'name': 'Controller',     'mdl': 0x00092f90, 'mf': 0x000927c0, 'sw': 0x00092ba8},  # noqa: E501
         'inverter':   {'via': 'controller', 'name': 'Micro Inverter', 'mdl': 0x00000032, 'mf': 0x00000014, 'sw': 0x0000001e},  # noqa: E501
         'input_pv1':  {'via': 'inverter',   'name': 'Module PV1'},
-        'input_pv2':  {'via': 'inverter',   'name': 'Module PV2', 'dep': {'reg': 0x00095b50, 'gte': 2}},  # noqa: E501
+        'input_pv2':  {'via': 'inverter',   'name': 'Module PV2', 'dep': {'reg': 0x00013880, 'gte': 2}},  # noqa: E501
-        'input_pv3':  {'via': 'inverter',   'name': 'Module PV3', 'dep': {'reg': 0x00095b50, 'gte': 3}},  # noqa: E501
+        'input_pv3':  {'via': 'inverter',   'name': 'Module PV3', 'dep': {'reg': 0x00013880, 'gte': 3}},  # noqa: E501
-        'input_pv4':  {'via': 'inverter',   'name': 'Module PV4', 'dep': {'reg': 0x00095b50, 'gte': 4}},  # noqa: E501
+        'input_pv4':  {'via': 'inverter',   'name': 'Module PV4', 'dep': {'reg': 0x00013880, 'gte': 4}},  # noqa: E501
     }
 
+    __comm_type_val_tpl = "{%set com_types = ['n/a','Wi-Fi', 'G4', 'G5', 'GPRS'] %}{{com_types[value_json['Communication_Type']|int(0)]|default(value_json['Communication_Type'])}}"    # noqa: E501
+
     __info_defs = {
         # collector values used for device registration:
         0x00092ba8:  {'name': ['collector', 'Collector_Fw_Version'],       'level': logging.INFO,  'unit': ''},  # noqa: E501
@@ -45,7 +47,6 @@ class Infos:
         0x00092f90:  {'name': ['collector', 'Chip_Model'],                 'level': logging.DEBUG, 'unit': ''},  # noqa: E501
         0x00095a88:  {'name': ['collector', 'Trace_URL'],                  'level': logging.DEBUG, 'unit': ''},  # noqa: E501
         0x00095aec:  {'name': ['collector', 'Logger_URL'],                 'level': logging.DEBUG, 'unit': ''},  # noqa: E501
-        0x00095b50:  {'name': ['collector', 'No_Inputs'],                  'level': logging.DEBUG, 'unit': ''},  # noqa: E501
 
         # inverter values used for device registration:
         0x0000000a:  {'name': ['inverter', 'Product_Name'],                'level': logging.DEBUG, 'unit': ''},  # noqa: E501
@@ -53,6 +54,7 @@ class Infos:
         0x0000001e:  {'name': ['inverter', 'Version'],                     'level': logging.INFO,  'unit': ''},  # noqa: E501
         0x00000028:  {'name': ['inverter', 'Serial_Number'],               'level': logging.DEBUG, 'unit': ''},  # noqa: E501
         0x00000032:  {'name': ['inverter', 'Equipment_Model'],             'level': logging.DEBUG, 'unit': ''},  # noqa: E501
+        0x00013880:  {'name': ['inverter', 'No_Inputs'],                   'level': logging.DEBUG, 'unit': ''},  # noqa: E501
 
         # proxy:
         0xffffff00:  {'name': ['proxy', 'Inverter_Cnt'],       'singleton': True,   'ha': {'dev': 'proxy', 'comp': 'sensor', 'dev_cla': None, 'stat_cla': None, 'id': 'inv_count_',     'fmt': '| int', 'name': 'Active Inverter Connections',    'icon': 'mdi:counter'}},  # noqa: E501
@@ -62,6 +64,7 @@ class Infos:
         0xffffff04:  {'name': ['proxy', 'Internal_Error'],     'singleton': True,   'ha': {'dev': 'proxy', 'comp': 'sensor', 'dev_cla': None, 'stat_cla': None, 'id': 'intern_err_',    'fmt': '| int', 'name': 'Internal Error',       'icon': 'mdi:counter', 'ent_cat': 'diagnostic', 'en': False}},  # noqa: E501
         0xffffff05:  {'name': ['proxy', 'Unknown_Ctrl'],       'singleton': True,   'ha': {'dev': 'proxy', 'comp': 'sensor', 'dev_cla': None, 'stat_cla': None, 'id': 'unknown_ctrl_',  'fmt': '| int', 'name': 'Unknown Control Type', 'icon': 'mdi:counter', 'ent_cat': 'diagnostic'}},  # noqa: E501
         0xffffff06:  {'name': ['proxy', 'OTA_Start_Msg'],      'singleton': True,   'ha': {'dev': 'proxy', 'comp': 'sensor', 'dev_cla': None, 'stat_cla': None, 'id': 'ota_start_cmd_', 'fmt': '| int', 'name': 'OTA Start Cmd',        'icon': 'mdi:counter', 'ent_cat': 'diagnostic'}},  # noqa: E501
+        0xffffff07:  {'name': ['proxy', 'SW_Exception'],       'singleton': True,   'ha': {'dev': 'proxy', 'comp': 'sensor', 'dev_cla': None, 'stat_cla': None, 'id': 'sw_exception_',  'fmt': '| int', 'name': 'Internal SW Exception', 'icon': 'mdi:counter', 'ent_cat': 'diagnostic'}},  # noqa: E501
         # 0xffffff03:  {'name':['proxy', 'Voltage'],                        'level': logging.DEBUG, 'unit': 'V',    'ha':{'dev':'proxy', 'dev_cla': 'voltage',     'stat_cla': 'measurement', 'id':'proxy_volt_',  'fmt':'| float','name': 'Grid Voltage'}},  # noqa: E501
 
         # events
@@ -87,7 +90,7 @@ class Infos:
         0x0000044c:  {'name': ['grid', 'Current'],                         'level': logging.DEBUG, 'unit': 'A',    'ha': {'dev': 'inverter', 'dev_cla': 'current',     'stat_cla': 'measurement', 'id': 'out_cur_',   'fmt': '| float', 'name': 'Grid Current', 'ent_cat': 'diagnostic'}},  # noqa: E501
         0x000004b0:  {'name': ['grid', 'Frequency'],                       'level': logging.DEBUG, 'unit': 'Hz',   'ha': {'dev': 'inverter', 'dev_cla': 'frequency',   'stat_cla': 'measurement', 'id': 'out_freq_',  'fmt': '| float', 'name': 'Grid Frequency', 'ent_cat': 'diagnostic'}},  # noqa: E501
         0x00000640:  {'name': ['grid', 'Output_Power'],                    'level': logging.INFO,  'unit': 'W',    'ha': {'dev': 'inverter', 'dev_cla': 'power',       'stat_cla': 'measurement', 'id': 'out_power_', 'fmt': '| float', 'name': 'Power'}},  # noqa: E501
-        0x000005dc:  {'name': ['env',  'Rated_Power'],                     'level': logging.DEBUG, 'unit': 'W',    'ha': {'dev': 'inverter', 'dev_cla': None,          'stat_cla': None,          'id': 'rated_power_', 'fmt': '| int', 'name': 'Rated Power', 'ent_cat': 'diagnostic'}},  # noqa: E501
+        0x000005dc:  {'name': ['env',  'Rated_Power'],                     'level': logging.DEBUG, 'unit': 'W',    'ha': {'dev': 'inverter', 'dev_cla': None,          'stat_cla': None,          'id': 'rated_power_', 'fmt': '| string + " W"', 'name': 'Rated Power', 'icon': 'mdi:lightning-bolt', 'ent_cat': 'diagnostic'}},  # noqa: E501
         0x00000514:  {'name': ['env',  'Inverter_Temp'],                   'level': logging.DEBUG, 'unit': '°C',   'ha': {'dev': 'inverter', 'dev_cla': 'temperature', 'stat_cla': 'measurement', 'id': 'temp_',       'fmt': '| int', 'name': 'Temperature'}},  # noqa: E501
 
         # input measures:
@@ -116,9 +119,13 @@ class Infos:
         0x00000bb8:  {'name': ['total', 'Total_Generation'],               'level': logging.INFO,  'unit': 'kWh',  'ha': {'dev': 'inverter', 'dev_cla': 'energy', 'stat_cla': 'total',            'id': 'total_gen_', 'fmt': '| float', 'name': 'Total Generation', 'icon': 'mdi:solar-power', 'must_incr': True}},  # noqa: E501
 
         # controller:
-        0x000c3500:  {'name': ['controller', 'Signal_Strength'],           'level': logging.DEBUG, 'unit': '%',    'ha': {'dev': 'controller', 'dev_cla': None,       'stat_cla': 'measurement', 'id': 'signal_',         'fmt': '| int', 'name': 'Signal Strength', 'icon': 'mdi:wifi'}},  # noqa: E501
+        0x000c3500:  {'name': ['controller', 'Signal_Strength'],           'level': logging.DEBUG, 'unit': '%',    'ha': {'dev': 'controller', 'dev_cla': None,       'stat_cla': 'measurement', 'id': 'signal_',              'fmt': '| int',           'name': 'Signal Strength', 'icon': 'mdi:wifi'}},  # noqa: E501
-        0x000c96a8:  {'name': ['controller', 'Power_On_Time'],             'level': logging.DEBUG, 'unit': 's',    'ha': {'dev': 'controller', 'dev_cla': 'duration', 'stat_cla': 'measurement', 'id': 'power_on_time_',                 'name': 'Power on Time',   'val_tpl': "{{ (value_json['Power_On_Time'] | float)}}", 'nat_prc': '3', 'ent_cat': 'diagnostic'}},  # noqa: E501
+        0x000c96a8:  {'name': ['controller', 'Power_On_Time'],             'level': logging.DEBUG, 'unit': 's',    'ha': {'dev': 'controller', 'dev_cla': 'duration', 'stat_cla': 'measurement', 'id': 'power_on_time_',       'fmt': '| float',         'name': 'Power on Time', 'nat_prc': '3', 'ent_cat': 'diagnostic'}},  # noqa: E501
-        0x000cf850:  {'name': ['controller', 'Data_Up_Interval'],          'level': logging.DEBUG, 'unit': 's',    'ha': {'dev': 'controller', 'dev_cla': None,       'stat_cla': 'measurement', 'id': 'data_up_intval_', 'fmt': '| int', 'name': 'Data Up Interval', 'icon': 'mdi:update', 'ent_cat': 'diagnostic'}},  # noqa: E501
+        0x000d0020:  {'name': ['controller', 'Collect_Interval'],          'level': logging.DEBUG, 'unit': 's',    'ha': {'dev': 'controller', 'dev_cla': None,       'stat_cla': None,          'id': 'data_collect_intval_', 'fmt': '| string + " s"', 'name': 'Data Collect Interval', 'icon': 'mdi:update', 'ent_cat': 'diagnostic'}},  # noqa: E501
+        0x000cfc38:  {'name': ['controller', 'Connect_Count'],             'level': logging.DEBUG, 'unit': '',     'ha': {'dev': 'controller', 'dev_cla': None,       'stat_cla': None,          'id': 'connect_count_',       'fmt': '| int',           'name': 'Connect Count',    'icon': 'mdi:counter', 'comp': 'sensor', 'ent_cat': 'diagnostic'}},  # noqa: E501
+        0x000c7f38:  {'name': ['controller', 'Communication_Type'],        'level': logging.DEBUG, 'unit': '',     'ha': {'dev': 'controller', 'dev_cla': None,       'stat_cla': None,          'id': 'comm_type_',           'name': 'Communication Type', 'val_tpl': __comm_type_val_tpl, 'comp': 'sensor', 'icon': 'mdi:wifi'}},  # noqa: E501
+        # 0x000c7f38:  {'name': ['controller', 'Communication_Type'],        'level': logging.DEBUG, 'unit': 's',    'new_value': 5},  # noqa: E501
+        0x000cf850:  {'name': ['controller', 'Data_Up_Interval'],          'level': logging.DEBUG, 'unit': 's',    'ha': {'dev': 'controller', 'dev_cla': None,       'stat_cla': None,          'id': 'data_up_intval_', 'fmt': '| string + " s"', 'name': 'Data Up Interval', 'icon': 'mdi:update', 'ent_cat': 'diagnostic'}},  # noqa: E501
 
     }
 
@@ -305,24 +312,27 @@ class Infos:
             must_incr = d['ha']['must_incr']
         else:
             must_incr = False
+        new_val = None
+        # if 'new_value' in d:
+        #     new_val = d['new_value']
 
-        return d['name'], d['level'], d['unit'], must_incr
+        return d['name'], d['level'], d['unit'], must_incr, new_val
 
-    def parse(self, buf) -> None:
+    def parse(self, buf, ind=0) -> None:
         '''parse a data sequence received from the inverter and
         stores the values in Infos.db
 
         buf: buffer of the sequence to parse'''
-        result = struct.unpack_from('!l', buf, 0)
+        result = struct.unpack_from('!l', buf, ind)
         elms = result[0]
         i = 0
-        ind = 4
+        ind += 4
         while i < elms:
             result = struct.unpack_from('!lB', buf, ind)
             info_id = result[0]
             data_type = result[1]
             ind += 5
-            keys, level, unit, must_incr = self.__key_obj(info_id)
+            keys, level, unit, must_incr, new_val = self.__key_obj(info_id)
 
             if data_type == 0x54:   # 'T' -> Pascal-String
                 str_len = buf[ind]
@@ -332,19 +342,29 @@ class Infos:
                 ind += str_len+1
 
             elif data_type == 0x49:  # 'I' -> int32
+                # if new_val:
+                #    struct.pack_into('!l', buf, ind, new_val)
                 result = struct.unpack_from('!l', buf, ind)[0]
                 ind += 4
 
             elif data_type == 0x53:  # 'S' -> short
+                # if new_val:
+                #     struct.pack_into('!h', buf, ind, new_val)
                 result = struct.unpack_from('!h', buf, ind)[0]
                 ind += 2
 
             elif data_type == 0x46:  # 'F' -> float32
+                # if new_val:
+                #     struct.pack_into('!f', buf, ind, new_val)
                 result = round(struct.unpack_from('!f', buf, ind)[0], 2)
                 ind += 4
+
             elif data_type == 0x4c:  # 'L' -> int64
+                # if new_val:
+                #     struct.pack_into('!q', buf, ind, new_val)
                 result = struct.unpack_from('!q', buf, ind)[0]
                 ind += 8
+
             else:
                 self.inc_counter('Invalid_Data_Type')
                 logging.error(f"Infos.parse: data_type: {data_type}"

app/src/inverter.py

@@ -28,7 +28,7 @@ class Inverter(AsyncStream):
     class methods:
         class_init():  initialize the common resources of the proxy (MQTT
                        broker, Proxy DB, etc). Must be called before the
-                       first Ib´verter instance can be created
+                       first inverter instance can be created
         class_close(): release the common resources of the proxy. Should not
                        be called before any instances of the class are
                        destroyed
@@ -156,6 +156,7 @@ class Inverter(AsyncStream):
         except ConnectionRefusedError as error:
             logging.info(f'{error}')
         except Exception:
+            self.inc_counter('SW_Exception')
             logging.error(
                 f"Inverter: Exception for {addr}:\n"
                 f"{traceback.format_exc()}")
@@ -181,6 +182,7 @@ class Inverter(AsyncStream):
         except MqttCodeError as error:
             logging.error(f'Mqtt except: {error}')
         except Exception:
+            self.inc_counter('SW_Exception')
             logging.error(
                 f"Inverter: Exception:\n"
                 f"{traceback.format_exc()}")

app/src/messages.py

@@ -83,10 +83,11 @@ class Message(metaclass=IterRegistry):
         self.unique_id = 0
         self.node_id = ''
         self.sug_area = ''
+        self.await_conn_resp_cnt = 0
         self.id_str = id_str
         self.contact_name = b''
         self.contact_mail = b''
-        self._recv_buffer = b''
+        self._recv_buffer = bytearray(0)
         self._send_buffer = bytearray(0)
         self._forward_buffer = bytearray(0)
         self.db = Infos()
@@ -182,6 +183,7 @@ class Message(metaclass=IterRegistry):
     def _init_new_client_conn(self, contact_name, contact_mail) -> None:
         logger.info(f'name: {contact_name} mail: {contact_mail}')
         self.msg_id = 0
+        self.await_conn_resp_cnt += 1
         self.__build_header(0x91)
         self._send_buffer += struct.pack(f'!{len(contact_name)+1}p'
                                          f'{len(contact_mail)+1}p',
@@ -282,23 +284,27 @@ class Message(metaclass=IterRegistry):
     '''
     def msg_contact_info(self):
         if self.ctrl.is_ind():
-            self.__build_header(0x99)
+            if self.server_side and self.__process_contact_info():
-            self._send_buffer += b'\x01'
+                self.__build_header(0x91)
-            self.__finish_send_msg()
+                self._send_buffer += b'\x01'
-            self.__process_contact_info()
+                self.__finish_send_msg()
             # don't forward this contact info here, we will build one
             # when the remote connection is established
+            elif self.await_conn_resp_cnt > 0:
+                self.await_conn_resp_cnt -= 1
+            else:
+                self.forward(self._recv_buffer, self.header_len+self.data_len)
             return
-        elif self.ctrl.is_resp():
-            return  # ignore received response from tsun
         else:
+            logger.warning('Unknown Ctrl')
             self.inc_counter('Unknown_Ctrl')
             self.forward(self._recv_buffer, self.header_len+self.data_len)
 
-    def __process_contact_info(self):
+    def __process_contact_info(self) -> bool:
         result = struct.unpack_from('!B', self._recv_buffer, self.header_len)
         name_len = result[0]
-
+        if self.data_len < name_len+2:
+            return False
         result = struct.unpack_from(f'!{name_len+1}pB', self._recv_buffer,
                                     self.header_len)
         self.contact_name = result[0]
@@ -309,33 +315,34 @@ class Message(metaclass=IterRegistry):
                                     self.header_len+name_len+1)
         self.contact_mail = result[0]
         logger.info(f'mail: {self.contact_mail}')
+        return True
 
     def msg_get_time(self):
         tsun = Config.get('tsun')
         if tsun['enabled']:
-            if self.ctrl.is_resp():
+            if self.ctrl.is_ind():
-                ts = self._timestamp()
+                if self.data_len >= 8:
-                result = struct.unpack_from('!q', self._recv_buffer,
+                    ts = self._timestamp()
-                                            self.header_len)
+                    result = struct.unpack_from('!q', self._recv_buffer,
-                logger.debug(f'tsun-time: {result[0]:08x}'
+                                                self.header_len)
-                             f'  proxy-time: {ts:08x}')
+                    logger.debug(f'tsun-time: {result[0]:08x}'
-            elif not self.ctrl.is_ind():
+                                 f'  proxy-time: {ts:08x}')
+            else:
+                logger.warning('Unknown Ctrl')
                 self.inc_counter('Unknown_Ctrl')
             self.forward(self._recv_buffer, self.header_len+self.data_len)
         else:
             if self.ctrl.is_ind():
-                ts = self._timestamp()
+                if self.data_len == 0:
-                logger.debug(f'time: {ts:08x}')
+                    ts = self._timestamp()
+                    logger.debug(f'time: {ts:08x}')
 
-                self.__build_header(0x99)
+                    self.__build_header(0x91)
-                self._send_buffer += struct.pack('!q', ts)
+                    self._send_buffer += struct.pack('!q', ts)
-                self.__finish_send_msg()
+                    self.__finish_send_msg()
 
-            elif self.ctrl.is_resp():
-                result = struct.unpack_from('!q', self._recv_buffer,
-                                            self.header_len)
-                logger.debug(f'tsun-time: {result[0]:08x}')
             else:
+                logger.warning('Unknown Ctrl')
                 self.inc_counter('Unknown_Ctrl')
 
     def parse_msg_header(self):
@@ -366,6 +373,7 @@ class Message(metaclass=IterRegistry):
         elif self.ctrl.is_resp():
             return  # ignore received response
         else:
+            logger.warning('Unknown Ctrl')
             self.inc_counter('Unknown_Ctrl')
 
         self.forward(self._recv_buffer, self.header_len+self.data_len)
@@ -380,6 +388,7 @@ class Message(metaclass=IterRegistry):
         elif self.ctrl.is_resp():
             return  # ignore received response
         else:
+            logger.warning('Unknown Ctrl')
             self.inc_counter('Unknown_Ctrl')
 
         self.forward(self._recv_buffer, self.header_len+self.data_len)
@@ -387,8 +396,8 @@ class Message(metaclass=IterRegistry):
     def __process_data(self):
         msg_hdr_len = self.parse_msg_header()
 
-        for key, update in self.db.parse(self._recv_buffer[self.header_len
+        for key, update in self.db.parse(self._recv_buffer, self.header_len
-                                                           + msg_hdr_len:]):
+                                         + msg_hdr_len):
             if update:
                 self.new_data[key] = True
 
@@ -398,6 +407,7 @@ class Message(metaclass=IterRegistry):
         elif self.ctrl.is_ind():
             pass
         else:
+            logger.warning('Unknown Ctrl')
             self.inc_counter('Unknown_Ctrl')
         self.forward(self._recv_buffer, self.header_len+self.data_len)
 

app/tests/test_infos.py

@@ -181,7 +181,7 @@ def test_parse_control(ContrDataSeq):
         pass
 
     assert json.dumps(i.db) == json.dumps(
-{"collector": {"Collector_Fw_Version": "RSW_400_V1.00.06", "Chip_Type": "Raymon", "Chip_Model": "RSW-1-10001", "Trace_URL": "t.raymoniot.com", "Logger_URL": "logger.talent-monitoring.com", "No_Inputs": 2}, "controller": {"Signal_Strength": 100, "Power_On_Time": 29, "Data_Up_Interval": 300}})        
+{"collector": {"Collector_Fw_Version": "RSW_400_V1.00.06", "Chip_Type": "Raymon", "Chip_Model": "RSW-1-10001", "Trace_URL": "t.raymoniot.com", "Logger_URL": "logger.talent-monitoring.com"}, "controller": {"Collect_Interval": 1, "Signal_Strength": 100, "Power_On_Time": 29, "Communication_Type": 1, "Connect_Count": 1, "Data_Up_Interval": 300}})        
 
 def test_parse_control2(Contr2DataSeq):
     i = Infos()
@@ -189,7 +189,7 @@ def test_parse_control2(Contr2DataSeq):
         pass
 
     assert json.dumps(i.db) == json.dumps(
-{"collector": {"Collector_Fw_Version": "RSW_400_V1.00.20", "Chip_Type": "Raymon", "Chip_Model": "RSW-1-10001", "Trace_URL": "t.raymoniot.com", "Logger_URL": "logger.talent-monitoring.com", "No_Inputs": 2}, "controller": {"Signal_Strength": 16, "Power_On_Time": 334, "Data_Up_Interval": 300}})
+{"collector": {"Collector_Fw_Version": "RSW_400_V1.00.20", "Chip_Type": "Raymon", "Chip_Model": "RSW-1-10001", "Trace_URL": "t.raymoniot.com", "Logger_URL": "logger.talent-monitoring.com"}, "controller": {"Collect_Interval": 1, "Signal_Strength": 16, "Power_On_Time": 334, "Communication_Type": 1, "Connect_Count": 1, "Data_Up_Interval": 300}})
 
 def test_parse_inverter(InvDataSeq):
     i = Infos()
@@ -209,7 +209,7 @@ def test_parse_cont_and_invert(ContrDataSeq, InvDataSeq):
 
     assert json.dumps(i.db) == json.dumps(
     {
-"collector": {"Collector_Fw_Version": "RSW_400_V1.00.06", "Chip_Type": "Raymon", "Chip_Model": "RSW-1-10001", "Trace_URL": "t.raymoniot.com", "Logger_URL": "logger.talent-monitoring.com", "No_Inputs": 2}, "controller": {"Signal_Strength": 100, "Power_On_Time": 29, "Data_Up_Interval": 300},
+"collector": {"Collector_Fw_Version": "RSW_400_V1.00.06", "Chip_Type": "Raymon", "Chip_Model": "RSW-1-10001", "Trace_URL": "t.raymoniot.com", "Logger_URL": "logger.talent-monitoring.com"}, "controller": {"Collect_Interval": 1, "Signal_Strength": 100, "Power_On_Time": 29, "Communication_Type": 1, "Connect_Count": 1, "Data_Up_Interval": 300},
 "inverter": {"Product_Name": "Microinv", "Manufacturer": "TSUN", "Version": "V5.0.11", "Serial_Number": "T170000000000001", "Equipment_Model": "TSOL-MS600"}})
 
 
@@ -269,13 +269,14 @@ def test_build_ha_conf1(ContrDataSeq):
 
     assert tests==5
 
-def test_build_ha_conf2(ContrDataSeq, InvDataSeq):
+def test_build_ha_conf2(ContrDataSeq, InvDataSeq, InvDataSeq2):
     i = Infos()
     for key, result in i.parse (ContrDataSeq):
         pass
-
     for key, result in i.parse (InvDataSeq):
         pass
+    for key, result in i.parse (InvDataSeq2):
+        pass
 
     tests = 0
     for d_json, comp, node_id, id in i.ha_confs(ha_prfx="tsun/", node_id="garagendach/", snr='123', singleton=False, sug_area = 'roof'):
@@ -408,13 +409,13 @@ def test_statistic_counter():
     assert val == None or val == 0
 
     i.static_init()                # initialize counter
-    assert json.dumps(i.stat) == json.dumps({"proxy": {"Inverter_Cnt": 0, "Unknown_SNR": 0, "Unknown_Msg": 0, "Invalid_Data_Type": 0, "Internal_Error": 0,"Unknown_Ctrl": 0, "OTA_Start_Msg": 0}})
+    assert json.dumps(i.stat) == json.dumps({"proxy": {"Inverter_Cnt": 0, "Unknown_SNR": 0, "Unknown_Msg": 0, "Invalid_Data_Type": 0, "Internal_Error": 0,"Unknown_Ctrl": 0, "OTA_Start_Msg": 0, "SW_Exception": 0}})
                                             
     val = i.dev_value(0xffffff00)  # valid and initiliazed addr
     assert val == 0
 
     i.inc_counter('Inverter_Cnt')
-    assert json.dumps(i.stat) == json.dumps({"proxy": {"Inverter_Cnt": 1, "Unknown_SNR": 0, "Unknown_Msg": 0, "Invalid_Data_Type": 0, "Internal_Error": 0,"Unknown_Ctrl": 0, "OTA_Start_Msg": 0}})
+    assert json.dumps(i.stat) == json.dumps({"proxy": {"Inverter_Cnt": 1, "Unknown_SNR": 0, "Unknown_Msg": 0, "Invalid_Data_Type": 0, "Internal_Error": 0,"Unknown_Ctrl": 0, "OTA_Start_Msg": 0, "SW_Exception": 0}})
     val = i.dev_value(0xffffff00)
     assert val == 1
 

app/tests/test_messages.py

@@ -67,11 +67,11 @@ def Msg2ContactInfo(): # two Contact Info messages
 
 @pytest.fixture
 def MsgContactResp(): # Contact Response message
-    return b'\x00\x00\x00\x14\x10R170000000000001\x99\x00\x01'
+    return b'\x00\x00\x00\x14\x10R170000000000001\x91\x00\x01'
 
 @pytest.fixture
 def MsgContactResp2(): # Contact Response message
-    return b'\x00\x00\x00\x14\x10R170000000000002\x99\x00\x01'
+    return b'\x00\x00\x00\x14\x10R170000000000002\x91\x00\x01'
 
 @pytest.fixture
 def MsgContactInvalid(): # Contact Response message
@@ -83,7 +83,7 @@ def MsgGetTime(): # Get Time Request message
 
 @pytest.fixture
 def MsgTimeResp(): # Get Time Resonse message
-    return b'\x00\x00\x00\x1b\x10R170000000000001\x99\x22\x00\x00\x01\x89\xc6\x63\x4d\x80'
+    return b'\x00\x00\x00\x1b\x10R170000000000001\x91\x22\x00\x00\x01\x89\xc6\x63\x4d\x80'
 
 @pytest.fixture
 def MsgTimeInvalid(): # Get Time Request message
@@ -316,13 +316,15 @@ def test_read_two_messages(ConfigTsunAllowAll, Msg2ContactInfo,MsgContactResp,Ms
 def test_msg_contact_resp(ConfigTsunInv1, MsgContactResp):
     ConfigTsunInv1
     m = MemoryStream(MsgContactResp, (0,), False)
+    m.await_conn_resp_cnt = 1
     m.db.stat['proxy']['Unknown_Ctrl'] = 0
     m.read()         # read complete msg, and dispatch msg
     assert not m.header_valid  # must be invalid, since msg was handled and buffer flushed
     assert m.msg_count == 1
+    assert m.await_conn_resp_cnt == 0
     assert m.id_str == b"R170000000000001" 
     assert m.unique_id == 'R170000000000001'
-    assert int(m.ctrl)==153
+    assert int(m.ctrl)==145
     assert m.msg_id==0
     assert m.header_len==23
     assert m.data_len==1
@@ -331,6 +333,46 @@ def test_msg_contact_resp(ConfigTsunInv1, MsgContactResp):
     assert m.db.stat['proxy']['Unknown_Ctrl'] == 0
     m.close()
 
+def test_msg_contact_resp_2(ConfigTsunInv1, MsgContactResp):
+    ConfigTsunInv1
+    m = MemoryStream(MsgContactResp, (0,), False)
+    m.await_conn_resp_cnt = 0
+    m.db.stat['proxy']['Unknown_Ctrl'] = 0
+    m.read()         # read complete msg, and dispatch msg
+    assert not m.header_valid  # must be invalid, since msg was handled and buffer flushed
+    assert m.msg_count == 1
+    assert m.await_conn_resp_cnt == 0
+    assert m.id_str == b"R170000000000001" 
+    assert m.unique_id == 'R170000000000001'
+    assert int(m.ctrl)==145
+    assert m.msg_id==0
+    assert m.header_len==23
+    assert m.data_len==1
+    assert m._forward_buffer==MsgContactResp
+    assert m._send_buffer==b''
+    assert m.db.stat['proxy']['Unknown_Ctrl'] == 0
+    m.close()
+
+def test_msg_contact_resp_3(ConfigTsunInv1, MsgContactResp):
+    ConfigTsunInv1
+    m = MemoryStream(MsgContactResp, (0,), True)
+    m.await_conn_resp_cnt = 0
+    m.db.stat['proxy']['Unknown_Ctrl'] = 0
+    m.read()         # read complete msg, and dispatch msg
+    assert not m.header_valid  # must be invalid, since msg was handled and buffer flushed
+    assert m.msg_count == 1
+    assert m.await_conn_resp_cnt == 0
+    assert m.id_str == b"R170000000000001" 
+    assert m.unique_id == 'R170000000000001'
+    assert int(m.ctrl)==145
+    assert m.msg_id==0
+    assert m.header_len==23
+    assert m.data_len==1
+    assert m._forward_buffer==MsgContactResp
+    assert m._send_buffer==b''
+    assert m.db.stat['proxy']['Unknown_Ctrl'] == 0
+    m.close()
+
 def test_msg_contact_invalid(ConfigTsunInv1, MsgContactInvalid):
     ConfigTsunInv1
     m = MemoryStream(MsgContactInvalid, (0,))
@@ -381,7 +423,7 @@ def test_msg_get_time_autark(ConfigNoTsunInv1, MsgGetTime):
     assert m.header_len==23
     assert m.data_len==0
     assert m._forward_buffer==b''
-    assert m._send_buffer==b'\x00\x00\x00\x1b\x10R170000000000001\x99"\x00\x00\x01\x8b\xdfs\xcc0'
+    assert m._send_buffer==b'\x00\x00\x00\x1b\x10R170000000000001\x91"\x00\x00\x01\x8b\xdfs\xcc0'
     assert m.db.stat['proxy']['Unknown_Ctrl'] == 0
     m.close()
 
@@ -394,7 +436,7 @@ def test_msg_time_resp(ConfigTsunInv1, MsgTimeResp):
     assert m.msg_count == 1
     assert m.id_str == b"R170000000000001" 
     assert m.unique_id == 'R170000000000001'
-    assert int(m.ctrl)==153
+    assert int(m.ctrl)==145
     assert m.msg_id==34
     assert m.header_len==23
     assert m.data_len==8
@@ -412,7 +454,7 @@ def test_msg_time_resp_autark(ConfigNoTsunInv1, MsgTimeResp):
     assert m.msg_count == 1
     assert m.id_str == b"R170000000000001" 
     assert m.unique_id == 'R170000000000001'
-    assert int(m.ctrl)==153
+    assert int(m.ctrl)==145
     assert m.msg_id==34
     assert m.header_len==23
     assert m.data_len==8

system_tests/test_tcp_socket.py

@@ -20,7 +20,7 @@ def MsgContactInfo(): # Contact Info message
 
 @pytest.fixture
 def MsgContactResp(): # Contact Response message
-    return b'\x00\x00\x00\x14\x10'+get_sn()+b'\x99\x00\x01'
+    return b'\x00\x00\x00\x14\x10'+get_sn()+b'\x91\x00\x01'
 
 @pytest.fixture
 def MsgContactInfo2(): # Contact Info message
@@ -28,7 +28,7 @@ def MsgContactInfo2(): # Contact Info message
 
 @pytest.fixture
 def MsgContactResp2(): # Contact Response message
-    return b'\x00\x00\x00\x14\x10'+get_invalid_sn()+b'\x99\x00\x01'
+    return b'\x00\x00\x00\x14\x10'+get_invalid_sn()+b'\x91\x00\x01'
 
 @pytest.fixture
 def MsgTimeStampReq(): # Get Time Request message
@@ -199,7 +199,7 @@ def test_send_contact_resp(ClientConnection, MsgContactResp):
     except TimeoutError:
         assert True
     else:
-        assert data ==''
+        assert data == b''
 
 def test_send_ctrl_data(ClientConnection, MsgTimeStampReq, MsgTimeStampResp, MsgContollerInd):
     s = ClientConnection