Commit Graph

67 Commits

Author SHA1 Message Date
Stefan Allius
62ea2a9e6f Refactoring async stream (#194)
* GEN3: Invalid Contact Info Msg
Fixes #191

* introduce ifc with FIFOs

* add object factory

* use AsyncIfc class with FIFO

* declare more methods as classmethods

* - refactoring

- remove _forward_buffer
- make async_write private

* remove _forward_buffer

* refactoring

* avoid mqtt handling for invalid serial numbers

* add two more callbacks

* FIX update_header_cb handling

* split AsyncStream in two classes

* split ConnectionG3(P) in server and client class

* update class diagramm

* refactor server creation

* remove duplicated imports

* reduce code duplication

* move StremPtr instances into Inverter class

* resolution of connection classes

- remove ConnectionG3Client
- remove ConnectionG3Server
- remove ConnectionG3PClient
- remove ConnectionG3PServer

* fix server connections

* fix client loop closing

* don't overwrite self.remote in constructor

* update class diagramm

* fixes

- fixes null pointer accesses
- initalize AsyncStreamClient with proper
  StreamPtr instance

* add close callback

* refactor close handling

* remove connection classes

* move more code into InverterBase class

* remove test_inverter_base.py

* add abstract inverter interface class

* initial commit

* fix sonar qube warnings

* rename class Inverter into Proxy

* fix typo

* move class InverterIfc into a separate file

* add more testcases

* use ProtocolIfc class

* add unit tests for AsyncStream class

* icrease test coverage

* reduce cognitive complexity

* increase test coverage

* increase tes coverage

* simplify heartbeat handler

* remove obsolete tx_get method

* add more unittests

* update changelog

* remove __del__ method for proper gc runs

* check releasing of ModbusConn instances

* call garbage collector to release unreachable objs

* decrease ref counter after the with block
2024-10-13 16:07:01 +02:00
Stefan Allius
be4c6ac77f S allius/issue182 (#183)
* GEN3: After inverter firmware update the 'Unknown Msg Type' increases continuously
Fixes #182

* add support for Controller serial no and MAC

* test hardening

* GEN3: add support for new messages of version 3 firmwares

* bump libraries to latest versions

- bump aiomqtt to version 2.3.0
- bump aiohttp to version 3.10.5

* improve test coverage

* reduce cognective complexity
2024-09-07 11:45:16 +02:00
Stefan Allius
ab5256659b reduce cognitive complexity 2024-09-03 18:32:44 +02:00
Stefan Allius
54de2aecfe Sonar qube 3 (#165)
* cleanup

* Add support for TSUN Titan inverter
Fixes #161


* fix SonarQube warnings

* fix error

* rename field "config"

* SonarQube reads flake8 output

* don't stop on flake8 errors

* flake8 scan only app/src for SonarQube

* update flake8 run

* ignore flake8 C901

* cleanup

* fix linter warnings

* ignore changed *.yml files

* read sensor list solarman data packets

* catch 'No route to' error and log only in debug mode

* fix unit tests

* add sensor_list configuration

* adapt unit tests

* fix SonarQube warnings
2024-08-16 21:07:08 +02:00
Stefan Allius
b364fb3f8e Dev 0.10 (#151)
* S allius/issue117 (#118)

* add shutdown flag

* add more register definitions

* add start commando for client side connections

* add first support for port 8899

* fix shutdown

* add client_mode configuration

* read client_mode config to setup inverter connections

* add client_mode connections over port 8899

* add preview build

* Update README.md

describe the new client-mode over port 8899 for GEN3PLUS

* MODBUS: the last digit of the inverter version is a hexadecimal number (#121)

* S allius/issue117 (#122)

* add shutdown flag

* add more register definitions

* add start commando for client side connections

* add first support for port 8899

* fix shutdown

* add client_mode configuration

* read client_mode config to setup inverter connections

* add client_mode connections over port 8899

* add preview build

* add documentation for client_mode

* catch os error and log thme with DEBUG level

* update changelog

* make the maximum output coefficient configurable (#124)

* S allius/issue120 (#126)

* add config option to disable the modbus polling

* read more modbus regs in polling mode

* extend connection timeouts if polling mode is disabled

* update changelog

* S allius/issue125 (#127)

* fix linter warning

* move sequence diagramm to wiki

* catch asyncio.CancelledError

* S allius/issue128 (#130)

* set Register.NO_INPUTS fix to 4 for GEN3PLUS

* don't set Register.NO_INPUTS per MODBUS

* fix unit tests

* register OUTPUT_COEFFICIENT at HA

* update changelog

* - Home Assistant: improve inverter status value texts

* - GEN3: add inverter status

* on closing send outstanding MQTT data to the broker

* force MQTT publish on every conn open and close

* reset inverter state on close

- workaround which reset the inverter status to
  offline when the inverter has a very low
  output power on connection close

* improve client modified
- reduce the polling cadence to 30s
- set controller statistics for HA

* client mode set controller IP for HA

* S allius/issue131 (#132)

* Make __publish_outstanding_mqtt public

* update proxy counter

- on client mode connection establishment or
  disconnecting update tje counection counter

* Update README.md (#133)

* reset inverter state on close

- workaround which reset the inverter status to
  offline when the inverter has a very low
  output power on connection close

* S allius/issue134 (#135)

* add polling invertval and method ha_remove()

* add client_mode arg to constructors

- add PollingInvervall

* hide some topics in client mode

- we hide topics in HA by sending an empty register
  MQTT topic during HA auto configuration

* add client_mode value

* update class diagram

* fix modbus close handler

- fix empty call and cleanup que
- add unit test

* don't sent an initial 1710 msg in client mode

* change HA icon for inverter status

* increase test coverage

* accelerate timer tests

* bump aiomqtt and schema to latest release (#137)

* MQTT timestamps and protocol improvements (#140)

* add TS_INPUT, TS_GRID and TS_TOTAL

* prepare MQTT timestamps

- add _set_mqtt_timestamp method
- fix hexdump printing

* push dev and debug images to docker.io

* add unix epoche timestamp for MQTT pakets

* set timezone for unit tests

* set name für setting timezone step

* trigger new action

* GEN3 and GEN3PLUS: handle multiple message

- read: iterate over the receive buffer
- forward: append messages to the forward buffer
- _update_header: iterate over the forward buffer

* GEN3: optimize timeout handling

- longer timeout in state init and reveived
- got to state pending only from state up

* update changelog

* cleanup

* print coloured logs

* Create sonarcloud.yml (#143)

* Update sonarcloud.yml

* Update sonarcloud.yml

* Update sonarcloud.yml

* Update sonarcloud.yml

* Update sonarcloud.yml

* build multi arch images with sboms (#146)

* don't send MODBUS request when state is not up (#147)

* adapt timings

* don't send MODBUS request when state is note up

* adapt unit test

* make test code more clean (#148)

* Make test code more clean (#149)

* cleanup

* Code coverage for SonarCloud (#150)


* cleanup code and unit tests

* add test coverage for SonarCloud

* configure SonarCloud

* update changelog

* Do no build on *.yml changes

* prepare release 0.10.0

* disable MODBUS_POLLING for GEN§PLUS in example config

* bump aiohttp to version 3.10.2

* code cleanup

* Fetch all history for all tags and branches
2024-08-09 23:16:47 +02:00
Stefan Allius
a42ba8a8c6 Dev 0.9 (#115)
* make timestamp handling stateless

* adapt tests for stateless timestamp handling

* initial version

* add more type annotations

* add more type annotations

* fix Generator annotation for ha_proxy_confs

* fix names of issue branches

* add more type annotations

* don't use depricated varn anymore

* don't mark all test as async

* fix imports

* fix solarman unit tests

- fake Mqtt class

* print image build time during proxy start

* update changelog

* fix pytest collect warning

* cleanup msg_get_time handler

* addapt unit test

* label debug images with debug

* dump droped packages

* fix warnings

* add systemtest with invalid start byte

* update changelog

* update changelog

* add exposed ports and healthcheck

* add wget for healthcheck

* add aiohttp

* use config validation for healthcheck

* add http server for healthcheck

* calculate msg prossesing time

* add healthy check methods

* fix typo

* log ConfigErr with DEBUG level

* Update async_stream.py

- check if processing time is < 5 sec

* add a close handler to release internal resources

* call modbus close hanlder on a close call

* add exception handling for forward handler

* update changelog

* isolate Modbus fix

* cleanup

* update changelog

* add heaithy handler

* log unrelease references

* add healtcheck

* complete exposed port list

* add wget for healtcheck

* add aiohttp

* use Enum class for State

* calc processing time for healthcheck

* add HTTP server for healthcheck

* cleanup

* Update CHANGELOG.md

* updat changelog

* add docstrings to state enum

* set new state State.received

* add healthy method

* log healthcheck infos with DEBUG level

* update changelog

* S allius/issue100 (#101)

* detect dead connections

- disconnect connection on Msg receive timeout
- improve connection trace (add connection id)

* update changelog

* fix merge conflict

* fix unittests

* S allius/issue108 (#109)

* add more data types

* adapt unittests

* improve test coverage

* fix linter warning

* update changelog

* S allius/issue102 (#110)

* hotfix: don't send two MODBUS commands together

* fix unit tests

* remove read loop

* optional sleep between msg read and sending rsp

* wait after read 0.5s before sending a response

* add pending state

* fix state definitions

* determine the connection timeout by the conn state

* avoid sending MODBUS cmds in the inverter's reporting phase

* update changelog

* S allius/issue111 (#112)

Synchronize regular MODBUS commands with the status of the inverter to prevent the inverter from crashing due to unexpected packets.

* inital checkin

* remove crontab entry for regular MODBUS cmds

* add timer for regular MODBUS polling

* fix Stop method call for already stopped timer

* optimize MB_START_TIMEOUT value

* cleanup

* update changelog

* fix buildx warnings

* fix timer cleanup

* fix Config.class_init()

- return error string or None
- release Schema structure after building thr config

* add quit flag to docker push

* fix timout calculation

* rename python to debugpy

* add asyncio log

* cleanup shutdown
- stop webserver on shutdown
- enable asyncio debug mode for debug versions

* update changelog

* update changelog

* fix exception in MODBUS timeout callback

* update changelog
2024-07-01 23:41:56 +02:00
Stefan Allius
87d59d046f add AT_COMMAND_BLOCKED counter 2024-05-30 19:32:14 +02:00
Stefan Allius
55fc834a1e reduce default loggings 2024-05-22 22:52:02 +02:00
Stefan Allius
e432441134 don't log Events as Infos 2024-05-21 18:56:52 +02:00
Stefan Allius
3ac48dad1f cleanup 2024-05-20 18:33:01 +02:00
Stefan Allius
3fd528bdbe improve logging 2024-05-07 21:20:12 +02:00
Stefan Allius
5d0c95d6e6 fix typo 2024-05-01 11:57:02 +02:00
Stefan Allius
ccc7e7959e change unit of the collect interval to minutes 2024-04-28 18:31:33 +02:00
Stefan Allius
95ebb92f05 cleanup
- chance log level from INFO to DEBUG
- remove experimental value Register.VALUE_1
- format Register.POWER_ON_TIME as integer
2024-04-22 20:20:39 +02:00
Stefan Allius
6f9d2d4fac GEN3PLUS: Add inverter status 2024-04-19 19:07:59 +02:00
Stefan Allius
32ab49b566 make depency check in reg_clr_at_midnight optional 2024-04-14 12:22:25 +02:00
Stefan Allius
97da24c839 add missing tests 2024-04-07 22:44:53 +02:00
Stefan Allius
93b89062f5 Read pv module details for HA from config file 2024-04-07 19:41:05 +02:00
Stefan Allius
214f3dfae5 Add manufacturuer and modell type for pv modules 2024-04-07 10:29:05 +02:00
Stefan Allius
2153d7c15c cleanup 2024-04-06 20:20:42 +02:00
Stefan Allius
8fc8a29be2 clear daily energy production at midnight 2024-04-06 00:04:25 +02:00
Stefan Allius
f3842d95d8 add testcases for building model names 2024-04-01 21:24:07 +02:00
Stefan Allius
ef8a461569 build gen 3 inverter modell name 2024-04-01 20:05:51 +02:00
Stefan Allius
73c35de3e5 add more values to Home Assistant 2024-04-01 15:00:15 +02:00
Stefan Allius
80f4dd722a remove useless parameter from _key_obj() 2024-04-01 02:08:28 +02:00
Stefan Allius
f38fea3807 move ignore_this_device() into base class Infos 2024-04-01 00:48:33 +02:00
Stefan Allius
695d8a8906 count AT commands in home assiatant 2024-03-31 23:56:18 +02:00
Stefan Allius
643c0026d8 count INVALID_MSG_FMT errors 2024-03-31 00:26:54 +01:00
Stefan Allius
3d375d86be add set_db_def_value() 2024-03-30 21:48:25 +01:00
Stefan Allius
71ec0570ac make _info_defs and _info_devs private 2024-03-30 11:58:38 +01:00
Stefan Allius
e3fdeecf82 parse gen3plus inverter data 2024-03-30 01:15:07 +01:00
Stefan Allius
738dd708ac refactor ha_confs() interface 2024-03-29 19:21:59 +01:00
Stefan Allius
37cb7cc1a1 implent register mapping 2024-03-29 10:48:09 +01:00
Stefan Allius
21e46ae456 refactor info class 2024-03-28 20:56:13 +01:00
Stefan Allius
ef1fd4f913 Gen 3 plus support (#38)
* add tsun_v2 default configuration

* Add port 10000 for gen 3 plus inverters

* add monitor_sn for solarman support

* listen on port 10000 for solarman inverters

* initial version for gen 3 plus support

* refactoring split gen3 and gen3plus

* refactoring

* refactoring classes

* refactor proxy statistic counter

* - fix loggin levels
- user super() in close() and __del__()

* add config for gen 3 plus

* Add solarman config support

* refacot Message.. classes

* rename class MessageG3 into Talent

* refactor close() handler

* refactor disc() handler

* move loop() into the base class AsyncStream

* move async_read, _write and _forward into base class

* Cleanup

* move server_loop and client_loop into basic class

* add msg forwarding for solarman V5 protocol

* move server_loop() and client_loop to class AsyncStream

* rename AsyncStreamxx ton Connectionxx

* fix unit tests

* make more attributes privae

* load .env file

* wait after last test

* ignore .env

* add response handler

* Update README.md

* update unreleased changes

* home assistant add more diagnostic values

* fix typo

* Update README.md

Definition of the inverter generations added to the compatibility table

* add ha couter for 'Internal SW Exceptions'

* Update README.md

Fixes an incorrect marking in the display of the configuration file

* Update README.md

Planning documented for MS-2000 support

* S allius/issue33 (#34)

* - fix issue 33

  The TSUN Cloud now responds to contact_info and get_time messages with
  an empty display message and not with a response message as before.
  We tried to parse data from the empty message, which led to an
  exception

* Add test with empty conn_ind from inverter

* version 0.5.5

* add tsun_v2 default configuration

* Add port 10000 for gen 3 plus inverters

* add monitor_sn for solarman support

* listen on port 10000 for solarman inverters

initial version for gen 3 plus support

* refactoring split gen3 and gen3plus

* refactoring

* refactoring classes

* refactor proxy statistic counter

* - fix loggin levels
- user super() in close() and __del__()

* add config for gen 3 plus

* Add solarman config support

* refacot Message.. classes

* rename class MessageG3 into Talent

* refactor close() handler

* refactor disc() handler

* move loop() into the base class AsyncStream

* move async_read, _write and _forward into base class

* Cleanup

* move server_loop and client_loop into basic class

* add msg forwarding for solarman V5 protocol

* move server_loop() and client_loop to class AsyncStream

* rename AsyncStreamxx ton Connectionxx

* fix unit tests

* make more attributes privae

load .env file

* wait after last test

* ignore .env

* add response handler
2024-03-27 01:40:29 +01:00
Stefan Allius
cef28b06cd add ha couter for 'Internal SW Exceptions' 2023-12-24 11:49:26 +01:00
Stefan Allius
3e217b96d9 home assistant add more diagnostic values 2023-12-15 23:27:06 +01:00
Stefan Allius
ea95e540ec - Fixed detection of the connected inputs/MPPTs
- Add data acquisition interval
- Add number of connections
- Add communication type
- Preparation for overwriting received data
2023-12-10 15:13:44 +01:00
Stefan Allius
3b75c45344 OTA update (#29)
* add pv module configuration

* add OTA start message counter

* add OTA start message counter

* fix test_statistic_counter
2023-11-22 18:33:56 +01:00
Stefan Allius
02861f70af - add int64 data type to info parser 2023-11-07 00:19:48 +01:00
limes007
647ef157d4 fix f-string 2023-11-04 23:29:53 +01:00
Stefan Allius
7f91994934 fix multiline bugs 2023-11-02 23:50:49 +01:00
Stefan Allius
2c4af0b7d8 lint code with flake8 2023-11-02 22:31:30 +01:00
Stefan Allius
85be9072db adapt interface of ha_confs 2023-11-01 21:09:57 +01:00
Stefan Allius
47878adb23 fix definition of the 'Unknown Control Type' counter 2023-10-30 23:43:18 +01:00
Stefan Allius
cfe64b1eae add Unknown_Ctrl error counter 2023-10-28 00:09:19 +02:00
Stefan Allius
0c9f953476 don't initialize must_incr values with zero
- when the connection is just established by the inverter.
  sometimes the inverters send invalid data with the value zero.
  In this case, we no longer initialize the must_incr values,
  to avoid sending invalid data to the mqtt broker and the
  Home Assistant
2023-10-26 20:23:53 +02:00
Stefan Allius
9264faaf3d avoid resetting daily generation counters 2023-10-21 21:38:36 +02:00
Stefan Allius
f96091affd Increase test coverage for the Infos class to 100% 2023-10-21 16:46:49 +02:00
Stefan Allius
311b36aaf1 avoid resetting total generation counters 2023-10-20 21:51:02 +02:00