
Also during my testing I switched everything over from TCP to UDP to reduce lag a couple seconds and tried to enable multicast but I’m not sure if it’s working.

I have tested the MJPEG stream from the substream too. The stream has never dropped from those systems that I can remember. To begin, I am running 4x 1080p hikvision cameras configured with a 1080p h.264 main stream and a ~700x500 h.264 substream and they are working perfectly using the same streams with my synology and frigate object detection. In my quest for something reliable and real time, I think I have now tried every option that is open to me and realized that none of them is really perfect, but I’ve learned enough that it’s at least worth sharing so that other people have some better idea of their options and which one might fit their use case first. Unfortunately this simple desire has been far from easy to achieve. When I started with Home Assistant a couple years ago, one of the main use cases was making a nice dashboard which included my cameras.
