Zone Temp/Humd EMS control of Unitary HVAC?

edit

I am trying to control the 5 zones of the OpenStudio DOE Prototype Building (Small Office) with Python EMS. The HVAC is the default Unitary System, each zone has its own and own Air Loop.

I can access whatever EMS Sensors/Actuators through the API, and can expose them via OpenStudio so this is not my trouble. My trouble is how to set up the HVAC model so that I can use high-level supervisory control to control the zone temperatures (humidity also) 24/7 - such as cooling and heating setpoints.

I see that there are existing Ruleset Schedules, Night Availability Managers, and the control type of the Unitary HVAC is 'load'. I am new to EnergyPlus and the HVAC systems so I do not understand all the dependencies that must be properly configured so I can have full control via Python for reinforcement learning purposes. I am working trial-by-error currently and not making much progress.

Below are some of the results that I am not happy with because the zone temps occasionally leave the control passband.

Any explanation or model components I should be aware of would be greatly appreciated. Thank you.