Hi - this is something that takes a bit of knowledge and experience to assess, although there are some guidelines to help and most likely won't need much modelling other than simple back-of-the-envelope calculation. I think using OpenStudio might be a bit too much for your needs?
There are two key concepts you need to consider, with one being heating/cooling capacity (imagine horsepower for a car) which you'll pay up front and the other being operating cost (e.g. like gas/oil change etc). In HVAC, the capacity is measured in 'ton' (1 ton = 12,000 BTU = 3.5 kW). So you need to estimate how much heating & cooling capacity your house will need and compare the upfront installation cost and the operarting cost over some reasonable period (say 10~20 years).
Some food for thought:
- In general, ground source heat pump system is the most expensive in terms of heating/cooling capacity (> $10k/ton), but most economical in terms of operating cost. In terms of overall lifecycle cost however, not really worth it for most homes.
- Air-source heat pump will also be somewhat expensive to install (~ $3k/ton) but If you live in moderate climate (e.g. San Francisco), operating cost can be competitive with natural gas. Note that the bulk of the total installation cost (~2/3) is actually labor cost, with equipment cost only about 30%, so you'll get a lot of variation in quotes. Unfortunately, very few contractors will give you a cost breakdown.
- If natural gas is available, gas furnace will be the cheapest in terms of capacity and for most places in the US, also cheapest to operate, although not terribly much cheaper than air-source heat pump. For new homes, this will require ducts to be built also.
- Most contractors will quote you a system that's much larger than what you need.
Not sure if this helps but I've seen a lot of people get lost researching this so hopefully it's somewhat helpful...
