Yes, you can create some fenestration materials/constructions with the OpenStudio Application (App). Yet not every EnergyPlus option is available through the App.
The most basic is the Simple Glazing System Window Material (requires rated U, SHGC & VT) - look up the "Materials" subtab under the "Constructions" tab:

At the opposite end, EnergyPlus can import LBNL Window data for a specific fenestration product. See here, and/or look up "5.4 Getting data from WINDOW program" here. I don't believe this is supported in the App, but it is mapped to the SDK behind the scenes, e.g.:
OS:Construction:WindowDataFile,
{90d22889-50da-4091-8355-fa2ff516e4e6}, !- Handle
Construction Window Data File 1; !- Name
It's very basic - something you could copy/paste in a saved OSM file (before reopening using the App).
You can also look up Frame and Dividers (see here).
... it really depends on what you're after, and what inputs you have on hand.