The concept of prefab is, indeed, to make housing prices more affordable over time.
The catch is that 'over time' part.
For prefab to become efficient, there has to be an economies of scale where the per-unit price drops significantly as the output increases.
Prefab is nothing new, really. Kit homes have been around forever (you used to be able to order a house from Sears). And, of course, the mobile home industry.
The 'new wave' of prefab over the past decade or so hasn't been a prefab movement meant to create huge economies of sale. The companies that are in the market now (that you see advertised in the likes of Dwell, for instance) are targetting a very particular demographic: Namely upper class professionals with a penchant for mid-century furnishings.
As such, these are higher end homes and hence their comparison to 'custom built' homes. By that they are likely referring to architect designed site-specific structures (vs. the "pick a model out of the catalog and we'll customize the counter top for you).
That's not to say all prefab outfits are targeting only higher-end consumers. For instance there are many 'manufactured homes' companies such as Woodlund that are clearly targeting the middle-class. And then there are niche companies like Escape who are specifically using lower-cost as a selling point even with their more custom designs.
Somewhat ironically, the more prefab options out there, the less likely prefab will drop in price, as each company never really gets to the point where the economies of scale drop the price significantly.
In addition, the price of prefab vs. site built can vary wildly based on labor costs--the on-site location vs. the factory. One can build a house faster in a prefab factory than on-site. And if the labor is cheaper at the factory, this can be where the big savings come in.
Finally, note that cost isn't always the driving factor for going with prefab. Speed of erection is a significant benefit as well. Especially if a house is going up in February in Minnesota, or during the raining season on the Washington coast.