I don’t know of any program that would directly accept a blend file for avatars, so I think you would be better off putting all assets in one blend file, and hiding/unhiding individual assets as desired. Most, if not all, file exporters in Blender allow you to export only the selection, which, as far as I know, is only considered to include visible objects.
To make things easier to find, you might group similar assets (i.e. assets that would clash with each other as they apply to the same body part) into labeled collections (for example, “Hats”, “Facewear”, “Shirts”, etc).