This is a really amateurish question but I'm completely lost at the movement. I trying to create a cloth with a wooly vest. However, no matter how I tried I cant seem to get the result I wanted. I've manage to sculpt and bake the fold to normal, However, I fail to create the "wool texture/feeling". which lead to this post
How does artist usually create pattern and details such as leather, wool, etc on to the cloth other model?
I've try directly sculpting on high poly model (through stencil) however the details is not consistent.
I've tried merging two normal map together(the one I've bake which include the fold and a wool normal map from the internet), it a didn't go so well as I've found it is best to bake your own normal then using other. The wool map does "join" correctly as shown on the image

texture painting heavily rely on the artist skill (which embarrassingly, I'm not at the level on the model) and it is not suitable for the style I'm trying to pursuit. I would like the final to be something like this where at a glance a person could the pattern,texture and the details of the fabric.

Hence, I'm completely lost. Clearly there is way to do it as many have able to create. I just cant seem to figure it out.
The link to my blender file it contain the mesh,3 normal maps and a node setup for merging N1 and N2 , the following are the name of the maps.
N1: the one with fold only(I acquire through baking from high to low poly) it is intended to mix with N2. N2: The normal I found on the internet , intended to mix with N1. If directly use on mesh, it will not "join" together properly refer to the image above. N3: I directly sculpt the "wool texture" on to the high poly and bake it to the low poly, as you can see the details is not very consistent



