I hate answering with a question, but you really should have a solid answer to what are your goals?
In short, you want to stress your system enough to cause adaptation. The four cycles go:
- Initial fitness (what you walk into the weight room with).
- Training (weakening your body through targeted stress).
- Recovery (your body is actively repairing you through active and passive rest).
- Supercompensation (your body is bigger/faster/stronger by a small margin).

The general rep-range data looks like this:

So to really determine "how close to failure", you really need to know what you're looking to accomplish. In general though training to failure consistently is a poor idea especially in the lower rep ranges. Right before "failure" happens your form deteriorates and exposes you to serious risk if you have sufficient load.
Doing bicep curls to failure is one thing, doing deadlifts to failure is another.
An overarching thing you might want to consider is getting on a good program. You'll find a lot of love towards Starting Strength by Mark Rippetoe. If you don't want to buy the book, you can find enough info online to cobble together the right ingredients but the ~$30 the book will cost is far more than worth the amount of time, injuries, and frustration you'll save.
Progressive Overload
Your main goal should be to progressively overload your body, causing you to routinely go into supercompensation throughout the week. It's these small incremental increases (think: filling up a bathtub one tablespoon at a time) that add up to strength and mass increases.
To systematically and predictably do that, you can't just head into the gym and toss weight around. You should be entering a gym knowing (a) what you're going to do and (b) how much of it you're going to do, and those should be calculated values.
You can get strong by randomly lifting weights, but if you want to get past the novice stage and progressive as an intermediate athlete you'll need to have solid programming.