So I'm editing a model and for a while now, the camera seems to "jump" to a different view whenever I click. I'm using the shift+F method for moving the camera, I'll move the view to see some vertices better to continue working and when I click the mouse to stop using the camera it jumps to a view that is really off.
For example, the area I'm working on(on the model) is the area I've circled in red and this is after I've pressed Shift+F and in free-roam camera mode(I think that's what it's called)
And this happens when I click to exit free roam camera to work on the model

So any ideas on how to fix this? I've changed the clipping distance minimum so I could get this close to edit(otherwise the vertices vanish) so please don't say to go to default, otherwise I can't do the internal work on the model.
Update: I know clicking with the right mouse button resets the initial position and no, I've been clicking with the left, not right button. Update: The fix/answer was posted in this question: Walk mode camera "jumps" back when stopped
In short, scroll as far in with the mouse wheel as you can, then it won't happen. That causes the camera to jump.
Numpad periodorShift+Bshortcuts for centering view, see http://blender.stackexchange.com/questions/696/how-to-reset-the-center-of-rotation-of-the-3d-view-when-it-is-not-the-center – Mr Zak Sep 01 '16 at 11:05