It sounds as though you have accidentally added some keyframes to your object's location (done by pressing I). If this is correct the Timeline view should look something like this, with little yellow bars at the keyframes:

Because the object's location is keyframed, its position is defined by the animation curve between the keyframes. If you move the object (without setting a new keyframe for that location) and then change frames or play the animation, Blender will automatically snap the object back to its proper position in the animation.
Pressing the < or > arrow keys will advance the animation by one frame left or right, the ^ or v keys will advance to the next or previous keyframes.
There are two ways to remove keyframes. You can go to each individual frame with a keyframe and press Alt+I > Delete Keyframe. Or you can go to the Graph Editor or Dope Sheet, select all the keyframes with A, and press Del or X > Delete Keyframes.
