I am not able to this shoot https://youtu.be/PkEsl5c5nC4. The starting is good but then it diverges.
These are my trackers and error is 0.41
this is my hold out material because I need the scene to be there
See that a slight diversion!!
I tried some tricks I saw on YouTube like reducing the weight of some bad tracks and using the refine options.
Thanks for posting the video, it makes it much easier to help you.
Your shot is by no means a complex shot. Even though your camera barely moves, there is plenty of information there.
To effectively reconstruct the scene you need to pay attention to parallax, meaning paying attention to how tracked elements that are close to the camera move at different speeds in the frame, than those elements that are far away.
The elements that are close to the camera and the furthest ones are the most useful to reconstruct 3D space.
The emphasis should be in trying to accurately track elements that are in fixed in real life, but at different distances from the camera. You effort should be in tracking elements in foreground, middle and background.
In this particular case add more tracking points on the cracks on the walls and the railing, as well as features in the lamp post behind the child, and then the background buildings and, most importantly: in the tree behind the building (which seems to be the object that is the farthest away). Don't track any of the cables, as they don't really have any distinctive features.
One more thing that is crucial in this kind of shots, that use a wide angle lens very close to the subject, is to determine the lens distortion correctly.
Please take a few minutes to read very carefully through the comprehensive post on motion tracking on this same site, to understand how to make better use the tools that will help you reconstruct the scene:
