Need Help with Video stabilization

No nothing to do with coding. Tell me what you know , I'll ask my doubts.

Here is my understanding of it, correct me where I am wrong.

Features (any kind SIFT/SURF/FAST etc) are tracked across the video frames to get the per-frame-pair affine transformation matrices. These matrices are multiplied. At any instant this product is called the Camera Path. A convex optimization problem is solved using these transforms and the camera path to obtain an update transform. This is used to modify the camera path to warp the input frame to a new frame of reference such that the video appears smooth. This is done by having a variable cropbox which shakes with the camera, thus providing the view within itself to be stationary (this is my own assuaging of it although it might be something else).

I've also tried this by directly (not tracking any features at all) warping (LK warp, using the iat toolbox) without tracking any features and still nothing. Along with this other things I've tried are, Optimizing dynamically over every frame (not summing up the objective function), Summing up the objective whole function and then optimizing over each frame etc.

When solving the convex optimization problem (as per those papers) is the objective added over a window of frames (Obj for frame 1 + Obj for frame 2 + Obj for frame 3 + ........... + Obj for frame N = TOTOBJ) and then minimized over each frame (min(TOTOBJ over Frame1) , min(TOTOBJ over Frame2), min(TOTOBJ over Frame3) .... min(TOTOBJ over FrameN)) or The objective is calculated for each frame and instantaneously minimized over that frame ( min(Obj1 over Frame1), min(Obj2 over Frame2), min(Obj3 over Frame3)... min(ObjN over FrameN) )

/r/computervision Thread Parent