fisheye | fisheye image calibration | Computer Vision library

Going for mp4 container wasn't a good choice after all. It turns out the RTP essentially yields raw h264 stream. To inspect its structure I converted the genuine mp4 recording to .264 like this:

I will answer this question in the spirit that it was asked, i.e. no code.

JavaFX has an effect framework.

There is no in-built fisheye effect.

You could create your own custom fisheye effect implementation and plug it into the effect framework if you are a skilled developer.

Easier would be to apply your algorithm using a WritableImage with a PixelWriter or Canvas. Perhaps that could even plug into the effect framework (if you actually needed to do that, which you probably don't) using an ImageInput.

For an example of applying an algorithm to the pixels in an input image see:

• Reduce number of colors and get color of a single pixel

Of course, you would use a fisheye algorithm (coded for JavaFX instead of the linked implementations) for a fisheye transform.

To animate use an AnimationTimer or, again for skilled developers, create a custom transition that plugs into the JavaFX animation framework.

You can add properties to your custom effect and manipulate them using additional properties defined on the custom transition you create.

Providing a complete solution is out of scope for a StackOverflow answer. To get help with individual tasks, split the problem up into different pieces, e.g. creating a custom effect, manipulating pixels to create a fisheye, animating an effect on an image or timeline, etc. Write the code and ask questions about the actual code with a minimal example for the problem portion you are trying to solve when you get stuck.

In ONVIF, For any feature to be confirmed as supported by the device usually can be done in one or more ways (including but not limited to the list) listed below

1. Reading the device/service capabilities using GetServiceCapabilities interface provided by every service
2. Reading the configurations provided by device using 'Get[entity]Configurations' interface
3. Reading the configuration parameter options using Get[entity]ConfigurationOptions interface

The "entity" varies depends on the feature, Check the entities list here

• https://www.onvif.org/specs/srv/media/ONVIF-Media2-Service-Spec.pdf Section 4.1 Media profile)

For your query about dewarp feature support, option 2 has to be checked. So you have to read VideoSourceConfiguration from the device via 'GetVideoSourceConfiguration' interface and check the response.

The response from device shall adhere to the specification as quoted below

Ref: https://www.onvif.org/specs/srv/media/ONVIF-Media2-Service-Spec-v1712.pdf

• Section : 5.2.2 Video source configuration
  • View Mode
    • Fisheye – Undewarped viewmode from a device supporting fisheye lens
    • Dewarp – Dewarped view mode for device supporting fisheye lens

The option you're looking for is roll?

I figured out my issue-

Is this a one-off project, or are you trying to write an automatic procedure?

If the former, you can just hand-tag corresponding points in the two images till you have enough coverage, particularly in the more "curvy" areas, then warp using (for example) a thin-plate spline defined by the point correspondences. This works iteratively: match some points, warp, figure out where the error is largest, add more point matches there. Doing all this in, say, Matlab or Python+OpenCv is quite easy.

You can then try to automate it using feature extraction and matching. The trouble is you'll need to use both point and line correspondences...

The image seems just wide-angle, not fisheye. Images from fisheye camera usually have black circle borders, and they look like seeing through a round hole. See picture c) below (from OpenCV doc):

The normal method of distinguishing wide-angle from fisheye is to check the FOV angle.

Given the camera intrinsic parameters (the cameraMatrix and distCoeffs, from calibration routine), you can calculate a new camera intrinsic matrix with the largest FOV and no distortion by calling getOptimalNewCameraMatrix(). Then the FOV angle in x-direction (it's usually larger than y-direction) is arctan(cx/fx)+arctan((width-cx)/fx), where fx is the focal length in x-direction, cx is the x-coordinate of principal point, and width is the image width.

In my experience, when FOV<80°, the Brown distortion model (k1, k2, k3, p1, p2) should be used. When 80°<140°, the Rational model (k1~k6, p1, p2) should be used. And when 140°<170°, the Fisheye model (k1, k2, k3, k4) should be used. More complex model (with more parameters) has better fitting ability, but also makes the calibration harder.

The Fisheye model performs better than Rational model in very large FOV cases, because it has higher order radial parameter. But when FOV angle is not very large, Rational model is a better choice, because it has tangential parameters.

The undistorted picture you provided seems fairly good. But if you care more about the precision, you should check the reprojection error, epipolar error (multi-camera), and even collineation error of the key points (checkerboard corners, circle centroids, or whatever features depending on your calibration pattern) in calibration procedure.

You need at least version 4.3 to use the v360 filter (see Changelog). For Linux you can download a new version or compile it yourself.

The online documentation is synced with the latest code (git master branch). So releases may not have features mentioned in the online documentation. Refer to your locally installed documentation if you are stuck using an old release: man ffmpeg-filters

I think you need to estimate new camera matrix for undistortion.