Remove Lens Distortion from OpenCV Images Using Camera Calibration

Problem: straight lines look curved in your camera frames

Most lenses introduce radial and tangential distortion. Straight edges near the frame border appear bowed, which breaks pixel measurements, stitching, and 3D reconstruction. OpenCV can estimate a camera's intrinsic matrix and distortion coefficients from checkerboard photos, then remove the distortion from new images.

Takeaway: calibrate once for a fixed lens, focus, and zoom setting, then apply the same coefficients to every frame from that camera. If you change zoom or focus, the coefficients are no longer valid.

Prerequisites

• OpenCV 3.x or 4.x installed for Python (imported as cv2). Core calibration functions are in the main package; no contrib modules are required.
• A printed checkerboard on a flat surface. Record the inner corner count (for example, 9 by 6) and the physical square size in millimeters.
• Ten to twenty photos of the same checkerboard from different angles and distances.
• Read access to the calibration image folder and write access to the output folder.

Capture calibration images

Place the checkerboard on a flat surface. Photograph it so the full board is visible, tilted at different angles, and positioned near the center and edges of the frame. Avoid glare and motion blur. Do not adjust zoom, focus, or aperture between shots; any optical change invalidates the calibration.

Detect checkerboard corners

import cv2
import numpy as np
import glob

# Inner corners of the checkerboard, not the total squares.
CHECKERBOARD = (9, 6)
SQUARE_SIZE_MM = 25.0

criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001)

# 3D points in real-world space, assuming the board lies on z=0.
objp = np.zeros((CHECKERBOARD[0] * CHECKERBOARD[1], 3), np.float32)
objp[:, :2] = np.mgrid[0:CHECKERBOARD[0], 0:CHECKERBOARD[1]].T.reshape(-1, 2)
objp *= SQUARE_SIZE_MM

objpoints = []  # 3D points in real world
imgpoints = []  # 2D points in image plane

images = glob.glob('calibration_images/*.jpg')
for fname in images:
    img = cv2.imread(fname)
    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    ret, corners = cv2.findChessboardCorners(gray, CHECKERBOARD, None)
    if ret:
        objpoints.append(objp)
        corners2 = cv2.cornerSubPix(gray, corners, (11, 11), (-1, -1), criteria)
        imgpoints.append(corners2)

Run this in a Python interpreter or script with the working directory set to the project folder. The script needs read permission for calibration_images/. If a board is not detected, that image is skipped; do not force it by lowering the corner count, because mismatched geometry corrupts the calibration.

Estimate the camera matrix and distortion coefficients

ret, camera_matrix, dist_coeffs, rvecs, tvecs = cv2.calibrateCamera(
    objpoints, imgpoints, gray.shape[::-1], None, None
)
print('Reprojection error:', ret)

calibrateCamera returns the mean reprojection error in pixels. This is the average distance between detected corners and corners reprojected from the estimated model. Below roughly 1.0 pixel is generally acceptable for machine-vision work; below 0.5 pixel is better. If the error is high, collect more varied views or check for blurred frames.

Choose the undistortion camera matrix

cv2.getOptimalNewCameraMatrix controls how much of the distorted border is kept. The alpha parameter ranges from 0 to 1:

Alpha	Effect
0	Crops black border pixels; keeps only valid undistorted pixels.
1	Retains the full image size, including black regions around the border.

h, w = img.shape[:2]
new_camera_matrix, roi = cv2.getOptimalNewCameraMatrix(
    camera_matrix, dist_coeffs, (w, h), 1, (w, h)
)

Use alpha=1 when you need the same frame size for downstream stitching. Use alpha=0 when you want to discard black borders and keep only valid pixels. The returned roi defines the valid pixel rectangle after cropping.

Undistort images or video frames

For a single image, cv2.undistort is simplest. For many frames, precompute the mapping once with cv2.initUndistortMap and reuse it with cv2.remap; this avoids recalculating the lookup table for every frame.

# Single image
undistorted = cv2.undistort(img, camera_matrix, dist_coeffs, None, new_camera_matrix)

# Crop to valid ROI if alpha was 0
x, y, crop_w, crop_h = roi
undistorted = undistorted[y:y + crop_h, x:x + crop_w]

# Reusable map for video or batch processing
map1, map2 = cv2.initUndistortMap(
    camera_matrix, dist_coeffs, None, new_camera_matrix, (w, h), cv2.CV_32FC1
)
undistorted = cv2.remap(img, map1, map2, cv2.INTER_LINEAR)

Run the remap version inside a frame loop. Writing output files requires write permission; do not overwrite the original calibration images until you have verified the result.

Check the result

1. Undistort a held-out checkerboard image that was not used during calibration.
2. Visually confirm that board edges and straight lines near the frame border are straight.
3. Reproject object points with cv2.projectPoints and compare them to detected corners to quantify residual error.
4. Check that the ROI is not empty; an empty or zero-sized ROI means the new camera matrix discarded the whole frame.

Limitations and recovery

• Calibration is tied to one optical setting. Changing zoom, focus, or aperture invalidates the matrix and coefficients; recalibrate for each fixed configuration.
• Undistorted images often contain black border regions. Always validate the ROI before cropping or stitching.
• If reprojection error is above roughly 1.0 pixel, discard blurry or glare-affected images and recalibrate with more varied board positions.
• Recovery: keep the original distorted images. Undistortion creates new pixel data but cannot recover information outside the original lens image circle.