Python，OpenCV —对齐并覆盖多个图像，一个接一个

Question

我的项目是对齐航空照片，以制作出一张马赛克地图。我的计划是从两张照片开始，将第二张照片与第一张照片对齐，然后从两张对齐的图像中创建一个“初始镶嵌”。完成此操作后，我将第三张照片与初始镶嵌图对齐，然后将第四张照片与该图的结果对齐，以此类推，从而逐步构建地图。

我有两种技术可以做到这一点，但是使用的更精确的技术calcOpticalFlowPyrLK()仅适用于两个图像阶段，因为两个输入图像的大小必须相同。因此，我尝试了一种新的解决方案，但是它的准确性较差，并且在每个步骤中引入的错误都会堆积起来，最终产生荒谬的结果。

我的问题有两个方面，但是，如果您知道一个答案，那么除非您愿意，否则不必都回答。首先，是否有办法使用类似于calcOpticalFlowPyrLK()但带有两个不同大小的图像的方法（这包括任何可能的解决方法）？其次，是否有一种方法可以修改检测器/描述符解决方案以使其更加准确？

这是仅适用于两个图像的准确版本：

# load images
base = cv2.imread("images/1.jpg")
curr = cv2.imread("images/2.jpg")

# convert to grayscale
base_gray = cv2.cvtColor(base, cv2.COLOR_BGR2GRAY)

# find the coordinates of good features to track  in base
base_features = cv2.goodFeaturesToTrack(base_gray, 3000, .01, 10)

# find corresponding features in current photo
curr_features = np.array([])
curr_features, pyr_stati, _ = cv2.calcOpticalFlowPyrLK(base, curr, base_features, curr_features, flags=1)

# only add features for which a match was found to the pruned arrays
base_features_pruned = []
curr_features_pruned = []
for index, status in enumerate(pyr_stati):
    if status == 1:
        base_features_pruned.append(base_features[index])
        curr_features_pruned.append(curr_features[index])

# convert lists to numpy arrays so they can be passed to opencv function
bf_final = np.asarray(base_features_pruned)
cf_final = np.asarray(curr_features_pruned)

# find perspective transformation using the arrays of corresponding points
transformation, hom_stati = cv2.findHomography(cf_final, bf_final, method=cv2.RANSAC, ransacReprojThreshold=1)

# transform the images and overlay them to see if they align properly
# not what I do in the actual program, just for use in the example code
# so that you can see how they align, if you decide to run it
height, width = curr.shape[:2]
mod_photo = cv2.warpPerspective(curr, transformation, (width, height))
new_image = cv2.addWeighted(mod_photo, .5, base, .5, 1)

这是不正确的图像，适用于多张图像（直到错误变得太大为止）：

# load images
base = cv2.imread("images/1.jpg")
curr = cv2.imread("images/2.jpg")


# convert to grayscale
base_gray = cv2.cvtColor(self.base, cv2.COLOR_BGR2GRAY)

# DIFFERENCES START
curr_gray = cv2.cvtColor(self.curr_photo, cv2.COLOR_BGR2GRAY)

# create detector, get keypoints and descriptors
detector = cv2.ORB_create()
base_keys, base_desc = detector.detectAndCompute(base_gray, None)
curr_keys, curr_desc = detector.detectAndCompute(curr_gray, None)

matcher = cv2.DescriptorMatcher_create("BruteForce-Hamming")

max_dist = 0.0
min_dist = 100.0

for match in matches:
     dist = match.distance
     min_dist = dist if dist < min_dist else min_dist
     max_dist = dist if dist > max_dist else max_dist

good_matches = [match for match in matches if match.distance <= 3 * min_dist ]

base_matches = []
curr_matches = []
for match in good_matches:
    base_matches.append(base_keys[match.queryIdx].pt)
    curr_matches.append(curr_keys[match.trainIdx].pt)

bf_final = np.asarray(base_matches)
cf_final = np.asarray(curr_matches)

# SAME AS BEFORE

# find perspective transformation using the arrays of corresponding points
transformation, hom_stati = cv2.findHomography(cf_final, bf_final, method=cv2.RANSAC, ransacReprojThreshold=1)

# transform the images and overlay them to see if they align properly
# not what I do in the actual program, just for use in the example code
# so that you can see how they align, if you decide to run it
height, width = curr.shape[:2]
mod_photo = cv2.warpPerspective(curr, transformation, (width, height))
new_image = cv2.addWeighted(mod_photo, .5, base, .5, 1)

最后，这是我正在使用的一些图像：

Answer 1

单应组成，所以如果你有间单应img1和img2之间，img2和img3那么这两个单应性的组合物提供之间的单应img1和img3。

您的尺寸不正确，因为您正试图匹配img3包含img1和的拼接图像img2。但是您不需要这样做。在每对连续的图像之间都有所有的单应性之前，请不要缝合它们。然后，您可以采用以下两种方式之一进行操作：从背面或正面进行工作。我将使用h31来指代扭曲img3为的坐标的单应性img1。

从前面（伪代码）：

warp img2 into coordinates of img1 with h21
warp img3 into coordinates of img1 with h31 = h32 @ h21
warp img4 into coordinates of img1 with h41 = h43 @ h31
...
stitch/blend images together

这@是矩阵乘法运算符，它将实现我们的单应性构图（请注意，用单应性的最后一项进行除法是最安全的，以确保它们均按相同比例缩放）。

从后面（伪代码）：

...
warp prev stitched img into coordinates of img3 with h43
stitch warped stitched img with img3
warp prev stitched img into coordinates of img2 with h32
stitch warped stitched img with img2
warp prev stitched img into coordinates of img1 with h21
stitch warped stitched img with img1

这个想法是，您可以从正面开始，然后将所有内容扭曲到第一个图像坐标系中，或者从背面开始，将所有内容扭曲到上一个图像并进行绣制，然后将那个已缝制的图像扭曲到先前的图像中，然后重复。我认为第一种方法可能更容易。无论哪种情况，您都必须担心单应性估计中误差的传播，因为它们会在多个组合单应性上累积。

这是将多个图像与同形异义词混合在一起的幼稚方法。更复杂的方法是使用捆绑调整，该调整考虑了所有图像的特征点。为了获得良好的融合效果，需要进行增益补偿以消除相机的增益调整和渐晕，然后进行多波段融合以防止模糊。见布朗和Lowe的开创性的论文在这里和一个光辉的榜样和免费demo软件在这里。