OCI镜像制作工具之skopeo

壤驷阳冰

2023-12-01

1. skopeo简介

1.1 简介

skopeo 是一个命令行工具，用于对容器镜像和镜像库执行各种操作，支持使用 OCI 镜像与原始的 Docker v2 镜像。可对容器镜像和容器存储进行操作。在没有dockerd的环境下，使用 skopeo 操作镜像是非常方便的。Skopeo 能够在容器注册表上检查存储库并获取图片层。Inspect 命令获取存储库的清单，它能够向您显示关于整个存储库或标记的类似 docker inspect 的 json 输出。与 docker inspect 不同，这个工具可以帮助您在拉取存储库或标记之前(使用磁盘空间)收集有用的信息。Inspect 命令可以显示给定存储库可用的标记、图像的标签、图像的创建日期和操作系统等等。

1.2 特性

skopeo 使用 API V2 注册表，例如 Docker 注册表、Atomic 注册表、私有注册表、本地目录和本地 OCI 布局目录。skopeo
不需要运行守护进程，它可以执行的操作包括：

通过各种存储机制复制镜像，例如，可以在不需要特权的情况下将镜像从一个注册表复制到另一个注册表
检测远程镜像并查看其属性，包括其图层，无需将镜像拉到本地
从镜像库中删除镜像
当存储库需要时，skopeo 可以传递适当的凭据和证书进行身份验证

2. 安装

RHEL/CentOS ≤ 7.x

sudo yum -y install skopeo

macOS

brew install skopeo

详细可参考：https://github.com/containers/skopeo/blob/main/install.md

3. 使用案例

$ skopeo -h
Various operations with container images and container image registries

Usage:
  skopeo [flags]
  skopeo [command]

Available Commands:
  copy                                          Copy an IMAGE-NAME from one location to another
  delete                                        Delete image IMAGE-NAME
  help                                          Help about any command
  inspect                                       Inspect image IMAGE-NAME
  list-tags                                     List tags in the transport/repository specified by the REPOSITORY-NAME
  login                                         Login to a container registry
  logout                                        Logout of a container registry
  manifest-digest                               Compute a manifest digest of a file
  standalone-sign                               Create a signature using local files
  standalone-verify                             Verify a signature using local files
  sync                                          Synchronize one or more images from one location to another

Flags:
      --command-timeout duration   timeout for the command execution
      --debug                      enable debug output
  -h, --help                       help for skopeo
      --insecure-policy            run the tool without any policy check
      --override-arch ARCH         use ARCH instead of the architecture of the machine for choosing images
      --override-os OS             use OS instead of the running OS for choosing images
      --override-variant VARIANT   use VARIANT instead of the running architecture variant for choosing images
      --policy string              Path to a trust policy file
      --registries.d DIR           use registry configuration files in DIR (e.g. for container signature storage)
      --tmpdir string              directory used to store temporary files
  -v, --version                    Version for Skopeo

Use "skopeo [command] --help" for more information about a command.

copy：复制一个镜像从 A 到 B，这里的 A 和 B 可以为本地 Docker 镜像或者 Registry 上的镜像；
inspect：查看一个镜像的 manifest 或者 image config 详细信息；
delete：删除一个镜像 tag，可以是本地 Docker 镜像或者 Registry 上的镜像；
list-tags：列出一个 Registry 上某个镜像的所有 tag；
login：登录到某个 Registry，和 docker login 类似；
logout：退出已经登录到某个 Registry 的 auth 信息，和 docker logout 类似；
manifest-digest：计算一个文件的 sha256sum 值；
standalone-sign、standalone-verify 这两个是和镜像加密相关的，使用的不是很多；
sync：同步一个镜像从 A 到 B，感觉和 copy 一样，但 sync 支持的参数更多，功能更强大。
command-timeout：命令超时时间；
insecure-policy：使用非安全的 policy，如果没有配置 policy 的话，需要加上该参数；
override-arch：处理镜像时覆盖客户端 CPU 体系架构，如在 AMD64 的机器上用 Skopeo 处理 ARM64 的镜像；
override-os：处理镜像时覆盖客户端 OS。
debug：开启 debug 模式，输出详细的日志；

在使用insecure-policy时的一些参数

--insecure-policy --src-tls-verify=false --dest-tls-verify=false

3.1 检查存储库

显示镜像属性
- 远程仓库

docker://: 是使用 Docker Registry HTTP API V2 进行连接远端

docker.io: 远程仓库

fedora:latest: 镜像名称及tag

$ skopeo inspect docker://docker.io/fedora:latest

利用jq查看指定的信息。

$ skopeo inspect docker://registry.fedoraproject.org/fedora:latest | jq .Digest

远程仓库也可以是如下面的其他类型：

atomic
containers-storage
dir
docker
docker-daemon
docker-tar
oci
ostree
查看本地镜像

$ skopeo inspect docker-daemon:mysql:8.0.19
{
    "Name": "docker.io/library/mysql",
    "Digest": "sha256:8084d869573a2e1be9d49d1b8dea49b54fc43301a3ead4b8124eedb809cf3c3f",
    "RepoTags": [],
    "Created": "2020-04-23T04:14:55.648457026Z",
    "DockerVersion": "18.09.7",
    "Labels": null,
    "Architecture": "amd64",
    "Os": "linux",
    "Layers": [
        "sha256:c2adabaecedbda0af72b153c6499a0555f3a769d52370469d8f6bd6328af9b13",
        "sha256:49003fe88142d189c33a6e52b97081c8d62b67e0d04e9e7421ca6a2495b0d766",
        "sha256:8d3b3830445d713c580008a547de7398ab721a083fda915526ce2cc557c4f40a",
        "sha256:49baacc63c3bb5a7ab5077ef6458d6e246710195d7794cb704940f3a40495c23",
        "sha256:24bd91e7be374167e51e37faa31e7ce143921afb0b968415c749625319386fdb",
        "sha256:d84f8cf1dc236ea2ddbc0393b0c33099bedb20fffbe68bdf170d2df161a1ca16",
        "sha256:ace74cb61ec0a3fd2847dc3a0b8b0cad349c95ce02732982e258fc1a337d073f",
        "sha256:ba6f785586f5ff477e87528a5d2c8d0215e22f5b9ac3a8f4b19a033cad7aa832",
        "sha256:c861c8d6e01539cc5b27fe773aca63958bc361391f6a980ece5322fd8dbca296",
        "sha256:1ac7b6bb690abc8073c2193635afb06b8494d2d0024dc08310b296975f92bb61",
        "sha256:b8e742ca24b882430bf4a6075aa97a8c1e8809f6e59d462cba05bec49d817ff3",
        "sha256:7fc1a766ce1413db3d5fceb7256c1e54639386673c24eadaf69b429b07eb9223"
    ],
    "Env": [
        "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
        "GOSU_VERSION=1.12",
        "MYSQL_MAJOR=8.0",
        "MYSQL_VERSION=8.0.19-1debian10"
    ]
}

3.2 拷贝镜像

在不使用 docker 的情况下从远端下载镜像

# 下载远程镜像到/tmp/下，命名为nginx.tar
skopeo --insecure-policy copy docker://nginx:1.17.6 docker-archive:/tmp/nginx.tar

# 将本地tar镜像文件导入到docker中
skopeo copy docker-archive:/tmp/nginx.tar docker-daemon:nginx:latest

#将一个镜像从 Registry 中同步到本地目录：
$ skopeo sync --src docker --dest dir k8s.gcr.io/pause:3.3 images
images
└── pause:3.3
    ├── 0184c1613d92931126feb4c548e5da11015513b9e4c104e7305ee8b53b50a9da
    ├── aeab776c48375e1a61810a0a5f59e982e34425ff505a01c2b57dcedc6799c17b
    ├── manifest.json
    └── version

#将镜像从本地目录同步到 Registry 中：
$ skopeo sync --src dir --dest docker images hub.k8s.li
INFO[0000] Copying image ref 1/1                         from="dir:images/pause:3.3" to="docker://hub.k8s.li/pause:3.3"
Getting image source signatures
Copying blob aeab776c4837 [--------------------------------------] 0.0b / 0.0b
Copying config 0184c1613d done
Writing manifest to image destination
Storing signatures
INFO[0002] Synced 1 images from 1 sources

#将镜从 Registry A 同步到 Registry B：
$ skopeo sync --src docker --dest docker k8s.gcr.io/pause:3.3 hub.k8s.li
INFO[0000] Tag presence check                            imagename="k8s.gcr.io/pause:3.3" tagged=true
INFO[0000] Copying image tag 1/1                         from="docker://k8s.gcr.io/pause:3.3" to="docker://hub.k8s.li/pause:3.3"
Getting image source signatures
Copying blob aeab776c4837 done
Copying config 0184c1613d done
Writing manifest to image destination
Storing signatures
INFO[0000] Synced 1 images from 1 sources

3.3 删除镜像

skopeo delete docker://localhost:5000/nginx:latest

3.4 注册表认证

认证文件默认存放在$HOME/.docker/config.json

文件内容

{
	"auths": {
		"myregistrydomain.com:5000": {
			"auth": "dGVzdHVzZXI6dGVzdHBhc3N3b3Jk",
			"email": "stuf@ex.cm"
		}
	}
}

3.5 列出 Registry 上的某个镜像的所有 tag

skopeo list-tags

$ skopeo list-tags docker://k8s.gcr.io/pause

3.6 批量同步镜像

3.6.1 同步镜像从一个 Registry 同步到另一个 Registry

镜像列表示例：images-list.txt

##k8s-images
kubesphere/kube-apiserver:v1.20.6
kubesphere/kube-scheduler:v1.20.6
kubesphere/kube-proxy:v1.20.6
kubesphere/kube-controller-manager:v1.20.6
kubesphere/kube-apiserver:v1.19.8
kubesphere/kube-scheduler:v1.19.8
kubesphere/kube-proxy:v1.19.8
kubesphere/kube-controller-manager:v1.19.8
kubesphere/kube-apiserver:v1.19.9
kubesphere/kube-scheduler:v1.19.9
kubesphere/kube-proxy:v1.19.9
kubesphere/kube-controller-manager:v1.19.9
kubesphere/kube-apiserver:v1.18.8
kubesphere/kube-scheduler:v1.18.8
kubesphere/kube-proxy:v1.18.8
kubesphere/kube-controller-manager:v1.18.8
kubesphere/kube-apiserver:v1.17.9
kubesphere/kube-scheduler:v1.17.9
kubesphere/kube-proxy:v1.17.9
kubesphere/kube-controller-manager:v1.17.9
kubesphere/pause:3.1
kubesphere/pause:3.2
kubesphere/etcd:v3.4.13
calico/cni:v3.16.3
calico/kube-controllers:v3.16.3
calico/node:v3.16.3
calico/pod2daemon-flexvol:v3.16.3
calico/typha:v3.16.3
kubesphere/flannel:v0.12.0

sync.sh

#!/bin/bash
GREEN_COL="\\033[32;1m"
RED_COL="\\033[1;31m"
NORMAL_COL="\\033[0;39m"

SOURCE_REGISTRY=$1
TARGET_REGISTRY=$2

: ${IMAGES_LIST_FILE:="images-list.txt"}
: ${TARGET_REGISTRY:="hub.k8s.li"}
: ${SOURCE_REGISTRY:="docker.io"}

BLOBS_PATH="docker/registry/v2/blobs/sha256"
REPO_PATH="docker/registry/v2/repositories"

set -eo pipefail

CURRENT_NUM=0
ALL_IMAGES="$(sed -n '/#/d;s/:/:/p' ${IMAGES_LIST_FILE} | sort -u)"
TOTAL_NUMS=$(echo "${ALL_IMAGES}" | wc -l)

skopeo_copy() {
 if skopeo copy --insecure-policy --src-tls-verify=false --dest-tls-verify=false \
 --override-arch amd64 --override-os linux -q docker://$1 docker://$2; then
 echo -e "$GREEN_COL Progress: ${CURRENT_NUM}/${TOTAL_NUMS} sync $1 to $2 successful $NORMAL_COL"
 else
 echo -e "$RED_COL Progress: ${CURRENT_NUM}/${TOTAL_NUMS} sync $1 to $2 failed $NORMAL_COL"
 exit 2
 fi
}

for image in ${ALL_IMAGES}; do
 let CURRENT_NUM=${CURRENT_NUM}+1
 skopeo_copy ${SOURCE_REGISTRY}/${image} ${TARGET_REGISTRY}/${image}
done

bash sync.sh docker.io localhost:5000

$ bash sync.sh docker.io localhost:5000
Progress: 1/143 sync docker.io/alpine:3.14 to localhost:5000/alpine:3.14 successful
Progress: 2/143 sync docker.io/busybox:1.31.1 to localhost:5000/busybox:1.31.1 successful
Progress: 3/143 sync docker.io/calico/cni:v3.16.3 to localhost:5000/calico/cni:v3.16.3 successful

3.6.2 从 Registry 中同步到本地目录

#!/bin/bash
GREEN_COL="\\033[32;1m"
RED_COL="\\033[1;31m"
NORMAL_COL="\\033[0;39m"

SOURCE_REGISTRY=$1
TARGET_REGISTRY=$2
IMAGES_DIR=$2

: ${IMAGES_DIR:="images"}
: ${IMAGES_LIST_FILE:="images-list.txt"}
: ${TARGET_REGISTRY:="hub.k8s.li"}
: ${SOURCE_REGISTRY:="docker.io"}

BLOBS_PATH="docker/registry/v2/blobs/sha256"
REPO_PATH="docker/registry/v2/repositories"

set -eo pipefail

CURRENT_NUM=0
ALL_IMAGES="$(sed -n '/#/d;s/:/:/p' ${IMAGES_LIST_FILE} | sort -u)"
TOTAL_NUMS=$(echo "${ALL_IMAGES}" | wc -l)

skopeo_sync() {
 if skopeo sync --insecure-policy --src-tls-verify=false --dest-tls-verify=false \
 --override-arch amd64 --override-os linux --src docker --dest dir $1 $2 > /dev/null; then
 echo -e "$GREEN_COL Progress: ${CURRENT_NUM}/${TOTAL_NUMS} sync $1 to $2 successful $NORMAL_COL"
 else
 echo -e "$RED_COL Progress: ${CURRENT_NUM}/${TOTAL_NUMS} sync $1 to $2 failed $NORMAL_COL"
 exit 2
 fi
}

convert_images() {
 rm -rf ${IMAGES_DIR}; mkdir -p ${IMAGES_DIR}
 for image in ${ALL_IMAGES}; do
 let CURRENT_NUM=${CURRENT_NUM}+1
 image_name=${image%%:*}
 image_tag=${image##*:}
 image_repo=${image%%/*}
 skopeo_sync ${SOURCE_REGISTRY}/${image} ${IMAGES_DIR}/${image_repo}
 manifest="${IMAGES_DIR}/${image}/manifest.json"
 manifest_sha256=$(sha256sum ${manifest} | awk '{print $1}')
 mkdir -p ${BLOBS_PATH}/${manifest_sha256:0:2}/${manifest_sha256}
 ln -f ${manifest} ${BLOBS_PATH}/${manifest_sha256:0:2}/${manifest_sha256}/data

 # make image repositories dir
 mkdir -p ${REPO_PATH}/${image_name}/{_uploads,_layers,_manifests}
 mkdir -p ${REPO_PATH}/${image_name}/_manifests/revisions/sha256/${manifest_sha256}
 mkdir -p ${REPO_PATH}/${image_name}/_manifests/tags/${image_tag}/{current,index/sha256}
 mkdir -p ${REPO_PATH}/${image_name}/_manifests/tags/${image_tag}/index/sha256/${manifest_sha256}

 # create image tag manifest link file
 echo -n "sha256:${manifest_sha256}" > ${REPO_PATH}/${image_name}/_manifests/tags/${image_tag}/current/link
 echo -n "sha256:${manifest_sha256}" > ${REPO_PATH}/${image_name}/_manifests/revisions/sha256/${manifest_sha256}/link
 echo -n "sha256:${manifest_sha256}" > ${REPO_PATH}/${image_name}/_manifests/tags/${image_tag}/index/sha256/${manifest_sha256}/link

 # link image layers file to registry blobs dir
 for layer in $(sed '/v1Compatibility/d' ${manifest} | grep -Eo "\b[a-f0-9]{64}\b"); do
 mkdir -p ${BLOBS_PATH}/${layer:0:2}/${layer}
 mkdir -p ${REPO_PATH}/${image_name}/_layers/sha256/${layer}
 echo -n "sha256:${layer}" > ${REPO_PATH}/${image_name}/_layers/sha256/${layer}/link
 ln -f ${IMAGES_DIR}/${image}/${layer} ${BLOBS_PATH}/${layer:0:2}/${layer}/data
 done
 done
}

convert_images

install.sh

使用这个脚本将 Registry 存储中的镜像转换成 skopeo dir 的方式，然后再将镜像同步到 Registry 中

#!/bin/bash
REGISTRY_DOMAIN="harbor.k8s.li"
REGISTRY_PATH="/var/lib/registry"

# 切换到 registry 存储主目录下
cd ${REGISTRY_PATH}

gen_skopeo_dir() {
   # 定义 registry 存储的 blob 目录 和 repositories 目录，方便后面使用
    BLOB_DIR="docker/registry/v2/blobs/sha256"
    REPO_DIR="docker/registry/v2/repositories"
    # 定义生成 skopeo 目录
    SKOPEO_DIR="docker/skopeo"
    # 通过 find 出 current 文件夹可以得到所有带 tag 的镜像，因为一个 tag 对应一个 current 目录
    for image in $(find ${REPO_DIR} -type d -name "current"); do
        # 根据镜像的 tag 提取镜像的名字
        name=$(echo ${image} | awk -F '/' '{print $5"/"$6":"$9}')
        link=$(cat ${image}/link | sed 's/sha256://')
        mfs="${BLOB_DIR}/${link:0:2}/${link}/data"
        # 创建镜像的硬链接需要的目录
        mkdir -p "${SKOPEO_DIR}/${name}"
        # 硬链接镜像的 manifests 文件到目录的 manifest 文件
        ln ${mfs} ${SKOPEO_DIR}/${name}/manifest.json
        # 使用正则匹配出所有的 sha256 值，然后排序去重
        layers=$(grep -Eo "\b[a-f0-9]{64}\b" ${mfs} | sort -n | uniq)
        for layer in ${layers}; do
          # 硬链接 registry 存储目录里的镜像 layer 和 images config 到镜像的 dir 目录
            ln ${BLOB_DIR}/${layer:0:2}/${layer}/data ${SKOPEO_DIR}/${name}/${layer}
        done
    done
}

sync_image() {
    # 使用 skopeo sync 将 dir 格式的镜像同步到 harbor
    for project in $(ls ${SKOPEO_DIR}); do
        skopeo sync --insecure-policy --src-tls-verify=false --dest-tls-verify=false \
        --src dir --dest docker ${SKOPEO_DIR}/${project} ${REGISTRY_DOMAIN}/${project}
    done
}

gen_skopeo_dir
sync_image

3.6.3 从 Registry 存储中 select 出镜像

先将镜像同步到一个固定的 Registry 中

$ bash sync.sh docker.io localhost:5000

再使用该脚本将镜像从 Registry 存储中捞出来：

#!/bin/bash
set -eo pipefail

IMAGES_LIST="$1"
REGISTRY_PATH="$2"
OUTPUT_DIR="$3"
BLOB_DIR="docker/registry/v2/blobs/sha256"
REPO_DIR="docker/registry/v2/repositories"

rm -rf ${OUTPUT_DIR}; mkdir -p ${OUTPUT_DIR}
for image in $(find ${IMAGES_LIST} -type f -name "*.list" | xargs grep -Ev '^#|^/' | grep ':'); do
    image_tag=${image##*:}
    image_name=${image%%:*}
    tag_link=${REGISTRY_PATH}/${REPO_DIR}/${image_name}/_manifests/tags/${image_tag}/current/link
    manifest_sha256=$(sed 's/sha256://' ${tag_link})
    manifest=${REGISTRY_PATH}/${BLOB_DIR}/${manifest_sha256:0:2}/${manifest_sha256}/data
    mkdir -p ${OUTPUT_DIR}/${BLOB_DIR}/${manifest_sha256:0:2}/${manifest_sha256}
    ln -f ${manifest} ${OUTPUT_DIR}/${BLOB_DIR}/${manifest_sha256:0:2}/${manifest_sha256}/data

    # make image repositories dir
    mkdir -p ${OUTPUT_DIR}/${REPO_DIR}/${image_name}/{_uploads,_layers,_manifests}
    mkdir -p ${OUTPUT_DIR}/${REPO_DIR}/${image_name}/_manifests/revisions/sha256/${manifest_sha256}
    mkdir -p ${OUTPUT_DIR}/${REPO_DIR}/${image_name}/_manifests/tags/${image_tag}/{current,index/sha256}
    mkdir -p ${OUTPUT_DIR}/${REPO_DIR}/${image_name}/_manifests/tags/${image_tag}/index/sha256/${manifest_sha256}

    # create image tag manifest link file
    echo -n "sha256:${manifest_sha256}" > ${OUTPUT_DIR}/${REPO_DIR}/${image_name}/_manifests/tags/${image_tag}/current/link
    echo -n "sha256:${manifest_sha256}" > ${OUTPUT_DIR}/${REPO_DIR}/${image_name}/_manifests/revisions/sha256/${manifest_sha256}/link
    echo -n "sha256:${manifest_sha256}" > ${OUTPUT_DIR}/${REPO_DIR}/${image_name}/_manifests/tags/${image_tag}/index/sha256/${manifest_sha256}/link
    for layer in $(sed '/v1Compatibility/d' ${manifest} | grep -Eo '\b[a-f0-9]{64}\b' | sort -u); do
        mkdir -p ${OUTPUT_DIR}/${BLOB_DIR}/${layer:0:2}/${layer}
        mkdir -p ${OUTPUT_DIR}/${REPO_DIR}/${image_name}/_layers/sha256/${layer}
        ln -f ${BLOB_DIR}/${layer:0:2}/${layer}/data ${OUTPUT_DIR}/${BLOB_DIR}/${layer:0:2}/${layer}/data
        echo -n "sha256:${layer}" > ${OUTPUT_DIR}/${REPO_DIR}/${image_name}/_layers/sha256/${layer}/link
    done
done

4.参考链接：

https://cloud.tencent.com/developer/article/1851464

https://bbs.huaweicloud.com/blogs/344975