ChartMuseum is an open-source Helm Chart Repository server written in Go (Golang), with support for cloud storage backends, including Google Cloud Storage, Amazon S3, Microsoft Azure Blob Storage, Alibaba Cloud OSS Storage, Openstack Object Storage, Oracle Cloud Infrastructure Object Storage, Baidu Cloud BOS Storage, Tencent Cloud Object Storage, Netease Cloud NOS Storage, DigitalOcean Spaces, Minio, and etcd.
Works as a valid Helm Chart Repository, and also provides an API for uploading charts.
Powered by some great Go technology:
GET /index.yaml
- retrieved when you run helm repo add chartmuseum http://localhost:8080/
GET /charts/mychart-0.1.0.tgz
- retrieved when you run helm install chartmuseum/mychart
GET /charts/mychart-0.1.0.tgz.prov
- retrieved when you run helm install
with the --verify
flagPOST /api/charts
- upload a new chart versionPOST /api/prov
- upload a new provenance fileDELETE /api/charts/<name>/<version>
- delete a chart version (and corresponding provenance file)GET /api/charts
- list all chartsGET /api/charts/<name>
- list all versions of a chartGET /api/charts/<name>/<version>
- describe a chart versionHEAD /api/charts/<name>
- check if chart exists (any versions)HEAD /api/charts/<name>/<version>
- check if chart version existsGET /
- HTML welcome pageGET /info
- returns current ChartMuseum versionGET /health
- returns 200 OKFollow "How to Run" section below to get ChartMuseum up and running at http://localhost:8080
First create mychart-0.1.0.tgz
using the Helm CLI:
cd mychart/
helm package .
Upload mychart-0.1.0.tgz
:
curl --data-binary "@mychart-0.1.0.tgz" http://localhost:8080/api/charts
If you've signed your package and generated a provenance file, upload it with:
curl --data-binary "@mychart-0.1.0.tgz.prov" http://localhost:8080/api/prov
Both files can also be uploaded at once (or one at a time) on the /api/charts
route using the multipart/form-data
format:
curl -F "chart=@mychart-0.1.0.tgz" -F "prov=@mychart-0.1.0.tgz.prov" http://localhost:8080/api/charts
You can also use the helm-push plugin:
helm push mychart/ chartmuseum
Add the URL to your ChartMuseum installation to the local repository list:
helm repo add chartmuseum http://localhost:8080
Search for charts:
helm search repo chartmuseum/
Install chart:
helm install chartmuseum/mychart --generate-name
Install binary using GoFish:
gofish install chartmuseum
==> Installing chartmuseum...
�� chartmuseum 0.13.1: installed in 95.431145ms
or you can use the installer script:
curl https://raw.githubusercontent.com/helm/chartmuseum/main/scripts/get-chartmuseum | bash
or download manually from the releases page,which also contains all package checksums and signatures.
Determine your version with chartmuseum --version
.
Show all CLI options with chartmuseum --help
. Common configurations can be seen below.
All command-line options can be specified as environment variables, which are defined by the command-line option, capitalized, with all -
's replaced with _
's.
For example, the env var STORAGE_AMAZON_BUCKET
can be used in place of --storage-amazon-bucket
.
When using a yaml file instead of the command line, Convert the -
of the parameter in the command to .
.Note that --storage
is converted to storage.backend
debug: true
port: 8080
storage.backend: local
storage.local.rootdir: <storage_path>
bearerauth: 1
authrealm: <authorization server url>
authservice: <authorization server service name>
authcertpath: <path to authorization server public pem file>
depth: 2
Make sure your environment is properly setup to access my-s3-bucket
For Amazon S3, endpoint
is automatically inferred.
chartmuseum --debug --port=8080 \
--storage="amazon" \
--storage-amazon-bucket="my-s3-bucket" \
--storage-amazon-prefix="" \
--storage-amazon-region="us-east-1"
For S3 compatible services like Minio, set the credentials using environment variables and pass the endpoint
.
export AWS_ACCESS_KEY_ID=""
export AWS_SECRET_ACCESS_KEY=""
chartmuseum --debug --port=8080 \
--storage="amazon" \
--storage-amazon-bucket="my-s3-bucket" \
--storage-amazon-prefix="" \
--storage-amazon-region="us-east-1" \
--storage-amazon-endpoint="my-s3-compatible-service-endpoint"
You need at least the following permissions inside your IAM Policy
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowListObjects",
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": "arn:aws:s3:::my-s3-bucket"
},
{
"Sid": "AllowObjectsCRUD",
"Effect": "Allow",
"Action": [
"s3:DeleteObject",
"s3:GetObject",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::my-s3-bucket/*"
}
]
}
In order to work with AWS service accounts you may need to set AWS_SDK_LOAD_CONFIG=1
in your environment.For more context, please see here.
For DigitalOcean, set the credentials using environment variable and pass the endpoint
.
Note below, that the region us-east-1
needs to be set, since that is how the DigitalOcean cli implementation functions. The actual region of your spaces location is defined by the endpoint. Below we are using Frankfurt as an example.
export AWS_ACCESS_KEY_ID="spaces_access_key"
export AWS_SECRET_ACCESS_KEY="spaces_secret_key"
chartmuseum --debug --port=8080 \
--storage="amazon" \
--storage-amazon-bucket="my_spaces_name" \
--storage-amazon-prefix="my_spaces_name_subfolder" \
--storage-amazon-region="us-east-1" \
--storage-amazon-endpoint="https://fra1.digitaloceanspaces.com"
The access_key and secret_key can be generated from the DigitalOcean console, under the section API/Spaces_access_keys.
Note: on certain S3-based storage backends, the LastModified
field on objectsis truncated to the nearest second. For more info, please see issue #152.
In order to mitigate this, you may use use the --storage-timestamp-tolerance
option.For example, to round to the nearest second, you could use --storage-timestamp-tolerance=1s
.For acceptable values to use for this field, please see here.
Make sure your environment is properly setup to access my-gcs-bucket
.
One way to do so is to set the GOOGLE_APPLICATION_CREDENTIALS
var in your environment, pointing to the JSON file containing your service account key:
export GOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/[FILE_NAME].json"
More info on Google Cloud authentication can be found here.
chartmuseum --debug --port=8080 \
--storage="google" \
--storage-google-bucket="my-gcs-bucket" \
--storage-google-prefix=""
Make sure your environment is properly setup to access mycontainer
.
To do so, you must set the following env vars:
AZURE_STORAGE_ACCOUNT
AZURE_STORAGE_ACCESS_KEY
chartmuseum --debug --port=8080 \
--storage="microsoft" \
--storage-microsoft-container="mycontainer" \
--storage-microsoft-prefix=""
Make sure your environment is properly setup to access my-oss-bucket
.
To do so, you must set the following env vars:
ALIBABA_CLOUD_ACCESS_KEY_ID
ALIBABA_CLOUD_ACCESS_KEY_SECRET
chartmuseum --debug --port=8080 \
--storage="alibaba" \
--storage-alibaba-bucket="my-oss-bucket" \
--storage-alibaba-prefix="" \
--storage-alibaba-endpoint="oss-cn-beijing.aliyuncs.com"
Make sure your environment is properly setup to access mycontainer
.
To do so, you must set the following env vars (depending on your openstack version):
OS_AUTH_URL
OS_PROJECT_NAME
or OS_TENANT_NAME
or OS_PROJECT_ID
or OS_TENANT_ID
OS_DOMAIN_NAME
or OS_DOMAIN_ID
OS_USERNAME
or OS_USERID
OS_PASSWORD
chartmuseum --debug --port=8080 \
--storage="openstack" \
--storage-openstack-container="mycontainer" \
--storage-openstack-prefix="" \
--storage-openstack-region="myregion"
For Swift V1 Auth you must set the following env vars:
ST_AUTH
ST_USER
ST_KEY
chartmuseum --debug --port=8080 \
--storage="openstack" \
--storage-openstack-auth="v1" \
--storage-openstack-container="mycontainer" \
--storage-openstack-prefix=""
Make sure your environment is properly setup to access my-ocs-bucket
.
More info on Oracle Cloud Infrastructure authentication can be found here.
chartmuseum --debug --port=8080 \
--storage="oracle" \
--storage-oracle-bucket="my-ocs-bucket" \
--storage-oracle-prefix="" \
--storage-oracle-compartmentid="ocid1.compartment.oc1..1234"
Make sure your environment is properly setup to access my-bos-bucket
.
To do so, you must set the following env vars:
BAIDU_CLOUD_ACCESS_KEY_ID
BAIDU_CLOUD_ACCESS_KEY_SECRET
chartmuseum --debug --port=8080 \
--storage="baidu" \
--storage-baidu-bucket="my-bos-bucket" \
--storage-baidu-prefix="" \
--storage-baidu-endpoint="bj.bcebos.com"
Make sure your environment is properly setup to access my-cos-bucket
.
To do so, you must set the following env vars:
TENCENT_CLOUD_COS_SECRET_ID
TENCENT_CLOUD_COS_SECRET_KEY
chartmuseum --debug --port=8080 \
--storage="tencent" \
--storage-tencent-bucket="my-cos-bucket" \
--storage-tencent-prefix="" \
--storage-tencent-endpoint="cos.ap-beijing.myqcloud.com"
Make sure your environment is properly setup to access my-nos-bucket
.
To do so, you must set the following env vars:
NETEASE_CLOUD_ACCESS_KEY_ID
NETEASE_CLOUD_ACCESS_KEY_SECRET
chartmuseum --debug --port=8080 \
--storage="netease" \
--storage-netease-bucket="my-nos-bucket" \
--storage-netease-prefix="" \
--storage-netease-endpoint="nos-eastchina1.126.net"
To use etcd as backend you need the CA certificate and the signed key pair.See here
chartmuseum --debug --port=8080 \
--storage="etcd" \
--storage-etcd-cafile="/path/to/ca.crt" \
--storage-etcd-certfile="/path/to/server.crt" \
--storage-etcd-keyfile="/path/to/server.key" \
--storage-etcd-prefix="" \
--storage-etcd-endpoint="http://localhost:2379"
Make sure you have read-write access to ./chartstorage
(will create if doesn't exist on first upload)
chartmuseum --debug --port=8080 \
--storage="local" \
--storage-local-rootdir="./chartstorage"
If both of the following options are provided, basic http authentication will protect all routes:
--basic-auth-user=<user>
- username for basic http authentication--basic-auth-pass=<pass>
- password for basic http authenticationYou may want basic auth to only be applied to operations that can change Charts, i.e. PUT, POST and DELETE. So to avoid basic auth on GET operations use
--auth-anonymous-get
- allow anonymous GET operationsIf all of the following options are provided, bearer auth will protect all routes:
--bearer-auth
- enables bearer auth--auth-realm=<realm>
- authorization server url--auth-service=<service>
- authorization server service name--auth-cert-path=<path>
- path to authorization server public pem fileUsing options above, ChartMuseum is configured with a public key, and will accept RS256 JWT tokens signed by the associated private key, passed in the Authorization
header. You can use the chartmuseum/auth Go library to generate valid JWT tokens.
In order to gain access to a specific resource, the JWT token must contain an access
section in the claims. This section indicates which resources the user is able to access. Here is an example token payload:
{
"exp": 1543995770,
"iat": 1543995470,
"access": [
{
"type": "artifact-repository",
"name": "org1/repo1",
"actions": [
"pull"
]
}
]
}
The type
is always "artifact-repository", the name
is the namespace/tenant (just use the string "repo" if using single-tenant server), and actions
is an array of actions the user can perform ("pull" and/or "push).
For more information about how this works, please see chartmuseum/auth-server-example.
If both of the following options are provided, the server will listen and serve HTTPS:
--tls-cert=<crt>
- path to tls certificate chain file--tls-key=<key>
- path to tls key fileIf the above HTTPS values are provided in addition to below, the server will listen and serve HTTPS and authenticate client requests against the CA certificate:
--tls-ca-cert=<cacert>
- path to tls certificate fileYou can specify the --gen-index
option if you only wish to use ChartMuseum to generate your index.yaml file. Note that this will only work with --depth=0
.
The contents of index.yaml will be printed to stdout and the program will exit. This is useful if you are satisfied with your current Helm CI/CD process and/or don't want to monitor another webservice.
--log-json
- output structured logs as json--log-health
- log incoming /health requests--log-latency-integer
- log latency as an integer (nanoseconds) instead of a string--disable-api
- disable all routes prefixed with /api--disable-delete
- explicitly disable the delete chart route--disable-statefiles
- disable use of index-cache.yaml--allow-overwrite
- allow chart versions to be re-uploaded without ?force querystring--disable-force-overwrite
- do not allow chart versions to be re-uploaded, even with ?force querystring--chart-url=<url>
- absolute url for .tgzs in index.yaml--storage-amazon-endpoint=<endpoint>
- alternative s3 endpoint--storage-amazon-sse=<algorithm>
- s3 server side encryption algorithm--storage-openstack-cacert=<path>
- path to a custom ca certificates bundle for openstack--chart-post-form-field-name=<field>
- form field which will be queried for the chart file content--prov-post-form-field-name=<field>
- form field which will be queried for the provenance file content--index-limit=<number>
- limit the number of parallel indexers--context-path=<path>
- base context path (new root for application routes)--depth=<number>
- levels of nested repos for multitenancy--cors-alloworigin=<value>
- value to set in the Access-Control-Allow-Origin HTTP header--read-timeout=<number>
- socket read timeout for http server--write-timeout=<number>
- socker write timeout for http serverAvailable via GitHub Container Registry (GHCR).
Example usage (local storage):
docker run --rm -it \
-p 8080:8080 \
-e DEBUG=1 \
-e STORAGE=local \
-e STORAGE_LOCAL_ROOTDIR=/charts \
-v $(pwd)/charts:/charts \
ghcr.io/helm/chartmuseum:v0.13.1
Example usage (S3):
docker run --rm -it \
-p 8080:8080 \
-e DEBUG=1 \
-e STORAGE="amazon" \
-e STORAGE_AMAZON_BUCKET="my-s3-bucket" \
-e STORAGE_AMAZON_PREFIX="" \
-e STORAGE_AMAZON_REGION="us-east-1" \
-v ~/.aws:/home/chartmuseum/.aws:ro \
ghcr.io/helm/chartmuseum:v0.13.1
There is a Helm chart for ChartMuseum itself.
You can also view it on Artifact Hub.
To install:
helm repo add chartmuseum https://chartmuseum.github.io/charts
helm install chartmuseum/chartmuseum
If interested in making changes, please submit a PR to chartmuseum/charts. Before doing any work, please check for any currently open pull requests. Thanks!
Multitenancy is supported with the --depth
flag.
To begin, start with a directory structure such as
charts
├── org1
│ ├── repoa
│ │ └── nginx-ingress-0.9.3.tgz
├── org2
│ ├── repob
│ │ └── chartmuseum-0.4.0.tgz
This represents a storage layout appropriate for --depth=2
. The organization level can be eliminated by using --depth=1
. The default depth is 0 (singletenant server).
Start the server with --depth=2
, pointing to the charts/
directory:
chartmuseum --debug --depth=2 --storage="local" --storage-local-rootdir=./charts
This example will provide two separate Helm Chart Repositories at the following locations:
http://localhost:8080/org1/repoa
http://localhost:8080/org2/repob
This should work with all supported storage backends.
To use the chart manipulation routes, simply place the name of the repo directly after "/api" in the route:
curl -F "chart=@mychart-0.1.0.tgz" http://localhost:8080/api/org1/repoa/charts
You may also experiment with the --depth-dynamic
flag, which should allow for dynamic depth levels (i.e. all of /api/charts
, /api/myrepo/charts
, /api/org1/repoa/charts
).
For large chart repositories, you may wish to paginate the results from the GET /api/charts
route.
To do so, add the offset
and limit
query params to the request. For example, to retrieve a list of 5 charts total, skipping the first 5 charts, you could use the following:
GET /api/charts?offset=5&limit=5
By default, the contents of index.yaml
(per-tenant) will be stored in memory. This means that memory usage will continue to grow indefinitely as more charts are added to storage.
You may wish to offload this to an external cache store, especially for large, multitenant installations.
When dealing with thousands of charts, you may experience latency with the default settings. This is because upon each request, the storage backend is scanned for changes compared to the cache.
If you are ok with index.yaml
being out-of-date for a fixed period of time, you can improve performance by using the --cache-interval=<interval>
option.When this setting is enabled, the charts available for each tenant are refreshed on a timer.
For example, to only check storage every 5 minutes, you can use --cache-interval=5m
.
For valid values to use for this setting, please see here.
Example of using Redis as an external cache store:
chartmuseum --debug --port=8080 \
--storage="local" \
--storage-local-rootdir="./chartstorage" \
--cache="redis" \
--cache-redis-addr="localhost:6379" \
--cache-redis-password="" \
--cache-redis-db=0
ChartMuseum exposes its Prometheus metrics at the /metrics
route on the main port. This can be disabled with the --disable-metrics
command-line flag or the DISABLE_METRICS
environment variable.
Note that the Kubernetes chart currently disables metrics by default (
DISABLE_METRICS=true
is set in the chart).
Below are the current application metrics exposed. Note that there is a per tenant (repo) label. The repo label corresponds to the depth parameter, so a depth=2 as the example above wouldhave repo labels named org1/repoa
and org2/repob
.
Metric | Type | Labels | Description |
---|---|---|---|
chartmuseum_charts_served_total | Gauge | {repo="*"} | Total number of charts |
chartmuseum_chart_versions_served_total | Gauge | {repo="*"} | Total number of chart versions available |
*: see above for repo label
There are other general global metrics harvested (per process, hence for all tenants). You can get the complete list by using the /metrics
route.
Metric | Type | Labels | Description |
---|---|---|---|
chartmuseum_request_duration_seconds | Summary | {quantile="0.5"}, {quantile="0.9"}, {quantile="0.99"} | The HTTP request latencies in seconds |
chartmuseum_request_duration_seconds_sum | |||
chartmuseum_request_duration_seconds_count | |||
chartmuseum_request_size_bytes | Summary | {quantile="0.5"}, {quantile="0.9"}, {quantile="0.99"} | The HTTP request sizes in bytes |
chartmuseum_request_size_bytes_sum | |||
chartmuseum_request_size_bytes_count | |||
chartmuseum_response_size_bytes | Summary | {quantile="0.5"}, {quantile="0.9"}, {quantile="0.99"} | The HTTP response sizes in bytes |
chartmuseum_response_size_bytes_sum | |||
chartmuseum_response_size_bytes_count | |||
go_goroutines | Gauge | Number of goroutines that currently exist |
The repository index (index.yaml) is dynamically generated based on packages found in storage. If you store your own version of index.yaml, it will be completely ignored.
GET /index.yaml
occurs when you run helm repo add chartmuseum http://localhost:8080
or helm repo update
.
If you manually add/remove a .tgz package from storage, it will be immediately reflected in GET /index.yaml
.
You are no longer required to maintain your own version of index.yaml using helm repo index --merge
.
The --gen-index
CLI option (described above) can be used to generate and print index.yaml to stdout.
Upon index regeneration, ChartMuseum will, however, save a statefile in storage called index-cache.yaml
used for cache optimization. This file is only meant for internal use, but may be able to be used for migration to simple storage.
Please see scripts/mirror-k8s-repos.sh
for an example of how to download all .tgz packages from the official Kubernetes repositories (both stable and incubator).
You can then use ChartMuseum to serve up an internal mirror:
scripts/mirror-k8s-repos.sh
chartmuseum --debug --port=8080 --storage="local" --storage-local-rootdir="./mirror"
"Preserve your precious artifacts... in the cloud!"
The following subprojects are maintained by ChartMuseum:
You can reach the ChartMuseum community and developers in the Kubernetes Slack #chartmuseum channel.
一. 安装docker-compose curl -L https://github.com/docker/compose/releases/download/1.29.0/docker-compose-uname -s-uname -m -o /usr/local/bin/docker-compose 上条命令自动安装版本为1.29.0 的docker-compose 二. 安装chartmu
在 k8s 集群中创建 chartmuseum 命名空间,编写 yaml 文件在 chartmuseum 命名空间中 使用 chartmuseum:latest 镜像创建本地私有 chart 仓库,设置其仓库存储目录为宿主机的 /data/charts 目录。编写 service.yaml 文件,为 chart 私有仓库创建 Service 访问策略,定义其 为 ClusterIP 访问模式。编写
常规方法编译harbor,会丢失chartmuseum-photon镜像 1, 下载chartmuseum代码: git clone https://github.com/helm/chartmuseum.git 如果在arm64平台下编译,还需要修改代码,将Makefile中: build-linux: export GOARCH=amd64 改为 build-linux: export GOA
1、概述 helm使得在k8s集群里面部署应用变得更简单,就像在linux系统里面使用yum安装软件一样,helm主要是利用的chart,首先看一下chart的结构: # tree zipkin zipkin ├── charts ├── Chart.yaml ├── templates │ ├── deployment.yaml │ ├── _helpers.tpl │ ├── in
引 Helm 简介 Helm 是一个 Kubernetes 的包管理工具,用于简化 Kubernetes 应用程序的部署和管理。Helm 允许用户将应用程序打包成 Charts,这些 Charts 包含了要部署的 Kubernetes 资源的定义,例如 Deployment、Service 和 ConfigMap 等。通过 Helm,用户可以轻松地在 Kubernetes 群集中部署和管理应用程序
备注: 预备环境需要安装helm 1. 安装chartmuseum 参考 # on Linux curl -LO https://s3.amazonaws.com/chartmuseum/release/latest/bin/linux/amd64/chartmuseum # on macOS curl -LO https://s3.amazonaws.com/chartmuseum/
下载chartmuseum curl -LO https://s3.amazonaws.com/chartmuseum/release/latest/bin/windows/amd64/chartmuseum chmod +x ./chartmuseum 启动 (在下载的chartmuseum的目录下) ./chartmuseum --port=8999 –storage=“local” –sto
注:配置chartmuseum 可以从官方查看说文档 https://chartmuseum.com/docs/#prometheus-metrics 此次主要介绍二进制的部署配置方式: 第一步,下载chartmuseum二进制文件,操作如下(本文介绍 linux): # on Linux curl -LO https://s3.amazonaws.com/chartmuseum/release/
Artifact Hub 共有仓库 Helm部署: 二进制安装: 下载地址:https://github.com/helm/helm/releases 解压(tar -zxvf helm-v3.0.0-linux-amd64.tar.gz) 在解压目中找到helm程序,移动到需要的目录中(mv linux-amd64/helm /usr/local/bin/helm) Helm创建Chart
chartmuseum 安装 备注: 预备环境需要安装helm 1. 安装chartmuseum 参考 # on Linux curl -LO https://s3.amazonaws.com/chartmuseum/release/latest/bin/linux/amd64/chartmuseum # on macOS curl -LO https://s3.amazona