Heat Software Config
This article describes Software Configuration Hooks new in Icehouse.
All the following topics depend on https://github.com/openstack/heat-templates/tree/master/hot/software-config.
Pre-requisites
Images with cloud-init, heat-config, os-collect-config, os-refresh-config, os-apply-config, heat-config-cfn-init, heat-config-puppet, heat-config-salt and heat-config-script packages installed
Nova metadata server up and running
Heat CFN API service up and running
How hook works
1. When the instance boots up, it runs the cloud-init script. Cloud-init creates the directories under /var/lib/cloud:
root@web-server:/var/lib/cloud# ls
data handlers instance instances scripts seed sem
2. Download the data from the metadata server
root@web-server:/var/lib/cloud/instance# ll
-rw-r--r-- 1 root root 52 Aug 7 08:21 boot-finished
drwxr-xr-x 2 root root 4096 Aug 7 08:16 boothooks/
-rw------- 1 root root 128 Aug 7 08:16 cloud-config.txt
-rw-r--r-- 1 root root 29 Aug 7 08:16 datasource
drwxr-xr-x 2 root root 4096 Aug 7 08:16 handlers/
-r-------- 1 root root 10500 Aug 7 08:16 obj.pkl
drwxr-xr-x 2 root root 4096 Aug 7 08:16 scripts/
drwxr-xr-x 2 root root 4096 Aug 7 08:21 sem/
-rw------- 1 root root 3894 Aug 7 08:16 user-data.txt
-rw------- 1 root root 3954 Aug 7 08:16 user-data.txt.i
-rw------- 1 root root 4 Aug 7 08:16 vendor-data.txt
-rw------- 1 root root 345 Aug 7 08:16 vendor-data.txt.i
3. Splits the multiple parts into the /var/lib/cloud/instance directory
3.1 Content-Type: text/part-handler & Content-Type: text/x-cfninitdata
Then content of this part-handers will be written to /var/lib/cloud/instance/handlers. The “part” handled here is subpart of user data.
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="part-handler.py"
#part-handler
def list_types():
return(["text/x-cfninitdata"])
def handle_part(data, ctype, filename, payload):
if ctype == "__begin__":
try:
os.makedirs('/var/lib/heat-cfntools', 0o700)
except OSError as e:
if e.errno != errno.EEXIST:
raise
return
if ctype == "__end__":
return
with open('/var/log/part-handler.log', 'a') as log:
timestamp = datetime.datetime.now()
log.write('%s filename:%s, ctype:%s\n' % (timestamp, filename, ctype))
if ctype == 'text/x-cfninitdata':
with open('/var/lib/heat-cfntools/%s' % filename, 'w') as f:
f.write(payload)
# TODO(sdake) hopefully temporary until users move to heat-cfntools-1.3
with open('/var/lib/cloud/data/%s' % filename, 'w') as f:
f.write(payload)
For example, the following code process "text/x-cfninitdata" sections by writing them to /var/lib/heat-cfntools/ and /var/lib/cloud/data/. The payload is like this:
{"deployments": [], "os-collect-config": {"cfn": {"stack_name": "stackname", "metadata_url": "http://heat-server:8000/v1/", "path": "web_server.Metadata", "secret_access_key": "ba455502219c41a58757a62c5425d7ce", "access_key_id": "8776148cc65a4f95b3c4a0b390f87625"}}}When part-handler executes , the directory looks like:
root@web-server:/var/lib/heat-cfntools# ls
cfn-init-data cfn-watch-server
4. Execute user's script
4.1 The first loop of os-collect-config
os-collect-config is a daemon process. It collects data from defined configuration sources and runs a defined hook whenever the metadata has been changed.
root@web-server:~# ps aux | grep os-collect-config
root 1229 2.5 1.2 99192 25816 ? Ss Aug07 33:39 /opt/stack/venvs/os-collect-config/bin/python /usr/local/bin/os-collect-config
Firstly, the default configuration of /etc/ os-collect-config.confjust contains default command options, like this:
[DEFAULT]
command = os-refresh-configAfterone loop of os-collect-config execution, the directory of/var/lib/os-collect-config and /var/run/os-collect-config are like this:
root@web-server:/var/run/os-collect-config# ll
-rw------- 1 root root 111667 Aug 7 08:21 cfn.json
-rw------- 1 root root 111667 Aug 7 08:21 cfn.json.orig
-rw------- 1 root root 1061 Aug 7 08:21 ec2.json
-rw------- 1 root root 1061 Aug 7 08:20 ec2.json.last
-rw------- 1 root root 1061 Aug 7 08:19 ec2.json.orig
-rw------- 1 root root 300 Aug 7 08:21 heat_local.json
-rw------- 1 root root 300 Aug 7 08:20 heat_local.json.last
-rw------- 1 root root 300 Aug 7 08:19 heat_local.json.orig
-rw------- 1 root root 124 Aug 7 08:21 os_config_files.json
The content of /etc/heat_local.conf sources from/var/lib/heat-cfntools/cfn-init-data by heat_local::Collector. And ec2.json sources from metadata service by ec2::Collector.
But now, there are no cfn.json and cfn.json.orig because/etc/os-collect-config.conf is not configured fully yet.
Secondly, when os-refresh-config is called, the scripts of PHASES('pre-configure', 'configure', 'post-configure','migration') under /opt/stack/os-config-refresh will be executed. “os-apply-config”is among these scripts.
Thirdly, os-apply-config will create/etc/os-collect-collect.conf and /var/run/heat-config/heat-config by rendering templates predefined under /usr/libexec/os-apply-config/templates. But now/var/run/heat-config/heat-config is still empty.
root@web-server:~# cat /etc/os-collect-config.conf
[DEFAULT]
command = os-refresh-config
[cfn]
metadata_url = http://heat-server:8000/v1/
stack_name = stack43
secret_access_key = ba455502219c41a58757a62c5425d7ce
access_key_id = 8776148cc65a4f95b3c4a0b390f87625
path = web_server.Metadata
4.2 The second loop of os-collect-config
Now, cfn.json and cfn.json.orig will be created by cfn::Collector in “os-collect-config” because now /etc/os-collect-config is configured correctly.
root@web-server:/var/run/os-collect-config# ll
-rw------- 1 root root 111667 Aug 7 08:21 cfn.json
-rw------- 1 root root 111667 Aug 7 08:21 cfn.json.orig
And /var/run/heat-config/heat-config will be filled with meaningful contents by “os-apply-config”.
root@web-server:/var/run/heat-config# ll
-rw-r--r-- 1 root root 110761 Aug 8 09:09 heat-config
Now /opt/stack/os-config-refresh/configure.d/55-heat-config run, not only it will invoke proper hook under /var/lib/heat-config/hooks, but also it will signal heat with output.
Troubleshooting
/var/log/cloud-init.log
/var/log/cloud-init-output.log
/var/log/part-handler.log
/var/log/upstart/os-collect-config.log
/var/log/os-apply-config.log
References
https://help.ubuntu.com/community/CloudInit
http://cloudinit.readthedocs.org/en/latest/topics/examples.html
http://foss-boss.blogspot.com/2011/01/advanced-cloud-init-custom-handlers.html
https://github.com/openstack/heat-templates/tree/master/hot/software-config
Appendix
1. A Limit of Cloud-init
The first limit is osapi_max_request_body_size of nova-api service, and its default value is 114688. Define it in nova/api/sizelimit.py as following:
#default request size is 112k
max_request_body_size_opt = cfg.IntOpt('osapi_max_request_body_size',
default=114688,
help='The maximum body size '
'per each osapi request(bytes)')
The second limit is MAX_USERDATA_SIZE, and its default value is 65535. Define it in nova/compute/api.py:
MAX_USERDATA_SIZE = 65535If your user-data size is larger than MAX_USERDATA_SIZE, it will complain directly like this “Instance User Data Too Large: User data too large. User data must be no larger than 65535 bytes once base64 encoded.”
2. One Template Sample
heat_template_version: 2013-05-23
description: Web Server Template
parameters:
image:
default: CentOS
type: string
description: Image use to boot a server
flavor:
default: m1.small
type: string
description: Flavor use to boot a server
key_name:
default: demo-key
type: string
description: keypair name use to boot a server
resources:
web_config:
type: OS::Heat::SoftwareConfig
properties:
group: script
config:
get_file: scripts/web.sh
web_deployment:
type: OS::Heat::SoftwareDeployment
properties:
config:
get_resource: web_config
server:
get_resource: web_server
web_server:
type: OS::Nova::Server
properties:
name: "Web Server"
image: { get_param: image }
flavor: { get_param: flavor }
key_name: { get_param: key_name }
user_data_format: SOFTWARE_CONFIG
outputs:
stdout:
value:
get_attr: [web_deployment, deploy_stdout]
stderr:
value:
get_attr: [web_deployment, deploy_stderr]
status_code:
value:
get_attr: [web_deployment, deploy_status_code]