OpenStack-Queens详细安装部署(十三)Nova集成Ceph

  • A+
所属分类:OpenStack  虚拟化/私有云

一、Nova集成Ceph

1. 配置ceph.conf

# 如果需要从ceph rbd中启动虚拟机,必须将ceph配置为nova的临时后端;
# 推荐在计算节点的配置文件中启用rbd cache功能;
# 为了便于故障排查,配置admin socket参数,这样每个使用ceph rbd的虚拟机都有1个socket将有利于虚拟机性能分析与故障解决;
# 相关配置只涉及全部计算节点ceph.conf文件的[client]与[client.cinder]字段,以compute01节点为例
[root@DT_Node-172_17_7_1 ~]# cd /data/ceph/deploy/
[root@DT_Node-172_17_7_1 /data/ceph/deploy]# echo '
[client]
rbd cache = true
rbd cache writethrough until flush = true
admin socket = /var/run/ceph/guests/$cluster-$type.$id.$pid.$cctid.asok
log file = /var/log/qemu/qemu-guest-$pid.log
rbd concurrent management ops = 20' >> /data/ceph/deploy/ceph.conf
[root@DT_Node-172_17_7_1 /data/ceph/deploy]# ceph-deploy --overwrite-conf admin compute1 compute2 compute3 compute4

# 创建ceph.conf文件中指定的socker与log相关的目录,并更改属主
[root@DT_Node-172_17_7_1 ~]# for i in compute1 compute2 compute3 compute4;do ssh -t root@$i 'mkdir -p /var/run/ceph/guests/ /var/log/qemu/'; done
[root@DT_Node-172_17_7_1 ~]# for i in compute1 compute2 compute3 compute4;do ssh -t root@$i 'chown qemu:libvirt /var/run/ceph/guests/ /var/log/qemu/'; done

OpenStack-Queens详细安装部署(十三)Nova集成Ceph
OpenStack-Queens详细安装部署(十三)Nova集成Ceph

2. 配置nova.conf

# 在全部计算节点配置nova后端使用ceph集群的vms池,以compute01节点为例
[root@compute01 ~]# vim /etc/nova/nova.conf
[libvirt]
virt_type=kvm
cpu_mode=host-model
#vif_driver=nova.virt.libvirt.vif.LibvirtGenericVIFDriver
hw_machine_type = x86_64=pc-i440fx-rhel7.2.0

images_type = rbd
images_rbd_pool = nova-ephemeral
images_rbd_ceph_conf = /etc/ceph/ceph.conf
rbd_user = nova-ephemeral
# uuid前后一致
rbd_secret_uuid = 92f87f28-cf15-4866-b5b6-5217e39d791e
disk_cachemodes="network=writeback"
live_migration_flag="VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_LIVE,VIR_MIGRATE_PERSIST_DEST,VIR_MIGRATE_TUNNELLED"
# 禁用文件注入
inject_password = false
inject_key = false
inject_partition = -2
# 虚拟机临时root磁盘discard功能,”unmap”参数在scsi接口类型磁盘释放后可立即释放空间
hw_disk_discard = unmap

# 变更配置文件,重启计算服务
[root@DT_Node-172_17_7_4 ~]# systemctl restart libvirtd.service openstack-nova-compute.service
[root@DT_Node-172_17_7_4 ~]# systemctl status libvirtd.service openstack-nova-compute.service 

OpenStack-Queens详细安装部署(十三)Nova集成Ceph

3. 配置live-migration
1)修改/etc/libvirt/libvirtd.conf

# 在全部计算节点操作,以compute01节点为例;
# 以下给出libvirtd.conf文件的修改处所在的行num
[root@DT_Node-172_17_7_4 ~]# egrep -vn "^$|^#" /etc/libvirt/libvirtd.conf 
# 取消以下三行的注释
22:listen_tls = 0
33:listen_tcp = 1
45:tcp_port = "16509"
# 取消注释,并修改监听端口
55:listen_addr = "172.17.7.4"
# 取消注释,同时取消认证
158:auth_tcp = "none"

2)修改/etc/sysconfig/libvirtd

# 在全部计算节点操作,以compute01节点为例;
# 以下给出libvirtd文件的修改处所在的行num
[root@DT_Node-172_17_7_4 ~]# egrep -vn "^$|^#" /etc/sysconfig/libvirtd
# 取消注释
9:LIBVIRTD_ARGS="--listen" 

4)重启服务

# libvirtd与nova-compute服务都需要重启
[root@DT_Node-172_17_7_4 ~]# systemctl restart libvirtd.service openstack-nova-compute.service
# 查看服务
[root@DT_Node-172_17_7_4 ~]# systemctl status libvirtd.service openstack-nova-compute.service       
[root@DT_Node-172_17_7_4 ~]# netstat -tunlp | grep 16509 
<a href="https://www.dwhd.org/wp-content/uploads/2018/09/2018-09-22_20-02-53.png"><img src="https://www.dwhd.org/wp-content/uploads/2018/09/2018-09-22_20-02-53.png" alt="" width="2794" height="1022" class="alignnone size-full wp-image-5990" /></a>

4. 验证
如果使用ceph提供的volume做启动盘,即虚拟机运行镜像文件存放在共享存储上,此时可以方便地进行live-migration。

1)创建基于ceph存储的bootable存储卷
# 当nova从rbd启动instance时,镜像格式必须是raw格式,否则虚拟机在启动时glance-api与cinder均会报错;
# 首先进行格式转换,将*.img文件转换为*.raw文件
[root@DT_Node-172_17_7_1 ~]# ls
anaconda-ks.cfg  cirros-0.4.0-x86_64-disk.img  galera-25.3.22-1.rhel7.el7.centos.x86_64.rpm  keystone_admin  keystone_demo
[root@DT_Node-172_17_7_1 ~]# . keystone_admin 
[openstack-admin]-[root@DT_Node-172_17_7_1 ~]#  qemu-img convert -f qcow2 -O raw ~/cirros-0.4.0-x86_64-disk.img ~/cirros-0.4.0-x86_64-disk.raw
[openstack-admin]-[root@DT_Node-172_17_7_1 ~]# ls
anaconda-ks.cfg  cirros-0.4.0-x86_64-disk.img  cirros-0.4.0-x86_64-disk.raw  galera-25.3.22-1.rhel7.el7.centos.x86_64.rpm  keystone_admin  keystone_demo
[openstack-admin]-[root@DT_Node-172_17_7_1 ~]#

# 生成raw格式镜像
[openstack-admin]-[root@DT_Node-172_17_7_1 ~]# openstack image create --disk-format raw --container-format bare --public --file ~/cirros-0.4.0-x86_64-disk.raw "cirros-0.4.0-ceph-raw"

OpenStack-Queens详细安装部署(十三)Nova集成Ceph

# 使用新镜像创建bootable卷
[openstack-admin]-[root@DT_Node-172_17_7_1 ~]# cinder create --image-id ed98895d-5832-4e6a-9035-13b64c5d0edb --volume-type ceph --name ceph-bootable1 2
# 查看新创建的bootable卷
[openstack-admin]-[root@DT_Node-172_17_7_1 ~]# cinder list 

OpenStack-Queens详细安装部署(十三)Nova集成Ceph

# 从基于ceph后端的volumes新建实例;
# “--boot-volume”指定具有”bootable”属性的卷,启动后,虚拟机运行在volumes卷
[openstack-admin]-[root@DT_Node-172_17_7_1 ~]# openstack keypair list
[openstack-admin]-[root@DT_Node-172_17_7_1 ~]# openstack flavor list
[openstack-admin]-[root@DT_Node-172_17_7_1 ~]# openstack network list 
[openstack-admin]-[root@DT_Node-172_17_7_1 ~]# cinder list
[openstack-admin]-[root@DT_Node-172_17_7_1 ~]# nova boot --flavor m1.tiny --boot-volume 617c3eac-53ab-4c33-87ee-f69b491e8535 --nic net-id=ab3b0202-0e60-4f6c-b313-7f9992ad78ee --security-group default --key-name lookback cirros-cephvolumes-instance1

OpenStack-Queens详细安装部署(十三)Nova集成Ceph

2)从ceph rbd启动虚拟机

# --nic:net-id指网络id,非subnet-id;
# 最后“cirros-cephrbd-instance1”为instance名
[openstack-admin]-[root@DT_Node-172_17_7_1 ~]# nova boot --flavor m1.tiny --image cirros-0.4.0-ceph-raw --nic net-id=ab3b0202-0e60-4f6c-b313-7f9992ad78ee --security-group default --key-name lookback cirros-cephrbd-instance1
# 查询生成的instance
[openstack-admin]-[root@DT_Node-172_17_7_1 ~]# nova list
lookback

发表评论

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen: