openstack-swift部署指南
实验拓扑图
控制节点
swift基本配置
在keystone上为swift创建用户名和密码
[root@controller ~(admin)]# openstack user create --password redhat swift
[root@controller ~(admin)]# openstack role add --user swift --project service admin
在keystone上为swift创建服务
[root@controller ~(admin)]# openstack service create --name swift object-store
在keystone上为swift创建endpoint
[root@controller ~(admin)]# openstack endpoint create --region RegionOne \
> object-store public http://controller:8080/v1/AUTH_%\(tenant_id\)s
[root@controller ~(admin)]# openstack endpoint create --region RegionOne \
> object-store internal http://controller:8080/v1/AUTH_%\(tenant_id\)s
[root@controller ~(admin)]# openstack endpoint create --region RegionOne \
> object-store admin http://controller:8080/v1
在控制节点上安装swift软件包
[root@controller ~(admin)]# yum install openstack-swift-proxy python-swiftclient \
> python-keystoneclient python-keystonemiddleware \
> memcached -y
已加载插件:fastestmirror, langpacks
Repository rdo-trunk-pike-tested is listed more than once in the configuration
Loading mirror speeds from cached hostfile
* epel: mirrors.sohu.com
....
作为依赖被安装:
liberasurecode.x86_64 0:1.5.0-1.el7
python-ceilometermiddleware.noarch 0:1.1.0-1.el7
python-dns.noarch 0:1.12.0-4.20150617git465785f.el7
python-pyeclib.x86_64 0:1.5.0-1.el7
python-swift.noarch 0:2.15.1-1.el7
完毕!
修改配置文件
修改/etc/swift/proxy-server.conf
在[DEFAULT]下添加
swift_dir = /etc/swift
修改[filter:authtoken]下面的内容为
[filter:authtoken]
paste.filter_factory = keystonemiddleware.auth_token:filter_factory
auth_uri = http://controller:5000
auth_url = http://controller:35357
memcached_servers = controller:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = swift
password = redhat
delay_auth_decision = True
signing_dir = /tmp/keystone-signing-swift
在[filter:authtoken]前面增加如下内容
[filter:keystoneauth]
use = egg:swift#keystoneauth
operator_roles = admin,user
存储节点
关闭防火墙、关闭selinux:
[root@swift-storageX ~]# setenforce 0
[root@swift-storageX ~]# sed -i '/SELINUX=/cSELINUX=disabled' /etc/sysconfig/selinux
[root@swift-storageX ~]#
[root@swift-storageX ~]# firewall-cmd --set-default-zone=trusted
success
[root@swift-storageX ~]#
设置/etc/hosts
[root@swift-storageX ~]# tail -2 /etc/hosts
192.168.26.201 controller.rhce.cc controller
192.168.26.202 swift-storage1.rhce.cc swift-storage1
192.168.26.203 swift-storage2.rhce.cc swift-storage2
[root@swift-storageX ~]#
安装必要的软件包
[root@swfit-storageX ~]# yum install rsync openstack-swift-account \
> openstack-swift-container openstack-swift-object -y
已加载插件:fastestmirror
base | 3.6 kB 00:00:00
extras | 3.4 kB 00:00:00
updates | 3.4 kB 00:00:00
Loading mirror speeds from cached hostfile
....
作为依赖被安装:
..
python2-six.noarch 0:1.10.0-9.el7
完毕!
准备需要的分区
[root@swfit-storageX ~]# pvcreate /dev/sdb
Physical volume "/dev/sdb" successfully created.
[root@swfit-storageX ~]# vgcreate vg0 /dev/sdb
Volume group "vg0" successfully created
[root@swfit-storageX ~]#
[root@swfit-storageX ~]# lvcreate -L 2G -n lv1 vg0
Logical volume "lv1" created.
[root@swfit-storageX ~]# lvcreate -L 2G -n lv2 vg0
Logical volume "lv2" created.
[root@swfit-storageX ~]#
[root@swfit-storageX ~]
[root@swfit-storageX ~]# mkfs.xfs /dev/vg0/lv1
[root@swfit-storageX ~]# mkfs.xfs /dev/vg0/lv2
[root@swift-storageX ~]# mkdir -p /srv/node/lv{1,2}
[root@swfit-storageX ~]# tree /srv/
/srv/
└── node
├── lv1
└── lv2
3 directories, 0 files
[root@swfit-storageX ~]#
修改/etc/fstab
[root@swfit-storageX ~]# tail -2 /etc/fstab
/dev/vg0/lv1 /srv/node/lv1 xfs defaults 0 0
/dev/vg0/lv2 /srv/node/lv2 xfs defaults 0 0
[root@swfit-storageX ~]#
[root@swfit-storageX ~]# mount -a
[root@swfit-storageX ~]# df -hT | tail -2
/dev/mapper/vg0-lv1 xfs 2.0G 33M 2.0G 2% /srv/node/lv1
/dev/mapper/vg0-lv2 xfs 2.0G 33M 2.0G 2% /srv/node/lv2
[root@swfit-storageX ~]#
修改权限
[root@swfit-storageX ~]# chown -R swift.swift /srv/node/
[root@swfit-storageX ~]
修改配置文件
vim /etc/rsyncd.conf,添加如下内容
uid = swift
gid = swift
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
address = 192.168.26.X #此处填写存储节点的IP
[account]
max connections = 2
path = /srv/node/
read only = False
lock file = /var/lock/account.lock
[container]
max connections = 2
path = /srv/node/
read only = False
lock file = /var/lock/container.lock
[object]
max connections = 2
path = /srv/node/
read only = False
lock file = /var/lock/object.lock
启动rsyncd并设置开机自动启动
[root@swfit-storageX ~]# systemctl start rsyncd.service
[root@swfit-storageX ~]# systemctl is-active rsyncd.service
active
[root@swfit-storageX ~]
[root@swfit-storageX ~]# systemctl enable rsyncd.service
Created symlink from /etc/systemd/system/multi-user.target.wants/rsyncd.service to /usr/lib/systemd/system/rsyncd.service.
[root@swfit-storageX ~]#
编辑/etc/swift/account-server.conf, 确保
[DEFAULT]后面内容如下:
bind_ip = 192.168.26.X
bind_port = 6202
user = swift
swift_dir = /etc/swift
devices = /srv/node
mount_check = True
[pipeline:main]后面内容如下:
pipeline = healthcheck recon account-serve
添加
[filter:recon]
use = egg:swift#recon
recon_cache_path = /var/cache/swift
[filter:healthcheck]
use = egg:swift#healthcheck
编辑/etc/swift/container-server.conf, 确保
[DEFAULT]后面的内容如下:
bind_ip = 192.168.26.X
bind_port = 6201
user = swift
swift_dir = /etc/swift
devices = /srv/node
mount_check = True
[pipeline:main]后面的内容如下:
pipeline = healthcheck recon container-server
增加
[filter:recon]
use = egg:swift#recon
recon_cache_path = /var/cache/swift
[filter:healthcheck]
use = egg:swift#healthcheck
编辑/etc/swift/object-server.conf,确保
[DEFAULT]后面的内容如下:
bind_ip = 192.168.26.X
bind_port = 6200
user = swift
swift_dir = /etc/swift
devices = /srv/node
mount_check = True
[pipeline:main]后面的内容如下:
pipeline = healthcheck recon object-server
增加
[filter:recon]
use = egg:swift#recon
recon_cache_path = /var/cache/swift
recon_lock_path = /var/lock
[filter:healthcheck]
use = egg:swift#healthcheck
[root@swfit-storageX ~]# mkdir -p /var/cache/swift
[root@swfit-storageX ~]# chown -R root:swift /var/cache/swift
[root@swfit-storageX ~]# chmod -R 775 /var/cache/swift
[root@swfit-storage ~]#
继续回到控制节点创建ring
创建builder文件
[root@controller ~(admin)]# cd /etc/swift/
[root@controller swift(admin)]# swift-ring-builder account.builder create 12 3 1
[root@controller swift(admin)]# swift-ring-builder container.builder create 12 3 1
[root@controller swift(admin)]# swift-ring-builder object.builder create 12 3 1
创建ring
[root@controller swift(admin)]# swift-ring-builder account.builder add z1-192.168.26.202:6202/lv1 100
WARNING: No region specified for z1-192.168.26.202:6202/lv1. Defaulting to region 1.
Device d0r1z1-192.168.26.202:6202R192.168.26.202:6202/lv1_"" with 100.0 weight got id 0
[root@controller swift(admin)]# swift-ring-builder account.builder add z1-192.168.26.202:6202/lv2 100
WARNING: No region specified for z1-192.168.26.202:6202/lv2. Defaulting to region 1.
Device d1r1z1-192.168.26.202:6202R192.168.26.202:6202/lv2_"" with 100.0 weight got id 1
[root@controller swift(admin)]#
[root@controller swift(admin)]#
[root@controller swift(admin)]#
[root@controller swift(admin)]# swift-ring-builder account.builder add z2-192.168.26.203:6202/lv1 100
WARNING: No region specified for z2-192.168.26.203:6202/lv1. Defaulting to region 1.
Device d2r1z2-192.168.26.203:6202R192.168.26.203:6202/lv1_"" with 100.0 weight got id 2
[root@controller swift(admin)]# swift-ring-builder account.builder add z2-192.168.26.203:6202/lv2 100
WARNING: No region specified for z2-192.168.26.203:6202/lv2. Defaulting to region 1.
Device d3r1z2-192.168.26.203:6202R192.168.26.203:6202/lv2_"" with 100.0 weight got id 3
[root@controller swift(admin)]#
[root@controller swift(admin)]#
[root@controller swift(admin)]#
[root@controller swift(admin)]# swift-ring-builder container.builder add z1-192.168.26.202:6201/lv1 100
WARNING: No region specified for z1-192.168.26.202:6201/lv1. Defaulting to region 1.
Device d0r1z1-192.168.26.202:6201R192.168.26.202:6201/lv1_"" with 100.0 weight got id 0
[root@controller swift(admin)]# swift-ring-builder container.builder add z1-192.168.26.202:6201/lv2 100
WARNING: No region specified for z1-192.168.26.202:6201/lv2. Defaulting to region 1.
Device d1r1z1-192.168.26.202:6201R192.168.26.202:6201/lv2_"" with 100.0 weight got id 1
[root@controller swift(admin)]#
[root@controller swift(admin)]#
[root@controller swift(admin)]#
[root@controller swift(admin)]# swift-ring-builder container.builder add z2-192.168.26.203:6201/lv1 100
WARNING: No region specified for z2-192.168.26.203:6201/lv1. Defaulting to region 1.
Device d2r1z2-192.168.26.203:6201R192.168.26.203:6201/lv1_"" with 100.0 weight got id 2
[root@controller swift(admin)]# swift-ring-builder container.builder add z2-192.168.26.203:6201/lv2 100
WARNING: No region specified for z2-192.168.26.203:6201/lv2. Defaulting to region 1.
Device d3r1z2-192.168.26.203:6201R192.168.26.203:6201/lv2_"" with 100.0 weight got id 3
[root@controller swift(admin)]#
[root@controller swift(admin)]#
[root@controller swift(admin)]#
[root@controller swift(admin)]#
[root@controller swift(admin)]# swift-ring-builder object.builder add z1-192.168.26.202:6200/lv1 100
WARNING: No region specified for z1-192.168.26.202:6200/lv1. Defaulting to region 1.
Device d0r1z1-192.168.26.202:6200R192.168.26.202:6200/lv1_"" with 100.0 weight got id 0
[root@controller swift(admin)]# swift-ring-builder object.builder add z1-192.168.26.202:6200/lv2 100
WARNING: No region specified for z1-192.168.26.202:6200/lv2. Defaulting to region 1.
Device d1r1z1-192.168.26.202:6200R192.168.26.202:6200/lv2_"" with 100.0 weight got id 1
[root@controller swift(admin)]#
[root@controller swift(admin)]#
[root@controller swift(admin)]#
[root@controller swift(admin)]# swift-ring-builder object.builder add z2-192.168.26.203:6200/lv1 100
WARNING: No region specified for z2-192.168.26.203:6200/lv1. Defaulting to region 1.
Device d2r1z2-192.168.26.203:6200R192.168.26.203:6200/lv1_"" with 100.0 weight got id 2
[root@controller swift(admin)]# swift-ring-builder object.builder add z2-192.168.26.203:6200/lv2 100
WARNING: No region specified for z2-192.168.26.203:6200/lv2. Defaulting to region 1.
Device d3r1z2-192.168.26.203:6200R192.168.26.203:6200/lv2_"" with 100.0 weight got id 3
[root@controller swift(admin)]#
查看内容:
rebalance操作
[root@controller swift(admin)]# swift-ring-builder account.builder rebalance
Reassigned 8192 (200.00%) partitions. Balance is now 0.00. Dispersion is now 0.00
[root@controller swift(admin)]#
[root@controller swift(admin)]# swift-ring-builder container.builder rebalance
Reassigned 8192 (200.00%) partitions. Balance is now 0.00. Dispersion is now 0.00
[root@controller swift(admin)]#
[root@controller swift(admin)]# swift-ring-builder object.builder rebalance
Reassigned 8192 (200.00%) partitions. Balance is now 0.00. Dispersion is now 0.00
[root@controller swift(admin)]#
[root@controller swift(admin)]#
拷贝ring文件到存储节点
[root@controller swift(admin)]# scp *gz swift-storageX:/etc/swift/
root@swift-storage's password:
account.ring.gz 100% 258 107.0KB/s 00:00
container.ring.gz 100% 260 88.8KB/s 00:00
object.ring.gz 100% 256 102.0KB/s 00:00
[root@controller swift(admin)]#
在所有节点同步配置swift.conf
[root@controller swift(admin)]# cat swift.conf
[swift-hash]
swift_hash_path_suffix = f8b53d82d4f66b2833e8
swift_hash_path_prefix = 23b9868bb5ee2b48c80c
[storage-policy:0]
name = Policy-0
default = yes
aliases = yellow, orange
[swift-constraints]
[root@controller swift(admin)]#
其中hash部分用命令
openssl rand -hex 10
生成。
在控制节点和存储节点上启动相关服务
控制节点
[root@controller swift(admin)]# systemctl restart openstack-swift-proxy.service
[root@controller swift(admin)]# systemctl is-active openstack-swift-proxy.service
active
[root@controller swift(admin)]# systemctl enable openstack-swift-proxy.service
Created symlink from /etc/systemd/system/multi-user.target.wants/openstack-swift-proxy.service to /usr/lib/systemd/system/openstack-swift-proxy.service.
[root@controller swift(admin)]#
存储节点
[root@swift-storageX swift]# systemctl start openstack-swift-account-auditor.service openstack-swift-account-replicator.service openstack-swift-account.service openstack-swift-container-auditor.service openstack-swift-container-replicator.service openstack-swift-container-sync.service openstack-swift-container-updater.service openstack-swift-container.service openstack-swift-object-auditor.service openstack-swift-object-replicator.service openstack-swift-object-updater.service openstack-swift-object.service
[root@swift-storage swift]#
[root@swift-storage swift]#
[root@swift-storage swift]# systemctl enable openstack-swift-account-auditor.service openstack-swift-account-replicator.service openstack-swift-account.service openstack-swift-container-auditor.service openstack-swift-container-replicator.service openstack-swift-container-sync.service openstack-swift-container-updater.service openstack-swift-container.service openstack-swift-object-auditor.service openstack-swift-object-replicator.service openstack-swift-object-updater.service openstack-swift-object.service
[root@swift-storage swift]#
swift的操作
基本操作
...
删除过期对象
...
权限设置
...
监测
[root@controller swift(admin)]# swift-recon -d
===============================================================================
--> Starting reconnaissance on 2 hosts (object)
===============================================================================
[2018-06-08 15:41:54] Checking disk usage now
Distribution Graph:
1% 4 *********************************************************************
Disk usage: space used: 135503872 of 8547991552
Disk usage: space free: 8412487680 of 8547991552
Disk usage: lowest: 1.58%, highest: 1.59%, avg: 1.58521298454%
===============================================================================
[root@controller ~(admin)]#