Masteringopenstack PDF
Masteringopenstack PDF
Masteringopenstack PDF
www.packtpub.com
https://chandanduttachowdhury.wordpress.com
www.PacktPub.com
www.PacktPub.com
www.PacktPub.com
https://www.packtpub.com/mapt
https://www.amazon.com/dp/1786463989
[email protected]
Chapter 1
Chapter 2
Chapter 3
Chapter 4
Chapter 5
Chapter 6
Chapter 7
Chapter 8
Chapter 9
Chapter 10
Chapter 11
Chapter 12
https://github.com/opens
tack/openstack-ansible
http://docs.openstack.org
/etc/openstack_ansible/
[computes]
compute1.example.com
compute2.example.com
compute3.example.com
compute[20:30].example.com
[email protected]
www.packtpub.com/authors
http://www.p
acktpub.com http://www.packtpub.c
om/support
https://github.com/PacktPubl
ishing/Mastering-OpenStack-SecondEdition
https://github.com/PacktPublishing/
http://www.packtpub.com/submit-errata
https://www.packtpub.com/books/conten
t/support
[email protected]
[email protected]
https
://www.openstack.org/user-stories
Chapter 5
http://docs.openstack.org
/mitaka/networking-guide/
Chapter 7
Chapter 10
Chapter 8
Chapter 3
Chapter 3
Chapter 9
http://en.wikipedia.org/wiki/Network_address_translati
on
Chapter 1
Chapter 1
https://www.virtualbox.org/
http://docs.ansible.com/ansible
/intro.html
http://docs.ansible.com/ansible/list_of_cloud_modules.html#ope
nstack
logserver1.example.com
[controllers]
ctl1.example.com
ctl2.example.com
[computes]
compute1.example.com
compute2.example.com
compute3.example.com
compute[20:30].example.com
/etc/ansible/hosts
---
- hosts: webservers
vars:
http_port: 8080
remote_user: root
tasks:
- name: ensure apache is at the latest version
yum: name=httpd state=latest
- name: write the apache config file
template: src=/https/www.scribd.com/srv/httpd.j2 dest=/etc/httpd.conf
notify:
- restart apache
handlers:
- name: restart apache
service: name=httpd state=restarted
https://github.com/openstack/openstack-ansible
Chapter 1
https://docs.openstack.org/developer/openstack-ansible/
http://docs.openstack.org/developer/openstack-ansible/
https://www.vagrantup.com/downloads.html
config.vm.provider "virtualbox" do |v|
v.memory = 8192
v.cpus = 8
end
https://www.vagrantup.com/docs/virtualbox/boxes.html
http://download.eclipse.org/egit/update
test/roles/bootstrap-host/defaults/main.yml
BOOTSTRAP_OPTS
/etc/openstack_deploy/user_variables.yml
nova.conf
nova_nova_conf_overrides:
DEFAULT:
remove_unused_original_minimum_age_seconds: 43200
libvirt:
cpu_mode: host-model
disk_cachemodes: file=directsync,block=none
database:
idle_timeout: 300
max_pool_size: 10
nova.conf
[DEFAULT]
remove_unused_original_minimum_age_seconds = 43200
[libvirt]
cpu_mode = host-model
disk_cachemodes = file=directsync,block=none
[database]
idle_timeout = 300
max_pool_size = 10
/etc/openstack_deploy/openstack_user_config.yml
http://docs.openstack.org/developer/open
stack-ansible/install-guide/configure-openstack.html
ansible-playbook openstack-ansible
bootstrap_host
lxc-ls
lxc-attach
lxc-ls
http://docs.openstack.org/developer/op
enstack-ansible/developer-docs/quickstart-aio.html
http://git.openstack.org/cgit/openstack/openstack-ansible/tree/
/etc/openstack_ansible/
Chapter 1
Chapter 9
Chapter 1
Chapter 2
https://docs.openstack
.org/developer/openstack-ansible/mitaka/install-guide/configur
e-federation.html
keystone.token.providers.fernet.Provider
keystone_token_provider
keystone_fernet_tokens_key_repository: "/etc/keystone/fernet-keys"
keystone_fernet_tokens_max_active_keys: 7
keystone_fernet_rotation: daily
keystone_fernet_auto_rotation_script: /opt/keystone-fernet-
rotate.sh
https://github.com/openstack/openstack-ansible-os_keystone/blob/
master/defaults/main.yml
/etc/nova/nova.conf
https://docs.openstack.org/mitaka/config-reference/compute/sched
uler.html
Chapter 1
http://de
veloper.openstack.org/#api
glance-api
glance-registry
glance-api
Chapter 5
Chapter 1
ipmitool
Chapter 10
https://docs.openstack.org/develope
r/openstack-ansible/mitaka/install-guide/configure-rabbitmq.ht
ml
Chapter 9
Chapter 1
Git ntp sudo
openssh-server
authorized_keys
lxc
lxc lxc
/var/lib/lxc
https://docs.openstack.org/developer/openstack-ansible/
Configuring OpenStack Ansible
/etc/openstack_deploy
openstack_user_config.yml
Network configuration
openstack_user_config.yaml
cidr_networks:
container: 172.47.36.0/22
tunnel: 172.47.40.0/22
storage: 172.47.44.0/22
used_ips
br-mgmt br-
storage br-vxlan br-vlan
openstack_user_config.yml.example
Configuring Host Groups
openstack_user_config.yml
repo-infra_hosts
shared-infra_hosts
os-infra_hosts
identity_hosts
network_hosts
compute_hosts nova-compute
storage-infra_hosts
storage_hosts
log_hosts
haproxy_hosts
Chapter 1
openstack_user_config.yml shared-infra_hosts
shared-infra_hosts:
cc-01:
ip: 172.47.0.10
cc-02:
ip: 172.47.0.11
cc-03:
ip: 172.47.0.12
os-infra_hosts
os-infra_hosts:
cc-01:
ip: 172.47.0.10
cc-02:
ip: 172.47.0.11
cc-03:
ip: 172.47.0.12
storage-infra_hosts
storage-infra_hosts:
cc-01:
ip: 172.47.0.10
cc-02:
ip: 172.47.0.11
cc-03:
ip: 172.47.0.12
identity_hosts
identity_hosts:
cc-01:
ip: 172.47.0.10
ccr-02:
ip: 172.47.0.11
cc-03:
ip: 172.47.0.12
repo-infra_hosts
repo-infra_hosts:
cc-01:
ip: 172.47.0.10
ccr-02:
ip: 172.47.0.11
cc-03:
ip: 172.47.0.12
haproxy_hosts
haproxy_hosts:
cc-01:
ip: 172.47.0.10
cc-02:
ip: 172.47.0.11
cc-03:
ip: 172.47.0.12
/etc/openstack_deploy/user_variables.yml
haproxy_keepalived_external_vip_cidr
haproxy_keepalived_internal_vip_cidr
haproxy_keepalived_external_interface
haproxy_keepalived_internal_interface
/etc/openstack_deploy/user_secrets.yml
http://docs.ansible.com/ansible/playbooks_vault.html
The playbooks
openstack-ansible
ansible-playbook
os-keystone-install.yml
os-glance-install.yml
os-cinder-install.yml
os-nova-install.yml
os-neutron-install.yml
os-heat-install.yml
os-horizon-install.yml
os-ceilometer-install.yml
os-aodh-install.yml
os-gnocchi-install.yml
os_keystone
...
roles:
- role: "os_keystone"
...
https://github.com/openstack/openstac
k-ansible-os_keystone
setup-hosts.yml
/etc/openstack_deploy/
setup-
infrastructure.yml /etc/openstack_deploy/
setup-openstack.yml
/etc/openstack_deploy/
haproxy-install.yml /etc/openstack_deploy/
Chapter 9
https://github.com/openstack/openstack-ansible-
os_ /blob/master/defaults/main.yml
main.yml defaults
/openstack-ansible-os-horizon/defaults/main.yml
horizon_enable_neutron_lbaas
user_variables.yml
user_variables
https://wiki.openstack.org/wiki/HypervisorSupportMatrix
libvirt
/etc/nova/nova.conf
compute_driver=libvirt.LibvirtDriver
libvirt_type=kvm
kvm_intel or kvm_amd
/etc/modules
kvm
kvm-intel
kvm-intel kvm-amd
ReplicationController
https://docs.opensta
ck.org/developer/magnum/
/etc/nova.conf
default_availability_zone
ServerGroupAffinityFilter ServerGroupAntiAffinityFilter
scheduler_default_filters = ServerGroupAffinityFilter,
ServerGroupAntiAffinityFilter
--hint group=svr-
grp1-uuid
vm1 vm2
vmwareapi.VMwareESXDriver
vmwareapi.VMwareVCDriver
vSphere-
Cluster_01
vSphere.extra
vSphere.extra
vSphere-Cluster_01
Chapter 1
Chapter 1
spec.org
http://spec.org/benchmarks.html#vi
rtual
Chapter 1
cpu_allocation_ratio and ram_allocation_ratio
/etc/nova/nova.conf
scheduler_default_filters
curl wget
reservation-id
public-keys/
security-groups
public-ipv4
ami-manifest-path
instance-type
instance-id
local-ipv4
local-hostname
placement/
ami-launch-index
public-hostname
hostname
ami-id
instance-action
key -O
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 228 100 228 0 0 1434 0 --:--:-- --:--:-- --:--:-- 1447
http://169.254.169.254/
user-data
user-data
custom_userdata_var
export VAR1=var1
export VAR2=var2
export VAR3=var3
Chapter 3
/etc/openstack_deploy/openstack_user_config.yml
compute_hosts
compute_hosts:
cn-01:
ip: 172.47.0.20
/etc/openstack_deploy/user_variables.yml
## Nova options
# Hypervisor type for Nova
nova_virt_type: kvm
# CPU overcommitment ratio
nova_cpu_allocation_ratio: 2.0
# RAM overcommitment ratio
nova_ram_allocation_ratio: 1.5
# Maximum number of virtual machines per compute node
nova_max_instances_per_host: 100
setup-
hosts.yml /etc/openstack_deploy/
--limit
metering-compute_hosts
/etc/openstack_deploy/conf.d/ceilometer.yml
...
metering-compute_hosts:
cn-01:
ip: 172.47.0.20
...
compute_hosts
openstack_user_config.yml
/etc/hosts
/etc/backup-manager.conf
/var/lib/nova/instances
mysql mysqldump
rsync
gzip
mysqldump
/etc/backup-manager.conf
/etc/backup-manager.conf
/var/lib/nova/
https://wiki.openstack.org/wiki/Raksha
https://wiki.openstack.org/wiki/Freezer
Chapter 3
Chapter 1
http://d
ocs.openstack.org/mitaka/config-reference/object-storage.html
http://de
veloper.openstack.org/api-ref-objectstorage-v1.html
Swift gateways
Chapter 1
<builder_file> account.builder container.builder
object.builder
<min_part_hours>
PyECLib
PyECLib
liberasurecode PyECLib
PyECLib
https://docs.openstack.org/devel
oper/swift/overview_erasure_code.html#pyeclib-external-erasure
-code-library
PyECLib
http://docs.openstack.org
/developer/swift/overview_policies.html
1.0526
https
://rwmj.wordpress.com/2009/11/08/filesystem-metadata-overhead/
Where to place what
2
/etc/openstack_deploy/conf.d/swift.yml
/etc/openstack_deploy/user_variables.yml
xfs
/etc/fstab
mount /srv/node/sdX
storage_network replication_network
erasure_coding
swift-proxy_hosts
swift_hosts swift-
proxy_hosts swift_hosts
https://github.com/openstack/openstack-ansible-os_swift/blob
/master/defaults/main.yml
/etc/openstack_deploy/conf.d/swift.yml
swift-proxy_hosts
...
swift-proxy_hosts:
cc-01:
ip: 172.47.0.10
container_vars:
swift_proxy_vars:
write_affinity: "r1"
cc-02:
ip: 172.47.0.10
container_vars:
swift_proxy_vars:
write_affinity: "r2"
cc-03:
ip: 172.47.0.10
container_vars:
swift_proxy_vars:
write_affinity: "r3"
Chapter 3
172.47.40.0/22
...
swift_hosts:
swn01:
ip: 172.47.44.10
container_vars:
swift_vars:
zone: 0
drives:
- name: sdd
- name: sde
- name: sdf
swn02:
ip: 172.47.44.11
container_vars:
swift_vars:
zone: 1
drives:
- name: sdd
- name: sde
- name: sdf
swn03:
ip: 172.47.44.12
container_vars:
swift_vars:
zone: 2
drives:
- name: sdd
- name: sde
- name: sdf
h
ttps://docs.openstack.org/developer/openstack-ansible/mitaka/i
nstall-guide/configure-swift.html
packtpub_tenant
+------------+-------+
| Property | Value |
+------------+-------+
| gigabytes | 1000 |
| snapshots | 50 |
| volumes | 50 |
+------------+-------+
packtpub
create
cinder-
volumes
volume-attach
volume-attach
libvirt Libvirt
LVM-
iSCSI backend
enabled_backends /etc/cinder/cinder.conf
enabled_backends
[vol-conf-grp-1]
volume_group=vol-grp-1
volume_driver=cinder.volume.drivers.lvm.LVMVolumeDriver
volume_backend_name=lvm-standard-bkend
[vol-conf-grp-2]
volume_group= vol-grp-2
volume_driver=cinder.volume.drivers.lvm.LVMVolumeDriver
volume_backend_name= lvm-standard-bkend
[vol-conf-grp-3]
volume_group= vol-grp-3
volume_driver=cinder.volume.drivers.lvm.LVMVolumeDriver
volume_backend_name= lvm-enhanced-bkend
scheduler_default_filters=
AvailabilityZoneFilter,CapacityFilter,CapabilitiesFilter
scheduler_default_weighers= CapacityWeigher
DriverFilter
GoodnessWeigher scheduler_default_filters
scheduler_default_weighers
[vol-conf-grp -1]
volume_group=vol-grp-1
volume_driver=cinder.volume.drivers.lvm.LVMVolumeDriver
volume_backend_name=lvm-standard-bkend
filter_function = "stats.total_capacity_gb < 500"
goodness_function = "(volume.size < 25) 100 : 50"
volume-
type volume-type
volume-type
volume-type
Chapter 3
/etc/openstack_deploy/openstack_user_config.yml
Chapter 1
storage-infra_hosts
...
storage-infra_hosts:
cc-01:
ip: 172.47.0.10
cc-02:
ip: 172.47.0.11
cc-03:
ip: 172.47.0.12
storage_hosts
storage_hosts:
lvm-storage1:
ip: 172.47.44.5
container_vars:
cinder_backends:
vol-conf-grp-1:
volume_backend_name: lvm-standard-bkend
volume_driver: cinder.volume.drivers.lvm.LVMVolumeDriver
volume_group: cinder-volumes
https://github.com/openstack/openstack-ansible-os_cinder/blo
b/master/defaults/main.yml
openstack-ansible
/
http
://docs.openstack.org/developer/manila/adminref/quick_start.ht
ml
driver_handles_share_servers
...
[DEFAULT]
enabled_share_backends = Backend1
enabled_share_protocols = NFS,CIFS
default_share_type = default_share_type
scheduler_driver = manila.scheduler.drivers.filter.FilterScheduler
[Backend1]
share_driver = manila.share.drivers.generic.GenericShareDriver
driver_handles_share_servers = True
service_instance_password = manila
service_instance_user = manila
service_image_name = manila-service-image
path_to_private_key = /home/stack/.ssh/id_rsa
path_to_public_key = /home/stack/.ssh/id_rsa.pub
https://github.com/openstack/manila/blob/mast
er/doc/source/adminref/multi_backends.rst
driver_handles_share_servers network subnet-id
neutron net-show
RADOS librbd
QEMU
h
ttp://ceph.com/docs/master/rados/configuration/filesystem-reco
mmendations/
ceph-mon
ceph-osd
ceph-osd
ceph-
osd ceph-mon
https://github.com/openstack/openstack-ansible/blob/master/playbooks
/ceph-install.yml
https://github.com/ceph/ceph-ansible
group_vars
all.yml.sample
group_vars/all.yml
group_vars/all.yml
...
journal_size: 2048
public_network: 172.47.36.0/22
cluster_network: 172.47.44.0/22
...
group_vars/osds.yml group_vars/mons.yml
osds.yml
...
osd_auto_discovery: True
journal_collocation: True
...
mons.yml
...
cephx: true
...
/etc/ansible/hosts
[mons]
ceph-host[01:03]
[osds]
ceph-host[01:03]
mdss]
ceph-host1
[rgws]
ceph-host2
site.yml
...
- hosts: mons
gather_facts: false
become: True
roles:
- ceph-mon
- hosts: osds
gather_facts: false
become: True
roles:
- ceph-osd
- hosts: mdss
gather_facts: false
become: True
roles:
- ceph-mds
- hosts: rgws
gather_facts: false
become: True
roles:
- ceph-rgw
ceph-host01| success >> {
"changed": false,
"ping": "pong"
}
site.yml
...
ceph-host01 : ok=13 changed=10 unreachable=0 failed=0
ceph-host02 : ok=13 changed=9 unreachable=0 failed=0
ceph-host03 : ok=13 changed=9 unreachable=0 failed=0
...
/etc/glance/glance-api.conf
direct_url = True
/etc/glance/glance-api.conf
glance-api
rbd_store_user rbd_store_pool
images
rbd_id.Image_Glance_ID
/etc/cinder/cinder.conf
/etc/nova/nova.conf
http://ceph.com/docs/mas
ter/rbd/rbd-openstack/
The Neutron plugins implement networking features by resource orchestration. They are broadly categorized
as the core plugin and the service plugins. Core plugin
Chapter 7
subnet-create
--gateway
--disable-dhcp
--dns-
nameserver
eth1.111
brq08c3182b-c3
/etc/neutron/plugins/ml2/ml2_conf.ini
[ml2]
tenant_network_types = vlan
type_drivers =..,vlan,..
mechanism_drivers = linuxbridge
[ml2_type_vlan]
network_vlan_ranges = default:100:300
[linux_bridge]
physical_interface_mappings = default:eth1
/etc/neutron/plugins/ml2/ml2_conf.ini
[ml2]
tenant_network_types = vxlan
type_drivers = ..,vxlan
mechanism_drivers = openvswitch
[ml2_type_vxlan]
vni_ranges = 1001:2000
[agent]
tunnel_types = vxlan
[ovs]
datapath_type = system
tunnel_bridge = br-tun
local_ip = <tunnel endpoint ip>
br-int br-int
br-tun br-ethX
br-tun br-tun
br-ex
ovs-vsctl
show
br-int VLAN 1
br-tun
br-tun
ovs-ofctl dump-flows
br-tun VXLAN ID 0x426 or
1062 VLAN ID 1
ovs-ofctl show br-tun
br-tun
br-int
br-tun br-ethX
ethX
eth1 br-eth1 ethX
br-ethX
/etc/neutron/neutron.conf
[DEFAULT]
service_plugins = router
/etc/neutron/l3_agent.ini
router-create
router-interface-add
ip netns
65ef2787-541c-4ff2-8b69-24ae48094d68
router1 10.56.11.1
https://www.cen
tos.org/docs/5/html/Deployment_Guide-en-US/ch-iptables.html
http://www.iptables.info/en/iptables-targets-and-jumps.html
security-group-create
security-group-rule-
create
80
/etc/neutron/neutron.conf
[DEFAULT]
service_plugins = router,firewall
[fwaas]
driver = netron_fwaas.services.firewall.drivers.linux.
iptables_fwaas.IptablesFwaasDriver
enabled = True
Next, we will enable the FWaaS dashboard in Horizon by updating the OpenStack dashboard configuration
file /usr/share/openstack-
dashboard/openstack_dashboard/local/local_settings.py as follows:
Finally, restart the Neutron server to load the service plugin and the web server to restart Horizon with the
firewall dashboard:
firewall-policy-create
firewall-create
firewall-rule-create
firewall-policy-update firewall-policy-insert-rule
firewall-policy-
insert-rule
--insert-before --insert-
after option
PackPub01 DC01
PacktPub02 DC02
192.168.47.0/24 172.24.4.X
192.168.48.0/24 172.24.4.Y
neutron-plugin-vpn-agent
/etc/neutron/vpn_agent.ini
Openswan
vpnaas
/etc/neutron/neutron.conf
[DEFAULT]
service_plugins =.. ,vpnaas
openswan
service_provider
...
[service_providers]
service_provider= VPN:openswan:neutron.services.vpn.
service_drivers.ipsec.IPsecVPNDriver:default
VPNaaS
/usr/share/openstack-
dashboard/openstack_dashboard/local/local_settings.py
'enable_VPNaaS': True,
neutron-server neutron-vpn-agent
https://www.openswan.org/
Creating the Internet Key Exchange policy
DC01
neutron
DC01
neutron
192.168.47.0/24
PacktPub02
172.24.4.227 192.168.48.0/24
DC01
AwEsOmEVPn
neutron
DC02
DC01 192.168.47.0/24
python-networking-ovn
/etc/neutron/plugins/ml2/ml2_conf.ini
[ml2]
tenant_network_types = geneve
extension_drivers = port_security
type_drivers = local,flat,vlan,geneve
mechanism_drivers = ovn,logger
[ml2_type_geneve]
max_header_size = 58
vni_ranges = 1:65536
geneve
geneve
geneve
geneve
ml2_config.ini
[ovn]
ovn_l3_mode = True
ovn_sb_connection = tcp:192.168.122.126:6642
ovn_nb_connection = tcp:192.168.122.126:6641
/etc/neutron/neutron.conf
service_plugins = L3_SERVICE
neutron net-list
ovn-nbctl ls-list
ovn-nbctl lsp-list show
ovn-nbctl lr-list
ovn-northd
ovn-sbctl lflow-list
Chapter 6
ovs-
vsctl show ovs-ofctl dump-flows
service_plugins =
neutron_lbaas.services.loadbalancer.plugin.LoadBalancerPluginv2
service_provider =
LOADBALANCERV2:Octavia:neutron_lbaas.drivers.octavia.driver.OctaviaDriver:d
efault
loadbalancer
loadbalancer
lbaaS
loadbalancer listener loadbalancer
listener port loadbalancer
loadbalancer
pool listener pool
ROUND_ROBIN lb-algorithm
pool
nova
pool
loadbalancer port 80 loadbalancer
ROUND_ROBIN
healthmonitor loadbalancer
Chapter 3
pp_test_user Testing_PP
+-------------+----------------------------------+
| Property | Value |
+-------------+----------------------------------+
| description | Test Environment Project |
| enabled | True |
| id | 832bd4c20caa44778f4acf5481d4a4a9 |
| name | Testing_PP |
+-------------+----------------------------------+
pp_test_user
+----------+----------------------------------+
| Property | Value |
+----------+----------------------------------+
| email | [email protected] |
| enabled | True |
| id | 4117dc3cc0054db4b8860cc89ac21278 |
| name | pp_test_user |
| username | pp_test_user |
+----------+----------------------------------+
member
+-------------------------------+--------+-----------+-----------+
| ID | Name |Project | User |
+-------------------------------+--------+-----------+-----------+
9fe2ff9ee4384b1894a90878d3e92bab|_member_|Testing_PP |pp_test_user
+-------------------------------+--------+-----------+-----------+
_member_
policy.json
/etc/OPENSTACK_SERVICE/policy.json
OPENSTACK_SERVICE
_member_
admin pp_test_user
admin
router_owner
+----------+----------------------------------+
| Property | Value |
+----------+----------------------------------+
| id | e4410d9ae5ad44e4a1e1256903887131 |
| name | router_owner |
+----------+----------------------------------+
pp_test_user
/etc/neutron/policy.json
create_router
...
"create_router": "rule:admin_only or role:router_owner"
...
pp_test_user
Testing_PP
# nova quota-update QUOTA_KEY QUOTA_VALUE TENANT_ID
nova quota-
show
keystone tenant-list
+-----------------------------+-------+
| Quota | Limit |
+-----------------------------+-------+
| instances | 20 |
| cores | 20 |
| ram | 25000 |
keystoneuser-list
keystone tenant-list
pp_test_user
pp_test_user
keystone user-list
nova quota-
show
+-----------+-------+
| Property | Value |
+-----------+-------+
| gigabytes | 500 |
| snapshots | 10 |
| volumes | 10 |
+-----------+-------+
Testing_PP
# cinder quota-update --QUOTA_NAME QUOTA_VALUE TENANT_ID
cinder
quota-show
keystone
tenant-list
/etc/cinder/cinder.conf
/etc/neutron/neutron.conf
...
quota_driver = neutron.db.quota_db_DbQuotaDriver
...
Testing_PP
keystone
tenant-list
neutron
quota-show
quota_items
...
[quotas]
quota_items = network, port
...
[quotas]
...
quota_secuirty_gourp = 20
quota_security_group_rule = 100
...
max_stacks_per_tenant = 200
max_resources_per_stack = 2000
max_template_size = 1000000
max_nested_stack_depth = 10
max_events_per_stack = 2000
heat_template_version:
description:
parameters:
param1
type:
label:
description:
default:
param2:
….
resources:
resource_name:
type: OS::*::*
properties:
prop1: { get_param: param1}
prop2: { get_param: param2}
.......
outputs:
output1:
description:
value: { get_attr: resource_name,attr] }
......
heat_template_version
description
parameters
resources
resource_name virtual_web
OS::Nova::Server
virtual_web
outputs
http://docs.openstack.org/developer/heat/template_guide/openstac
k.html
Modularizing the stacks
----Templates
| ------------------ pp_stack.yaml
| ------------------ Lib
| --------------- env.yaml
| --------------- mariadb.yaml
| --------------- privateNet.yaml
| --------------- publicNet.yaml
| --------------- httpd.yaml
| --------------- loadbalancer.yaml
pp_stack.yaml
env.yaml
Resource_registry
Parameters
Parameter_defaults
resource_registry
Lib
PacktPub
resource_registry:
Lib::PacktPub::MariaDB: mariadb.yaml
Lib::PacktPub::PrivNet:privateNet.yaml
Lib::PacktPub::PubNet:publicNet.yaml
Lib::PacktPub::Httpd:httpd.yaml
Lib::PacktPub::LoadBalancer:loadbalancer.yaml
mariadb.yaml
heat_template_version: 2013-05-23
parameters:
image:
type: string
default: centos7
flavor:
type: string
default: m1.medium
key:
type: string
default: my_key
private_network:
type: string
default: Private_Network
database_name:
type: string
database_user:
type: string
resources:
database_root_password:
type: OS::Heat::RandomString
database_password:
type: OS::Heat::RandomString
database_root_password:
type: OS::Heat::RandomString
security_group:
type: OS::Neutron::SecurityGroup
properties:
name: db_server_security_group
rules:
- protocol: tcp
port_range_min: 3306
port_range_max: 3306
port:
type: OS::Neutron::Port
properties:
network: { get_param: private_network }
security_groups:
- { get_resource: security_group }
mariadb_instance:
type: OS::Nova::Server
properties:
image: { get_param: image }
flavor: { get_param: flavor }
key_name: { get_param: key }
networks:
- port: { get_resource: port }
user_data_format: RAW
user_data:
str_replace:
params:
__database_root_password__: { get_attr:
[database_root_password, value] }
__database_name__: { get_param: database_name }
__database_user__: { get_param: database_user }
__database_password__: { get_attr:
[database_password, value] }
template: |
#!/bin/bash -v
yum -y install mariadb mariadb-server
systemctl enable mariadb.service
systemctl start mariadb.service
mysqladmin -u root password $db_rootpassword
cat << EOF | mysql -u root --
password=$db_rootpassword
CREATE DATABASE $db_name;
GRANT ALL PRIVILEGES ON $db_name.* TO "$db_user"@"%"
IDENTIFIED BY "$db_password";
FLUSH PRIVILEGES;
EXIT
EOF
outputs:
name:
description: Database Name.
value: { get_attr: [mariadb_instance, name] }
ip:
description: Database IP address.
value: { get_attr: [mariadb_instance, first_address] }
port:
description: Database port number.
value: { get_resource: port }
database_password:
description: Database password.
value: { get_attr: [database_password, value] }
privateNet.yaml
heat_template_version: 2013-05-23
parameters:
public_network:
type: string
default: Public_Network
cidr:
type: string
default: '10.10.10.0/24'
dns:
default: '8.8.8.8'
resources:
private_network:
type: OS::Neutron::Net
private_subnet:
type: OS::Neutron::Subnet
properties:
network_id: { get_resource: private_network }
cidr: 10.10.10.0/24
dns_nameservers: { get_param: dns }
router:
type: OS::Neutron::Router
properties:
external_gateway_info:
network: { get_param: public_network }
router-interface:
type: OS::Neutron::RouterInterface
properties:
router_id: { get_resource: router }
subnet: { get_resource: private_subnet }
outputs:
name:
description: Private Network.
value: { get_attr: [private_network, name] }
publicNet.yaml
heat_template_version: 2013-05-23
description: Associate floating IP to servers to access
public network.
parameters:
port:
type: string
public_network:
type: string.
default: Public_Network
resources:
floating_ip:
type: OS::Neutron::FloatingIP
properties:
floating_network: { get_param: public_network }
floating_ip_assoc:
type: OS::Neutron::FloatingIPAssociation
properties:
floatingip_id: { get_resource: floating_ip }
port_id: { get_param: port }
outputs:
ip:
description: The floating IP address assigned to the server.
value: { get_attr: [floating_ip, floating_ip_address] }
httpd.yaml
heat_template_version: 2013-05-23
description: Installs a web server running httpd.
parameters:
image:
type: string
default: centos7
flavor:
type: string
default: m1.small
key:
type: string
default: my_key
private_network:
type: string
default: Private_Network
resources:
security_group:
type: OS::Neutron::SecurityGroup
properties:
name: web_server_sg
rules:
- remote_ip_prefix: 0.0.0.0/0
protocol: tcp
port_range_min: 80
port_range_max: 80
- remote_ip_prefix: 0.0.0.0/0
protocol: tcp
port_range_min: 443
port_range_max: 443
port:
type: OS::Neutron::Port
properties:
network: { get_param: private_network }
security_groups:
- { get_resource: security_group }
ws_instance:
type: OS::Nova::Server
properties:
image: { get_param: image }
flavor: { get_param: flavor }
key_name: { get_param: key }
networks:
- port: { get_resource: port }
user_data_format: RAW
user_data:
str_replace:
template: |
#!/bin/bash -ex
yum -y install httpd
systemctl enable httpd.service
systemctl start httpd.service
setsebool -P httpd_can_network_connect_db=1
outputs:
name:
description: Web Server instance.
value: { get_attr: [ws_instance, name] }
ip:
description: Web Server IP address.
value: { get_attr: [ws_instance, first_address] }
port:
description: Web Server Port number.
value: { get_resource: port }
loadbalancer.yaml load-balancer
load-balancer
heat_template_version: 2013-05-23
description: A load-balancer server
parameters:
image:
type: string
key_name:
type: string
flavor:
type: string
pool_id:
type: string
user_data:
type: string
metadata:
type: json
network:
type: string
resources:
server:
type: OS::Nova::Server
properties:
flavor: {get_param: flavor}
image: {get_param: image}
key_name: {get_param: key_name}
metadata: {get_param: metadata}
user_data: {get_param: user_data}
user_data_format: RAW
networks: [{network: {get_param: network} }]
member:
type: OS::Neutron::PoolMember
properties:
pool_id: {get_param: pool_id}
address: {get_attr: [server, first_address]}
protocol_port: 80
outputs:
server_ip:
description: Load Balancer IP Address
value: { get_attr: [server, first_address] }
lb_member:
description: LB member details.
value: { get_attr: [member, show] }
pp_stack.yaml
heat_template_version: 2013-05-23
description: Create Multi-Tier Application Stack
parameters:
image:
type: string
default: centos7
flavor:
type: string
default: m1.medium
key:
type: string
default: my_key
public_network:
type: string
default: Public_Network
resources:
network:
type: Lib::PacktPub::PrivNet
properties:
public_network: { get_param: public_network }
mariadb:
type: Lib::PacktPub::MariaDB
properties:
image: { get_param: image }
flavor: { get_param: flavor }
key: { get_param: key }
private_network: { get_attr: [network, name] }
database_name: website
database_user: website_user
server:
type: Lib::PacktPub::Httpd
properties:
image: { get_param: image }
flavor: { get_param: flavor }
key: { get_param: key }
private_network: { get_attr: [network, name] }
mariadb: { get_attr: [mariadb, ip] }
database_name: website
database_user: website_user
database_password: { get_attr: [mariadb,
database_password] }
public_ip:
type: Lib::PacktPub::PubNet
properties:
port: { get_attr: [server, port] }
public_network: { get_param: public_network }
outputs:
ip:
description: Web Server Public IP
value: { get_attr: [public_ip, ip] }
https://www.terraform.io/docs/providers/openstack/
https://golang.org/
local PATH
terraform
<filepath>
PATH
terraform
.TF
variables.tf
provider.tf
infra.tf
postscript.sh
variable.tf
variable "OS_USERNAME" {
description = "The username for the Tenant."
default = "pp_user"
}
variable "OS_TENANT" {
description = "The name of the Tenant."
default = "pp_tenant"
}
variable "OS_PASSWORD" {
description = "The password for the Tenant."
default = "367811794c1d45b4"
}
variable "OS_AUTH_URL" {
description = "The endpoint url to connect to the Cloud Controller
OpenStack."
default = "http://10.0.10.10:5000/v2.0"
}
variable "OS_REGION_NAME" {
description = "The region to be used."
default = "RegionOne"
}
variable "image" {
description = "Default image for web server"
default = "centos"
}
variable "flavor" {
description = "Default flavor for web server instance"
default = "m1.small"
}
variable "ssh_key_file" {
description = "Public SSH key for passwordless access the server."
default = "~/.ssh/pubkey"
}
variable "ssh_user_name" {
description = "Default SSH user configured in the centos image uploaded by
glance."
default = "centos"
}
variable "private_network" {
description = "Default private network created in OpenStack"
default = "Private_Network"
}
variable "private_subnet" {
description = "Default private subnet network which the web server will be
attached to"
default = "Private_Subnet"
}
variable "router" {
description = "Default Neutron Router created in OpenStack"
default = "pp_router"
}
variable "external_gateway" {
description = "Default External Router Interface ID"
default = "ac708df9-23b1-42dd-8bf1-458189db71c8"
}
variable "public_pool" {
description = "Default public network to assign floating IP for external
access"
default = "Public_Network"
}
postscript.sh httpd
#!/bin/bash
yum -y install httpd
systemctl enable httpd.service
systemctl start httpd.service
chkconfig --level 2345 httpd on
provider.tf
provider "openstack" {
user_name = "${var.OS_USERNAME}"
tenant_name = "${var.OS_TENANT}"
password = "${var.OS_PASSWORD}"
auth_url = "${var.OS_AUTH_URL}"
}
infra.tf
rule {
from_port = 22
to_port = 22
ip_protocol = "tcp"
cidr = "192.168.0.0/16"
}
rule {
from_port = 80
to_port = 80
ip_protocol = "tcp"
cidr = "0.0.0.0/0"
}
rule {
from_port = 443
to_port = 443
ip_protocol = "tcp"
cidr = "0.0.0.0/0"
}
}
network {
uuid = "${var.Private_Network}"
}
remote-exec
postscript.sh user_data
provisioner "remote-exec" {
connection {
user = "${var.ssh_user_name}"
secret_key_ = "/root/.ssh/id_rsa"
timeout = "20m"
}
user_data = "${file("postscript.sh")}"
}
}
user_data
inline = [
"yum -y install httpd",
"systemctl enable httpd.service"
"systemctl start httpd.service",
"chkconfig --level 2345 httpd on"
]
Chapter 1
Chapter 3 Chapter 4
Chapter 5
http://docs.openstack.org/admin-guide/compute-configuring-migrat
ions.html#section-configuring-compute-migrations
Chapter 1
Chapter 1
Chapter 1
Chapter 1
Chapter 1
Chapter 1
https://github.com/codership/galera
https://www.rabbitmq.com
/ha.html
Implementing HA on MySQL
haproxy1 haproxy2
/etc/keepalived/keepalived.conf
net.ipv4.ip_nonlocal_bind=1
vrrp_script chk_haproxy {
script "killall -0 haproxy"
interval 2
weight 2
}
vrrp_instance MYSQL_VIP {
interface eth0
virtual_router_id 120
priority 111 # Second HAProxy is 110
advert_int 1
virtual_ipaddress {
192.168.47.47/32 dev eth0
}
track_script {
chk_haproxy
}
}
vrrp_instance MYSQL_VIP
haproxy
sst
sstpassword
wsrep Galera
/etc/mysql/conf.d/wsrep.cnf db01.packtpub.com
wsrep_provider=/usr/lib64/galera/libgalera_smm.so
wsrep_cluster_address="gcomm://"
wsrep_sst_method=rsync
wsrep_sst_auth=sst:sstpass
db02.packtpub.com
wsrep_provider=/usr/lib64/galera/libgalera_smm.so
wsrep_cluster_address="gcomm://192.168.47.125"
wsrep_sst_method=rsync
wsrep_sst_auth=sst:sstpass
db03.packtpub.com
wsrep_provider=/usr/lib64/galera/libgalera_smm.so
wsrep_cluster_address="gcomm://192.168.47.126"
wsrep_sst_method=rsync
wsrep_sst_auth=sst:sstpass
db01.packtpub.com gcomm
db03.packtpub.com
/etc/mysql/conf.d/wsrep.cnf
wresp_cluster_address ="gcomm://192.168.47.127"
mysql
wsrep_ready = ON
db01.packtpub.com
wsrep_cluster_size 3
wsrep_cluster_status.status
Nova: /etc/nova/nova.conf
sql_connection=mysql://nova:[email protected]/nova
Keystone: /etc/keystone/keystone.conf
sql_connection=mysql://keystone:[email protected]/keystone
Glance: /etc/glance/glance-registry.conf
sql_connection=mysql://glance:[email protected]/glance
Neutron: /etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini
sql_connection=mysql://neutron:[email protected]/neutron
Cinder: /etc/cinder/cinder.conf
sql_connection=mysql://cinder:[email protected]/cinder
sed awk
ini
conf http://www.pixelbeat.org/programs/crudini
/
/etc/nova/nova.conf
Implementing HA on RabbitMQ
cc02 cc03
rabbimqctl
cc02
cc03
/etc/rabbitmq/rabbitmq.config
[{rabbit,
[{cluster_nodes, {['rabbit@cc01', 'rabbit@cc02', 'rabbit@cc03'], ram}}]}].
192.168.47.47
192.168.47.47
/etc/nova/nova.conf
/etc/glance/glance-api.conf
/etc/neutron/neutron.conf
/etc/cinder/cinder.conf
http://www.linux-ha.org/wiki/Resource_Agents
corosync
/etc/corosync/corosync.conf
Interface {
ringnumber: 0
bindnetaddr: 192.168.47.0
mcastaddr: 239.225.47.10
mcastport: 4000
....}
http://docs.openstack.or
g/high-availability-guide/content/_set_up_corosync_unicast.htm
l
/etc/corosync/authkey
/etc/corosync/corosync.conf
pacemaker corosync
cc01
192.168.47.48
cc01
cc01
/usr/lib/ocf/resource.d/openstack
p_nova_api
nova-api
p_cert nova-cert
p_consoleauth nova-
consoleauth
p_scheduler nova-scheduler
p_novnc nova-vnc
p_keystone
p_glance-api
p_glance-registry glance-
registry
p_neutron-server
dnsmasq
VRRP in Neutron
nn02
Chapter 1
/etc/neutron/neutron.conf
...
rabbit_host = 192.168.47.47
...
/etc/neutron/l3_agent.ini
[Default]
interface_driver =
neutron.agent.linux.interface.OVSInterfaceDriver
LinuxBridge
interface_driver
neutron.agent.linux.interface.BridgeInterfaceDriver
/etc/neutron/l3_agent.ini
router_delete_namespaces
...
router_delete_namespaces = True
...
l3_ha = True
...
max_l3_agents_per_router = 3
...
...
min_l3_agents_per_router = 2
...
--ha true
--ha
qrouter-2782ff83-15b0-4e92-83de-3b569d20cd09
qrouter-7151c4ca-336f-43ef-99bc-396a3329ac2f
qrouter-a029775e-204b-45b6-ad86-0ed2e507d5
qrouter-2782ff83-15b0-4e92-83de-3b569d20cd09
169.254.0.1
l3_ha_net_cidr
/etc/neutron/neutron.conf
Router_Name
/var/lib/neutron/ha_confs/ROUTER_NETNS/keepalived.conf
ROUTER_NETNS
neutron-keepalived-state-change.log
...
DEBUG neutron.agent.l3.keepalived_state_change [-] Wrote router
2782ff83-15b0-4e92-83de-3b569d20cd09 state master write_state_change
...
More HA in Neutron
Chapter 1
/etc/openstack_deploy/openstack_user_config.yml
VIP
global_overrides
...
global_overrides:
internal_lb_vip_address: 172.47.0.47
external_lb_vip_address: 192.168.47.47
haproxy_keepalived_internal_vip_cidr
haproxy_keepalived_external_vip_cidr
/etc/openstack_deploy/user_variables.yml
global_overrides
tunnel_bridge
management_bridge br-vxlan
br-mgmt
br-mgmt
br-vxlan
br-vlan
br-vlan
br-storage
Chapter 1
...
haproxy_hosts:
cc-01:
ip: 172.47.0.10
cc-02:
ip: 172.47.0.11
cc-03:
ip: 172.47.0.12
/vars/configs/haproxy_config.yml
....
- service:
haproxy_service_name: glance_api
haproxy_backend_nodes: "{{ groups['glance_api'] | default([]) }}"
haproxy_ssl: "{{ haproxy_ssl }}"
haproxy_port: 9292
haproxy_balance_type: http
haproxy_backend_options:
- "httpchk /healthcheck"
...
haproxy_config.yml
user_variables.yml
/vars/configs/keepalived_haproxy.yml
haproxy_keepalived_priority_master: 101
haproxy_keepalived_priority_backup: 99
br-mgmt enp0s3
user_variables.yml
haproxy_keepalived_internal_interface: br-mgmt
haproxy_keepalived_external_interface: enp0s3
haproxy-install.yml
avg
sum
min
max
count
Chapter 12
http://gnocchi.xyz/
cc01.packtpub
/etc/mongodb.conf
bind_ip = 172.47.0.10
/var/lib/mongodb/journal
/etc/mongodb.conf
file:smallfiles = true
MongoDB
ceilometer
/etc/ceilometer/ceilometer.conf
Ceilometer
[service_credentials]
os_username=ceilometer
os_password=service_password
os_tenant_name=service
os_auth_url=http:// 172.47.0.10:35357/v2.0
...
[keystone_authtoken]
auth_host=172.47.0.10
auth_port=35357
auth_protocol=http
auth_uri=http://172.47.0.10:5000/v2.0
admin_user=ceilometer
admin_password=service_password
admin_tenant_name=service
...
[publisher]
metering_secret=ceilo_secret
https://github.com/openstack/openstack-ansible
/blob/master/playbooks/os-ceilometer-install.yml
cn01.packtpub
/etc/nova/nova.conf
...
notification_driver = nova.openstack.common.notifier.rpc_notifier
notification_driver = ceilometer.compute.nova_notifier
instance_usage_audit = True
instance_usage_audit_period = hour
notify_on_state_change = vm_and_task_state
/etc/ceilometer/ceilometer.conf $ADMIN_TOKEN
...
[publisher_rpc]
metering_secret= 47583f5423df27685ced
Configure RabbitMQ access:
...
[DEFAULT]
rabbit_host = cc01
rabbit_password = $RABBIT_PASS
...
[keystone_authtoken]
auth_host = cc01
auth_port = 35357
auth_protocol = http
admin_tenant_name = service
admin_user = ceilometer
admin_password = service_password
...
[service_credentials]
os_auth_url = http://cc01.packtpub:5000/v2.0
os_username = ceilometer
os_tenant_name = service
os_password = service_password
log_dir
...
[DEFAULT]
log_dir = /var/log/ceilometer
alarmservice
aodh
aodh
aodh
aodh
aodh aodh
...
[database]
connection=mysql+pymysql://aodh:AODH_PASSWORD@cc01/aodh
Change the following in the RabbitMQ section:
...
[oslo_messaging_rabbit]
rabbit_host=172.47.0.10
rabbit_port=5672
rabbit_password=RABBIT_PASS
rpc_backend=rabbit
...
[service_credentials]
auth_type = password
auth_url = http://cc01:5000/v3
project_name = services
username = aodh
password = aodh_pass
interface = internalURL
keystone
...
[keystone_authtoken]
auth_strategy = keystone
auth_type=passowrd
project_name = services
username = aodh
password = aodh_pass
auth_uri=http://cc01:5000
auth_url=http://cc01:35357
memcached_servers = 172.47.0.10:11211
aodh
nagiosadmin
/etc/nagios/cgi.cfg
http://NAGIOS_SERVER_IP/nagios
/usr/lib/nagios/plugins
/etc/nagios/nrpe.cfg
...
allowed_hosts = NAGIOS_SERVER_IP
...
Watching OpenStack
https://github.com/cirrax/openstack-nag
ios-plugins
check_http
check_http
check_http
check_http
check_http
check_http
check_procs
check_procs
check_procs
check_mysql
check_dhcp
check_rabbitmq_server
https://www.monitoring-plugins.org/ind
ex.html
https://exchange.nagios.org/directory/Plugins
https
://github.com/nagios-plugins-rabbitmq/nagios-plugins-rabbitmq
/etc/nagios/nagios.cfg
...
cfd_dir = /etc/nagios/servers
...
define host{
use linux-server
host_name cn01
alias compute01
address 172.28.128.18
}
define service {
use generic-service
host_name cn01
check_command check_nrpe!check_nova_compute
notification_period 24x7
service_description Compute Service
}
check_nrpe /etc/nagios
nagios_commands_checks.cfg
...
cfg_file=/etc/nagios/nagios_command_checks.cfg
define command{
command_line $USER1$/check_nrpe -H $HOSTADDRESS$ -c $ARG1$
command_name check_nrpe
}
...
command[compute]=/usr/lib64/nagios/plugins/check_procs -C nova-compute -u
nova -c 1:4
allowed_hosts
...
allowed_hosts=127.0.0.1,172.28.128.47
Compute Service
CRITICAL
-c 1:4 CRITICAL
ps pgrep
Binary
Host
Zone
Status
State
Updated_At
nova-scheduler nova-api
nova-manage ps
/var/log/nova*.log
Chapter 11
Binary
Host
Zone
Status
State up
down
Updated_At
Disabled Reason
pgrep
pgrep ps
heat-engine
service-list
5000
lsof netstat
tcp6 0 0 :::5000 :::* LISTEN
telnet 80
IPtables firewall
http://docs.openstack.org/mitaka/config-reference/firewalls-default-ports.ht
ml
ID
Name
Status
http://docs.openstack.org/developer/nova/vmstates.html
Task State
Power State
Networks
nova hypervisor-list
MariaDB> use nova;
MariaDB [nova]> update instances set host='cc02.pp'
where host='cc01.pp';
/etc/libvirt/qemu/instance-*.xml
--hard
/var/lib/nova/instances/INSTANCE_UIID/console.log
INSTANCE_UIID UUID
qrouter-UUID
qdhcp-UUID
tcpdump
br-int br-ex
ip netns
10.15.15.63
br-int br-tun
br-ex
enp0s3
enp0s3
/etc/neutron/l3_agent.ini
tcpdump
os-nova-install Chapter 2
vars YAML log_dirs
/var/log/nova
...
vars
log_dirs:
- src: "/openstack/log/{{ inventory_hostname }}--nova"
dest: "/var/log/nova"
/var/log
/var/log/
/var/log/nova/
/var/log/glance/
/var/log/keystone/
/var/log/horizon/
/var/log/cinder/
/var/log/swift/
/var/log/syslog/
/var/log/manila/
/var/lib/nova/instances/instance-ID/
/var/log/neutron/
/var/log/ceilometer/
/var/log/aodh/
/var/log/gnocchi/
/var/log/heat/
/var/log/httpd
/var/log/apache2
/var/log/apache2/
/var/log/httpd/
/var/log/mariadb/
/var/log/rabbitmq/
/var/log/libvirt/
console.log
/var/lib/nova/instances/instance-ID
/var/log/ceph
/etc/nova/nova.conf,
False True
debug=True
True False
/etc/nova/nova.conf
log_config_append = /etc/nova/logging.conf
logging.conf /etc/nova/
[loggers]
keys=nova
[handlers]
keys=consoleHandler
[formatters]
keys=simpleFormatter
[logger_nova]
level=DEBUG
handlers=consoleHandler
qualname=nova
propagate=0
[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=simpleFormatter
args=(sys.stdout,)
[formatter_simpleFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
[elasticsearch-5.x]
name=Elasticsearch repository for 5.x packages
baseurl=https://artifacts.elastic.co/packages/5.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
yum
elasticsearch
...
network.host: 0.0.0.0
...
9200
elasticsearch.yml
cluster:
...
name: elk_pp_cluster
...
node.master: true
node.data: false
...
node.master: false
node.data: true
...
node.master: false
node.data: false
Extending ElasticSearch capabilities
https://www.elastic.co/guide/en/elasticsearch/plugins/master/index.html
elasticsearch
elasticsearch-plugin
x-pack
...
* javax.net.ssl.SSLPermission setHostnameVerifier
http://docs.oracle.com/javase/8/docs/technotes/guides/secur
ity/permissions.html
[y/N]y
x-pack
Installing Kibana
[kibana-5.x]
name=Kibana repository for 5.x packages
baseurl=https://artifacts.elastic.co/packages/5.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
yum
kibana on
Configuring Kibana
servers.host
/etc/kibana/kibana.yml
server.host: 0.0.0.0
elasticsearch_url: http//localhost:9200
5601
httpd-tools
/etc/nginx/conf.d/
server {
listen 80;
server_name els001.pp;
auth_basic "Restricted Access";
auth_basic_user_file /etc/nginx/htpasswd.users;
location / {
proxy_pass http://localhost:5601;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
}
server_name hostname
httpd_can_network_connect
x-pack
[email protected]
elastic changeme
Installing LogStash
subjectAltName [v3_ca]
/etc/pki/tls/openssl.cnf file
...[v3_ca]
subjectAltName=IP: 172.28.128.19
/etc/pki/tls
logstash-
forwarder.crt
LogStash in action
input {
stdin { }
...
}
filter {
...
}
output {
stdout {
.....
}
www.elastic.co/support/matrix#show_logstash_plugins
input {
beats {
port => 5044
ssl => true
ssl_certificate => "/etc/pki/tls/certs/logstash-
forwarder.crt"
ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
}
}
/etc/logstash/conf.d/
elasticsearch
output {
elasticsearch {
hosts => ["localhost:9200"]
sniffing => true
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}
9200
filebeat-YYYY.MM.dd
Configuration OK
Output:
{
"acknowledged" : true
}
logstash-*
Preparing LogStash clients
[beats]
name=Elastic Beats Repository
baseurl=https://packages.elastic.co/beats/yum/el/$basearch
enabled=1
gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch
gpgcheck=1
/etc/filebeat/filebeat.yml
...
### Logstash as output
logstash
# The Logstash hosts
hosts: ["172.47.0.10:5044"]
...
bulk_max_size: 2048
tls
...
tls:
certificate_authorities: ["/etc/pki/tls/certs/logstash-
forwarder.crt"]
prospectors
...
prospectors:
paths:
- /var/log/keystone/keystone.log
prospectors
Control Plane
keystone
...
prospectors:
paths:
- /var/log/keystone/keystone.log
document_type: Control Plane
fields:
tags: ["keystone"]
filebeat
Filtering OpenStack logs
filter
filter{
if [type] == "openstack" {
grok {
patterns_dir =>
"/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-
patterns-core-4.0.2/patterns/"
match=>[ "message","%{TIMESTAMP_ISO8601:timestamp} %
{NUMBER:response} %{AUDITLOGLEVEL:level} %{NOTSPACE:module} [%
{GREEDYDATA:program}] %{GREEDYDATA:content}"]
add_field => ["openstask"]
add_tag => ["openstackmetrics"]
}
multiline {
negate => false
pattern => "^%{TIMESTAMP_ISO8601}%{SPACE}%{NUMBER}%
{SPACE} TRACE"
what => "previous"
stream_identity => "%{host}.%{filename}"
}
mutate {
gsub => ['logmessage', "" ," " ]
}
date {
type => "openstack"
match => [ "timestamp", "yyyy-MM-dd HH:mm:ss.SSS" ]
}
}
}
grok
logstash-patterns-core
https://github.com/logstash-plugins/logstash-patterns-core/tree/
master/patterns.
grok
patterns_dir
where
match
add_field
add_tag
https://www.elastic.co/guide/en/logstash/current/plugins-filte
rs-grok.html
AUDITLOGLEVEL
AUDITLOGLEVEL
AUDITLOGLEVEL([C|c]ritical|CRITICAL[A|a]udit|AUDIT|[D|d]ebug|DEBUG|[N|n]oti
ce|NOTICE|[I|i]nfo|INFO|[W|w]arn(:ing)|WARN(:ING)|[E|e]rr(:or)|ERR
(:OR)|[C|c]rit(:ical)|CRIT(:ICAL)|[F|f]atal|FATAL|[S|s]evere|SEVER
E)
http://grokdebug.herokuapp.com/
gork
multiline
ceilometer api
%{filename}
https://www.e
lastic.co/guide/en/logstash/current/plugins-codecs-multiline.h
tml
mutate
gusb
logmessage space
https://www.ela
stic.co/guide/en/logstash/current/plugins-filters-mutate.html
date
yyyy-MM-dd
HH:mm:ss.SSS
2015-04-07 20:51:42.833
https://www.elast
ic.co/guide/en/logstash/current/plugins-filters-date.html
grok openstack
multiline mutate
date
...
paths:
- /var/log/horizon/*
fields:
tags: ["horizon", "dashboard", "openstack"]
-
paths:
- /var/log/nova/*
fields:
tags: ["nova", "compute", "openstack"]
-
paths:
- /var/log/neutron/*
fields:
tags: ["neutron", "network", "openstack"]
...
file {
path => ['/var/log/keystone/*.log']
tags => ['keystone', 'oslofmt']
type => "openstack, identity"
}
file {
path => ['/var/log/nova/*.log']
tags => ['nova', 'oslofmt']
type => "openstack, compute"
}
file {
path => ['/var/log/horizon/*.log']
tags => ['horizon', 'oslofmt']
type => "openstack, dashboard"
}
file {
path => ['/var/log/neutron/*.log']
tags => ['neutron', 'oslofmt']
type => "openstack, network"
}
openstack
...
if "nova" in [tags] {
grok {
match => { "logmessage" => "[-] %{NOTSPACE:requesterip}
[%{NOTSPACE:req_date} %{NOTSPACE:req_time}] %
{NOTSPACE:method} %{NOTSPACE:url_path} %
{NOTSPACE:http_ver} %{NUMBER:response} % {NUMBER:bytes}
%{NUMBER:seconds}" } add_field => ["nova", "compute"]
add_tag => ["novametrics"]
}
mutate {
gsub => ['logmessage',""",""]
}
}
nova
nova compute
grok_error
...
if !("_grokparsefailure" in [tags]) {
grok {
add_tag => "grok_error"
}
}
...
Visualizing OpenStack logs
filebeat-*
filebeat-*
Y-Axis
X-Axis
fields.tags: dashboard and 50
b51498ca-0a59-42bd-945a-18246668186d
#rabbitmqctl status
ERROR neutron.agent.l3_agent [-] The external network bridge 'br-ex' does
not exist
8a7cc14f-5d97-43b9-9cc9-63db87e57ca0
...
Bridge br-ex
Port br-ex
Interface br-ex
type: internal
...
Chapter 9
memcached
http://docs.openstack.org/developer/keystone/configuration.htm
l
memcached
memcached
/etc/sysconfig/clock
memcached
/etc/sysconfig/memcached
CACHESIZE=4096
memcached
[token]
driver = keystone.token.persistence.backends.memcache.Token
caching = True
...
memcached
backend
[cache]
enabled = True
config_prefix = cache.keystone
backend = dogpile.cache.memcached
backend memcached
backend_argument = url:localhost:11211
keystone
memcached
memcached
get_hits
accepting_conns
bytes
bytes_read
bytes_written
cmd_get
cmd_set
get_hits
get_misses
nova.conf
Chapter 9
/etc/haproxy/haproxy.cfg
...
listen memcached-cluster 192.168.47.47:11211
balance roundrobin
maxconn 10000
mode tcp
server cc01 192.168.47.100:11211 check inter 5s rise 2 fall 3
server cc02 192.168.47.101:11211 check inter 5s rise 2 fall 3
server cc03 192.168.47.102:11211 check inter 5s rise 2 fall 3
cc01 cc02
/etc/nova/nova.conf
...
memcached_servers = cc01:11211,cc02:11211,cc03:11211
...
/etc/openstack-
dashboard/local_settings.py
...
CACHES = {
'default': {
'BACKEND' : 'django.core.cache.backends.memcached.
MemcachedCache',
'LOCATION' : '192.168.47.47:11211',
}
}
...
...
listen horizon 192.168.47.47:80
balance roundrobin
maxconn 10000
mode tcp
server cc01 192.168.47.100:80 cookie cc01 check inter 5s rise
2 fall 3
server cc02 192.168.47.101:80 cookie cc02 check inter 5s rise
2 fall 3
server cc03 192.168.47.102:80 cookie cc03 check inter 5s rise
2 fall 3
https://github.com/openstack/tempest
Chapter 2
install_rally.sh
keystone_admin
deployment check
/rally/sample/tasks/scenarios
ScenarioClass.scenario_method:
-
args:
...
runner:
...
context
...
sla:
...
ScenarioClass.scenario_method
args
runners
constant
constant_for_duration
periodic
serial
context
sla
KeystoneBasic.authenticate_user_and_validate_token
perf_keystone_pp.yaml
KeystoneBasic.authenticate_user_and_validate_token:
-
args: {}
runner:
type: "constant"
times: 50
concurrency: 50
context:
users:
tenants: 5
users_per_tenant: 10
sla:
failure_rate:
max: 1
abort-on-sla-failure
keystone_v2.fetch_token
keystone_v2.validate_token
...
sla:
max_avg_duration: 5
max_seconds_per_iteration: 5
failure_rate:
max: 0
performance_degradation:
max_degradation: 50
outliers:
max: 1
max_avg_duration
max_seconds_per_iteration
failure_rate
max
performance_degradation
max_degradation
outlier
max
mod_wsgi
http://eventlet.net/
/etc/httpd/conf.d/keystone_wsgi_main.conf 5000
<VirtualHost *:5000>
ServerName cc01
root
DocumentRoot "/var/www/cgi-bin/keystone"
<Directory "/var/www/cgi-bin/keystone">
Options Indexes FollowSymLinks MultiViews
AllowOverride None
Require all granted
</Directory>
/var/log/httpd
ErrorLog "/var/log/httpd/keystone_wsgi_main_error.log"
CustomLog "/var/log/httpd/keystone_wsgi_main_access.log" combined
WSGIDaemonProcessdirective
WSGIApplicationGroup %{GLOBAL}
WSGIDaemonProcess keystone_main display-name=keystone-main
group=keystone processes=4 threads=32 user=keystone
WSGIProcessGroup keystone_main
WSGIScriptAlias / "/var/www/cgi-bin/keystone/main"
mod_wsgi
import os
from keystone.server import wsgi as wsgi_server
name = os.path.basename(__file__)
application = wsgi_server.initialize_application(name)
</VirtualHost>
/etc/httpd/conf.d/keystone_wsgi_admin.conf 35357
<VirtualHost *:35357>
ServerName cc01
DocumentRoot "/var/www/cgi-bin/keystone"
<Directory "/var/www/cgi-bin/keystone">
Options Indexes FollowSymLinks MultiViews
AllowOverride None
Require all granted
</Directory>
/var/log/httpd
ErrorLog "/var/log/httpd/keystone_wsgi_admin_error.log"
CustomLog "/var/log/httpd/keystone_wsgi_admin_access.log" combined
WSGIDaemonProcess
WSGIApplicationGroup %{GLOBAL}
WSGIDaemonProcess keystone_main display-name=keystone-admin
group=keystone processes=8 threads=32 user=keystone
WSGIProcessGroup keystone_admin
WSGIScriptAlias / "/var/www/cgi-bin/keystone/admin"
import os
from keystone.server import wsgi as wsgi_server
name = os.path.basename(__file__)
application = wsgi_server.initialize_application(name)
</VirtualHost>
http://pyshaker.readthedocs.io/en/latest/index.html
pip
/venv//lib/python2.7/site-
packages/shaker/resources/image_builder_templates/
venv/lib/python2.7/site-packages/shaker/engine/config.py
IMAGE_BUILDER_OPTS
ubuntu centos debian
/lib/python2.7/site-
packages/shaker/scenarios
YAML
title: ...
description:
...
deployment:
template: ...
accommodation: [..., ...]
execution:
progression: ...
tests:
-
title: ...
class: ...
method: ...
-
...
compute_nodes
http://pyshaker.readthedocs.io/en/latest/usage.html#test-classes
/lib/python2.7/site-
packages/shaker/scenarios/openstack/
title: OpenStack L2
description:
Benchmark Layer 2 connectivity performance. The shaker scenario will
launch pairs of instances in separate compute node.
deployment:
template: l2.hot
accommodation: [pair, single_room, compute_nodes: 1]
execution:
progression: quadratic
tests:
-
title: Download
class: flent
method: tcp_download
-
title: Upload
class: flent
method: tcp_upload
-
title: Bi-directional
class: flent
method: tcp_bidirectional
-
title: UDP-Bursts
class: flent
method: bursts
flent
tcp_download
tcp_upload
tcp_biderctional
bursts
https://github.com/tohojo/flent/tree/master/flent/
/var/www/html/bench/
httpd
bench
python-croniter
json
var/www/html/bench/l2_iteration01.html
[DEFAULT]
…
global_physnet_mtu = 9000
…
advertise_mtu = True
[DEFAULT]
…
network_device_mtu = 9000
[DEFAULT]
…
network_device_mtu = 9000
[DEFAULT]
…
network_device_mtu = 9000
[ml2]
…
path_mtu = 9000
segment_mtu = 9000
53 160 161
56 388
54
53 401
55 401
53 401
55
307
54 23
78
22 320
321 322 21 314
83 316
84 230 316 317 318
84 316
74 314
78 322
316
52 316
56
58 62 133 162
70 162
60 163
65 166 167 168 169
60 169 170 172
58 165
52 164
60 164
110
163
281
56 110
110
151 110
156 112
109
14 150
153 125 127
14 18
153
150 152 103
14 103
154 155 103
183 103
46 235
28
89 107
91 107
90 50
50
75 50
83 50
85 50
84
87 91
76 92
82 94
82 91
85 176
216 293
230 119
15
163 131
163 393
163
163 391
216 393
393
136
136 58 62
136 70
136
136 68
66
46 384
45 53
48
199 200
280
199 200
300
307 198
105 198
115
426
15
59
131
362
360 361
360 51
359 16
358
364 366 367 22 319
363 319
372 256
368
369 370 256
367 256
374 376 377 378 15
379
357
358
282
354 355 283
357 283 284 286 288
355 289
356 299 300
355
355 293 295 298
357 289 290 291 292
356
134 277
385 277
413 282
136
121 267 268
272 66
272 65
268
272 48
269
270 271 87
273 88
273 87
269 270
122
273 125 127
276 123
274 123
273 122
275
22 123
22 241 243
22 121
121
242 121
242 114
242
242 86
242
201 202 203
266 121
307 309 310
268 205
23 205
85
73 206 207
110 116
208 210
104
196
104 152
105
406
77 79 15
13 139
84
11
46 230 276
76
78
78
79 157
77 157
78 157
78 157
78
15 15 157
157
158
159 160
388
158 159
225
221
226 227 228 229
221
225
221
26
221
13
181
184 185 186
393
280
23
276
103
269
178 182 186
100
267
267
226
268
39
274
138
413
350
280
30
277
31
278
219
281
219
279
218
218
152
92 112
113 116
34 112
34 122
32 17
34 17
60 82 103
34 18
18 18 104
18
20 18
21 18
20 82
20 15
19
21 17 173
23 173
22 18
22 17
23 17
23 18
20 18
183 173
381
104
241
255
137 241
104 243
257 258 260 262 264
225 256
212
217 218 94
218 219 95
220 94
21 181 94
13 98
138 88
138 163
139
139 131
135
403 393
406 407
403 405 307
408 409 410 412 413
416 417 35
120 35
293 119
78
243
191 192 246 247 249 251 253
244
272
15 272
13 15
34
83
134
213 134
214 134
135
196
196 197 184
230
176 136
13 137
137
128 130 137
131
131 145
128 146
130 146
270 391 146
85 231
86
85
320 65
314
63
86 98
85 73
31 274
319
25
22 104
313 314 220
217
402
402 193
231 191
36 184
255 183
256 195
256 177
256 218
256
22 193 194
78
110 194
221 182
184 201
183
183 204 207
183 201 202 203
191 192 205
184 206 207
183 208 210
179 180 181 203 204
181
177 178
112
277 300 282
301 302 304 413
305 307
193
362
181
103
51
177 178
152 406