AGILE IT MANAGEMENTSCRUM, DEVOPS, LEAN IT prácticas de devops despliegue día 3 despliegue: entorno 1. gestión de la configuración: puppet 2. monitorización: nagios 3. despliegue: fabric gestión de configuración despliegue gestión de la configuración Name Language Mutual auth Encrypts Verify mode Ansible Python Yes Yes Yes CFEngine C Yes Yes Yes Chef Ruby Yes Yes Yes OCS Inventory NG with GLPI Perl, PHP, C++ No Yes ? Puppet Ruby Yes Yes Yes Rex Perl Yes Yes ? Rundeck Java Yes Yes Yes Salt Python Yes Yes Yes despliegue: puppet - práctica 1/6 ## 0. puppet agent: configuration sudo apt-get install puppetmaster ## 1. puppet server: config generation sudo puppetmasterd --genconfig > /etc/puppet/puppet.conf ## 2. puppet server: run in console sudo puppet master --verbose --no-daemonize # 3. puppet server: certificates sudo puppet cert --list sudo puppet cert sign --all # 3. puppet server: installing modules sudo puppet module search puppetlabs sudo puppet module install ... https://forge.puppetlabs.com/ ## 1. puppet agent: configuration sudo apt-get install screen vim git puppet sudo echo "<IP> puppet" >> /etc/hosts ping puppet sudo puppet agent --test ## 2. puppet agent: enable sudo puppet agent --enable despliegue: puppet - práctica 2/6 manifests/site.pp import 'nodes/*.pp' $puppetserver = 'puppet.example.com' ########## demo file: /tmp/hello ### file { '/tmp/hello': ensure => present, content => "Hello, world. Written by carlessanagustin.com \n", } >>>>>>> ########## MUST ### package { 'ntp': ensure => installed } package { 'screen': ensure => installed } package { 'vim': ensure => installed } package { 'git': ensure => installed } package { 'puppet': ensure => installed } despliegue: puppet - práctica 3/6 manifests/nodes/vagrant. pp node 'vagrant' { ########## apache2 ### class { "apache": } ########## python ### class { 'python': version => 'system', dev => true, virtualenv => true, gunicorn => true, } >>>>>>> ########## django ### package { 'python-django': ensure => installed } package { 'python-testtools': ensure => installed } package { 'python-django-nose': ensure => installed } } despliegue: puppet - práctica 4/6 manifests/nodes/upc001.pp node 'upc001' { include mysite package { 'python-virtualenv': ensure => installed } } despliegue: puppet - práctica 5/6 modules/mysite/manifests/init.pp class mysite { include python package { "flake8": ensure => installed, provider => pip; } } despliegue: puppet - práctica 6/6 modules/python/manifests/init.pp class python { package { "python": ensure => latest; "python-dev": ensure => "2.7.5-5ubuntu1"; "python-setuptools": ensure => installed; "python3": ensure => latest; 'python-pip': ensure => installed; 'python3-pip': ensure => latest; } monitorización despliegue monitorización de sistemas y redes Name IP SLA Reports Trending Trend Prediction Auto Discovery Agentless SNMP Syslog Cacti Yes Yes Yes Via plugin Yes Yes Yes Ganglia No Yes No Via gmond check in No Via plugin No Monitorix No Yes Yes No No Yes No Munin No Yes Yes No No Yes No Nagios Via plugin Yes No Via plugin Supported Via plugin Via plugin Opsview Yes Yes No Yes Yes Yes Yes Solarwinds Yes Yes Yes Yes Partial Yes Yes Zabbix Yes Yes No Yes Supported Yes Yes monitorización: nagios - práctica @ server ## install $ sudo apt-get -y nagios3 ## test cfg command $ nagios3 -v /etc/nagios3/nagios.cfg ## service control $ sudo service nagios3 restart @ agent ## install $ sudo apt-get -y install nagios- plugins nagios-nrpe-server nagios- nrpe-plugin monitorización: nagios - práctica: monitoreo del servicio web define host { use linux-server host_name sparta.naginet alias sparta address 10.00.00.00 } define service { use generic-service host_name sparta.naginet service_description HTTP check_command check_http } define service { use generic-service host_name sparta.naginet service_description HTTPS check_command check_http!-S max_check_attempts 3 check_interval 5 retry_interval 1 check_period 24x7 notification_interval 60 notification_period 24x7 contacts nagiosadmin } monitorización: nagios - práctica: ejecución automática @ server: /etc/nagios3/conf. d/services/mysql_nrpe.cfg define service { use generic-service host_name roma.naginet service_description MYSQL check_command check_nrpe! check_mysql } @ agent: /etc/nagios/nrpe.d/upctalent.cfg command[check_mysql] =/usr/lib/nagios/plugins/check_mysql -u nagios - d nagios -p yourpassword ------ ## create database and user mysql> CREATE DATABASE nagios; mysql> CREATE USER 'nagios'@'<IP_here>' IDENTIFIED BY 'yourpassword'; mysql> GRANT ALL PRIVILEGES ON nagios.* to nagios@localhost; ## test mysql --host=<IP_here> --user=nagios -- password=yourpassword monitorización: nagios - práctica: contactos y periodos define contact { use generic-contact contact_name pepito alias pepito garcia email
[email protected] contactgroups ops } define contactgroup { contactgroup_name ops alias Network operators } define service { ... contacts pepito, joselito } define service { ... contact_groups ops check_period 24x7 notification_period workhours } define timeperiod { timeperiod_name workhours alias Normal Work Hours monday 09:00-17:00 tuesday 09:00-17:00 wednesday 09:00-17:00 thursday 09:00-17:00 friday 09:00-17:00 } despliegue del código despliegue herramientas de despliegue Agent requs. Communicati on Language Web GUI Remote commands Push ControlTier SSH Server, JRE SSH Java Yes Yes Yes Capistrano Ruby SSH Ruby on Rails Yes Fabric SSH server SSH Python No Yes Yes Despliegue de aplicaciones web: Fabric - fabfile.py from fabric.api import * env.hosts = ['localhost'] env.user = "vagrant" def update_django_project(): with cd('/tmp'): run('echo carlessanagustin.com >> hello.txt') # with prefix('source /path/to/your/virtualenv/bin/activate'): # run('pip install -r pip_requirement_file.txt') # run('python manage.py cart') def update_django_project_fake(): run('uname -s') def restart_webserver(): """ Restarts remote nginx and uwsgi. """ sudo("service uwsgi restart") sudo("/etc/init.d/nginx restart") def deploy(): update_django_project_fake() #restart_webserver() Despliegue de aplicaciones web: Capistrano - deploy.rb 1/3 ## capistrano 2.5.14 # start: cap install or capify . require "rvm/capistrano" set :rvm_ruby_string, '1.9.3@yoursite' set :rvm_type, :user require 'bundler/capistrano' require './config/deploy_tasks/console_tasks' begin case targetname when 'pre' then set :rails_env, 'pre' set :branch, !exists?(:branch) ? 'pre' : "#{branch}" role :web, "10.10.00.1" role :app, "10.10.00.1" role :db, "10.10.00.1", :primary => true when 'www' then set :rails_env, 'production' set :branch, !exists?(:name_of_dir) ? 'master' : "# {branch}" role :web, "10.10.00.1" role :app, "10.10.00.1" role :db, "10.10.00.1", :primary => true else raise Capistrano::Error, "Your targetname is invalid, please use [pre|database]" end rescue raise Capistrano::Error, "You need to pass -S targetname=[pre|www]\n\nFor example:\n cap -S targetname=pre deploy" end Despliegue de aplicaciones web: Capistrano - deploy.rb 2/3 set :application, "#{targetname}.yoursite.com" set :deploy_to, "/home/web/apps/#{application}" set :user, "web" ## ssh login set :runner, "web" ## app runner set :repository, "
[email protected]:yoursite.git" set :deploy_via, :remote_cache set :copy_exclude, [".git", ".gitignore"] ## excluded files + folder set :scm, :git set :use_sudo, false ssh_options[:forward_agent] = true default_run_options[:pty] = true # Unicorn stuff set :unicorn_binary, "bundle exec unicorn_rails" set :unicorn_config, "#{current_path}/config/unicorn.rb" set :unicorn_pid, "#{current_path}/tmp/pids/unicorn. pid" ## capistrano recipe ## https://github. com/capistrano/capistrano/wiki/Capistrano-Tasks namespace :deploy do task :start, :roles => :app, :except => { :no_release => true } do run "cd #{current_path} && #{try_sudo} # {unicorn_binary} -c #{unicorn_config} -E #{rails_env} -D" end task :stop, :roles => :app, :except => { :no_release => true } do run "#{try_sudo} kill `cat #{unicorn_pid}`" end Despliegue de aplicaciones web: Capistrano - deploy.rb 3/3 task :graceful_stop, :roles => :app, :except => { : no_release => true } do run "#{try_sudo} kill -s QUIT `cat #{unicorn_pid}`" end task :reload, :roles => :app, :except => { :no_release => true } do run "#{try_sudo} kill -s USR2 `cat #{unicorn_pid}`" end task :restart, :roles => :app, :except => { :no_release => true } do stop start end end Despliegue de aplicaciones web: Capistrano - unicorn.rb if ENV['RAILS_ENV'] and ENV['RAILS_ENV'] == 'production' rails_env = 'www' listen "/tmp/.unicorn-www.sock" else rails_env = 'pre' listen "/tmp/.unicorn-pre.sock" end worker_processes 4 working_directory "/home/web/apps/#{rails_env}. yoursite.com/current/" timeout 3000 pid "/home/web/apps/#{rails_env}.yoursite. com/current/tmp/pids/unicorn.pid" stderr_path "/home/web/apps/#{rails_env}.yoursite. com/current/log/unicorn.stderr.log" stdout_path "/home/web/apps/#{rails_env}.yoursite. com/current/log/unicorn.stdout.log" preload_app true GC.respond_to?(:copy_on_write_friendly=) and GC.copy_on_write_friendly = true before_fork do |server, worker| defined?(ActiveRecord::Base) and ActiveRecord::Base.connection.disconnect! end after_fork do |server, worker| defined?(ActiveRecord::Base) and ActiveRecord::Base.establish_connection end Despliegue de aplicaciones web: Capistrano - console_tasks.rb Capistrano::Configuration.instance(:must_exist).load do namespace :rails do desc "Remote console" task :console, :roles => :app do ## function called after these lines.... run_interactively "bundle exec rails console # {rails_env}" end end def run_interactively(command, server=nil) ## propia de capistrano server ||= find_servers_for_task(current_task).first port = exists?(:port) ? fetch(:port) : 22 exec %Q(ssh -l #{user} #{server.host} -p #{port} -t "rvm_path=/home/web/.rvm /home/web/.rvm/bin/rvm- shell '#{rvm_ruby_string}' -c 'cd #{current_path} && # {command}'") end end ejemplos reales día 3 ejemplo 3.1: Andy Sykes on monitoring @ forward3d ● sensu: http://sensuapp.org/ ● graphite: http://graphite.readthedocs.org/en/latest/ ● flapjac: http://flapjack.io/ fuente: https://www.youtube.com/watch?v=Q9BagdHGopg ejemplo 3.2a: python stack (oriol rius) ● openam (AuthN, AuthZ, Accounting, Auditing) ● python + flask + eve ● rabbitmq ● celery ● mongodb, influxdb ● bitbucket + jira + confluence ● fluentd + logstash + elastichsearch + kibana ● collectd + graphite + skylive + oculus + grafana ● haproxy + varnish + nginx + gunicorn ● nexica cloud ● ovh bare metal ● ansible ● vagrant + docker fuente: http://oriolrius.cat/ ejemplo 3.2b: python stack (oriol rius) fuente: http://oriolrius.cat/ ejemplo 3.3: python stack (randall degges) ● python + django ● rabbitMQ (queueing) ● celery (task queue) ● memcached ● mysql ● github ● puppet ● monit + munin + nagios ● haproxy ● nginx ● gunicorn ● jenkins ● rackspace fuente: http://www.rdegges.com/devops-django-part-2-the-pain-of-deployment/ Muchas gracias Carles San Agustin
[email protected] www.carlessanagustin.com +34 655 441 378