2013年4月11日 星期四

MiCloud Chef Server Dataset

說明:

MiCloud Chef Server主機為Joyent所包裝,提供預設Chef Server相關功能,讓您可以透過網路上所提供的各式Cookbook/Recepie來快速部屬您的雲端環境。以下文章以MiCloud Chef Server為Chef Server,並透過user cookbook來進行Chef Client: Ubuntu server上的user設定部屬...

訂購:



在MiCloud上,您可以透過Web選單(或API)訂購Chef Server主機:
主機建立完成後,您可透過SSH連線進入主機...檢查chef套件,包含以下服務:
[root@Chef-Server-01 /opt/chef-local]# svcs -a | grep chef
online          7:16:47 svc:/pkgsrc/chef-expander:default
online          7:16:49 svc:/pkgsrc/chef-solr:default
online          7:16:50 svc:/pkgsrc/chef-server:default
online          7:16:50 svc:/pkgsrc/chef-server-webui:default

設定Chef Server

搞定KEY

設定Chef Server之private key相關檔案,這邊會用到的包含validation.pem與webui.pem,作用為webui服務與驗證使用。預設MiCloud Chef Server將設定檔案放置在/etc/chef中,這邊複製到~/.chef下,作為系統預設讀取用...
[root@Chef-Server-01 ~]# mkdir ~/.chef && cp /etc/chef/*.pem ~/.chef/

Knife設定

[root@Chef-Server-01 ~]# knife configure -i
WARNING: No knife configuration file found
Where should I put the config file? [/root/.chef/knife.rb] <--預設位置就ok
Please enter the chef server URL: [http://Chef-Server-01.local:4000] http://211.78.xxx.xxx:4000 <--在MiCloud開立的主機IP,Port預設為4000
Please enter a clientname for the new client: [root] Chef-Server-01  <-- 該chef server名稱,自取,但是之後PEM要以此命名
Please enter the existing admin clientname: [chef-webui]
Please enter the location of the existing admin client's private key: [/etc/chef/webui.pem] .chef/webui.pem
Please enter the validation clientname: [chef-validator]
Please enter the location of the validation key: [/etc/chef/validation.pem] .chef/validation.pem
Please enter the path to a chef repository (or leave blank):
Creating initial API user...
Created client[Chef-Server-01]
Configuration file written to /root/.chef/knife.rb
可透過knife client list檢視目前server上註冊的client數量
[root@Chef-Server-01 ~]# knife client list
  Chef-Server-01
  admin
  chef-validator
  chef-webui
方才的步驟,已經產生了一各叫做knife.rb的檔案,檢視一下,其實設定的部份...也可以手動修改
[root@Chef-Server-01 ~]# cat /root/.chef/knife.rb
log_level                :info
log_location             STDOUT
node_name                'Chef-Server-01'
client_key               '/root/.chef/Chef-Server-01.pem'
validation_client_name   'chef-validator'
validation_key           '/root/.chef/validation.pem'
chef_server_url          'http://211.78.xxx.xxx:4000'
cache_type               'BasicFile'
cache_options( :path => '/root/.chef/checksums' )
同時,程序中也建立了Chef-Server-01.pem檔案,這是剛剛設定的clientname加上附檔名.pem
[root@Chef-Server-01 ~]# cat ~/.chef/Chef-Server-01.pem
-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEA3rbftvEAqS1m6MRWvX6Xg3mjjQ0BZX0ycT+EVbSn9ZfOm97H
gDkWvfSVD59hBgfb0k6tTTlGUWeeHY37F4UVjwq2TFO+wWbUhgaXJWO0u/vseHD8
C/S2XqgyU6Qv3bMk3Ejl+wLlESQn5gGYLfs+7gDMJR6w99xubvNFQHUPUEQk+kvM
HT/U5DNwifvcmVodgdXUuEqG+Qv2ChDa24a4...................QhaLz1wNwRWzHolUtUx
OLZEbwKBgC1jpvjiCeaY+Qkp7qX5kqsJ+uF69KHEUP70gAmYfenVYmgfda8pBkWp
fa01GR/qF9WkMM2wbThaQmx/OqVD6uXI627rOhPVbUmUKWv/qLdhqwnlHSrS037o
Dxwh2L+0f7vNxFrbclpSVJ3GhldaMeRgIniMd2VaKtI4ghc9gABC
-----END RSA PRIVATE KEY-----
[root@Chef-Server-01 ~]#

新增Chef Workstation

這邊將Chef Server與Workstation裝在同一台,因此有許多設定可以共用,這邊直接透過指令建立chef-workstation,指令執行完,系統會吐出workstation的Private Key內容,請將此內容存入”~/.chef/chef-workstation.pem”
[root@Chef-Server-01 ~]# knife client create chef-workstation -d -a
Created client[chef-workstation]
-----BEGIN RSA PRIVATE KEY-----
MIIEogIBAAKCAQEAsz53YHrkPE0RRIyzwuRfFdt6jv3U3GygL6Sjz2DgCDVN1IbC
puzkKJ9FEWZjdcfjitrtHnja8WcwTu3deoAtjR7NnMpB1LF+xh4MtR/6lkjo2Yyl
yObecFAwnNjxiNNx5ajmCEEfH5ZVrQpzPR0vMrnZ6VOmUjyWG7dJ5od+1HHae8/3
lC9O78By8crZ+xkTBblTc58eKi8j9tcphzPKeDNbryngA38o9HVmlrM3nGNMT6mz
POWWxftQKcmwXRaRQrJSSA9MqOk0Q/VkRMYmJvkzxkqNN7PSkJwuNadflZEilKH6
B1crIjusXW6hkaj9yb/MUwnplgm.......OSRP0KeI9zx8sUjKwciBW4X5s4dpyh
gEgE6qGg8up6tpFeUbm1v8kD6QY5WXRrnZ0hdwwEw/swSUyZCRxDMcsXy1sndWhC
ME/bAoGAYK1JF83X+X3fbTRrwcoY8tA+VVelIRFrLy3m9qK/UkIKeH9Q6JOmBgvd
U7kRXoo/oyHNmYj3uKGEbNulOLdNtq8jsJno/wawb+t38lAcoRJCiKARI8zYzeiW
AZo11HmAEp7ZcJifJdzNJlkFtTwrN/mCurGvdxApZbWZJFgz7XU=
-----END RSA PRIVATE KEY-----
[root@Chef-Server-01 ~]#
(Save the chef-workstation private key output to ~/.chef/chef-workstation.pem)
建立完workstation後,可以透過show來顯示workstation的相關資訊:
[root@Chef-Server-01 ~]# knife client show chef-workstation
_rev:        1-1df01ef4d0bea06e131baca050d156d1
admin:       true
chef_type:   client
json_class:  Chef::ApiClient
name:        chef-workstation
public_key:  -----BEGIN PUBLIC KEY-----
             MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAsz53YHrkPE0RRIyzwuRf
             Fdt6jv3U3GygL6Sjz2DgCDVN1IbCpuzkKJ9FEWZjdcfjitrtHnja8WcwTu3deoAt
             jR7NnMpB1LF+xh4MtR/6lkjo2YylyObecFAwnNjxiNNx5ajmCEEfH5ZVrQpzPR0v
             MrnZ6VOmUjyWG7dJ5od+1HHae8/3lC9O78By8crZ+xkTBblTc58eKi8j9tcphzPK
             eDNbryngA38o9HVmlrM3nGNMT6mzPOWWxftQKcmwXRaRQrJSSA9MqOk0Q/VkRMYm
             JvkzxkqNN7PSkJwuNadflZEilKH6B1crIjusXW6hkaj9yb/MUwnplgmYe6TPPzBZ
             UQIDAQAB
             -----END PUBLIC KEY-----
至GitHub Clone一個chef repository專案

複製一個Chef Repository

Chef均是透過git的方式來維護一個一個的部屬設定,opscode提供一個基礎專案,讓Chef的使用者可以基於此專案開發後續的部屬設定...
[root@Chef-Server-01 ~]# git clone git://github.com/opscode/chef-repo.git /opt/chef-local
Cloning into '/opt/chef-local'...
remote: Counting objects: 209, done.
remote: Compressing objects: 100% (126/126), done.
Receiving objects:  71% (149remote: Total 209 (delta 75), reused 167 (delta 49)
Receiving objects: 100% (209/209), 35.05 KiB, done.
Resolving deltas: 100% (75/75), done.
檢視Clone下來的專案的內容:
[root@Chef-Server-01 ~]# cd /opt/chef-local/
[root@Chef-Server-01 /opt/chef-local]# ls -l
總計 33
drwxr-xr-x 2 root root     3 4月  11 08:06 certificates
-rw-r--r-- 1 root root   156 4月  11 08:06 chefignore
drwxr-xr-x 2 root root     3 4月  11 08:06 config
drwxr-xr-x 2 root root     3 4月  11 08:06 cookbooks
drwxr-xr-x 2 root root     3 4月  11 08:06 data_bags
drwxr-xr-x 2 root root     3 4月  11 08:06 environments
-rw-r--r-- 1 root root 10850 4月  11 08:06 LICENSE
-rw-r--r-- 1 root root  2171 4月  11 08:06 Rakefile
-rw-r--r-- 1 root root  3521 4月  11 08:06 README.md
drwxr-xr-x 2 root root     3 4月  11 08:06 roles
[root@Chef-Server-01 /opt/chef-local]# mkdir .chef
[root@Chef-Server-01 /opt/chef-local]# cp ~/.chef/validation.pem ~/.chef/chef-workstation.pem .chef/
(Chef workstation在執行建立Chef Client時候,需要連線Client作Client設定,因此這邊需產生 ssh key 並且寫到 chef clien的/root/.ssh/authorized_keys中....)

初始化Chef Client

透過knife bootstrap可以初始化Chef Client,其中最後一個參數為Chef Client的IP位置,這邊因為把ssh-kry認證加入,所以不用再帶入帳號與密碼
[root@Chef-Server-01 /opt/chef-local]# knife bootstrap 211.78.xxx.xxx
Bootstrapping Chef on 211.78.xxx.xxx
211.78.xxx.xxx [2013-04-11T08:10:45+00:00] INFO: *** Chef 10.18.2 ***
211.78.xxx.xxx [2013-04-11T08:10:46+00:00] INFO: Client key /etc/chef/client.pem is not present - registering
211.78.xxx.xxx [2013-04-11T08:10:46+00:00] INFO: HTTP Request Returned 404 Not Found: Cannot load node Ubuntu-Chef-Client-01
211.78.xxx.xxx [2013-04-11T08:10:46+00:00] INFO: Setting the run_list to [] from JSON
211.78.xxx.xxx [2013-04-11T08:10:46+00:00] INFO: Run List is []
211.78.xxx.xxx [2013-04-11T08:10:46+00:00] INFO: Run List expands to []
211.78.xxx.xxx [2013-04-11T08:10:46+00:00] INFO: HTTP Request Returned 404 Not Found: No routes match the request: /reports/nodes/Ubuntu-Chef-Client-01/runs
211.78.xxx.xxx [2013-04-11T08:10:46+00:00] INFO: Starting Chef Run for Ubuntu-Chef-Client-01
211.78.xxx.xxx [2013-04-11T08:10:46+00:00] INFO: Running start handlers
211.78.xxx.xxx [2013-04-11T08:10:46+00:00] INFO: Start handlers complete.
211.78.xxx.xxx [2013-04-11T08:10:46+00:00] INFO: Loading cookbooks []
211.78.xxx.xxx [2013-04-11T08:10:46+00:00] WARN: Node Ubuntu-Chef-Client-01 has an empty run list.
211.78.xxx.xxx [2013-04-11T08:10:47+00:00] INFO: Chef Run complete in 0.471286644 seconds
211.78.xxx.xxx [2013-04-11T08:10:47+00:00] INFO: Running report handlers
211.78.xxx.xxx [2013-04-11T08:10:47+00:00] INFO: Report handlers complete
完成初始化後,可在Chef server上執行list,檢視Chef client是否加入
[root@Chef-Server-01 /opt/chef-local]# knife client list
  Chef-Server-01
  Ubuntu-Chef-Client-01
  admin
  chef-validator
  chef-webui
  chef-workstation
root@Ubuntu-Chef-Client-01:~# ls -l /etc/chef/
client.pem       client.rb        first-boot.json  validation.pem


安裝Cookbook

Opscode提供一個cookbook的community:http://community.opscode.com/ 許多高手在此分享各式服務與架構的cookbook,下面展示安裝一個叫user的cookbook,相關資訊可以參考:http://community.opscode.com/cookbooks/user 
(cookbook folder need to be a git repository)
[root@Chef-Server-01 /opt/chef-local/cookbooks]# knife cookbook site install user
Installing user to /var/chef/cookbooks
ERROR: The cookbook repo /var/chef/cookbooks is not a git repository.
Use `git init` to initialize a git repo
預設的參數檔因為未指示cookbook放置的地方,因此執行時候會出錯,需修改knife.rb加入cookbook_path,並且要在cookbook folder完成git初始化的設定...
# vi ~/.chef/knife.rb
log_level                :info
log_location             STDOUT
node_name                'Chef-Server-01'
client_key               '/root/.chef/Chef-Server-01.pem'
validation_client_name   'chef-validator'
validation_key           '/root/.chef/validation.pem'
chef_server_url          'http://211.78.xxx.xxx:4000'
cache_type               'BasicFile'
cache_options( :path => '/root/.chef/checksums' )
cookbook_path [ '/opt/chef-local/cookbooks' ]  #<-- specify where the cookbooks
(first time install cookbook... need to do the first commit with the cookbook repostory)
[root@Chef-Server-01 /opt/chef-local/cookbooks]# knife cookbook site install user
Installing user to /opt/chef-local/cookbooks
ERROR: The default branch 'master' does not exist
If this is a new git repo, make sure you have at least one commit before installing cookbooks
(Error again, you need commit git....)
[root@Chef-Server-01 /opt/chef-local/cookbooks]# knife cookbook site install user
Installing user to /opt/chef-local/cookbooks
Checking out the master branch.
Creating pristine copy branch chef-vendor-user
Downloading user from the cookbooks site at version 0.3.0 to /opt/chef-local/cookbooks/user.tar.gz
Cookbook saved: /opt/chef-local/cookbooks/user.tar.gz
Removing pre-existing version.
Uncompressing user version 0.3.0.
removing downloaded tarball
1 files updated, committing changes
Creating tag cookbook-site-imported-user-0.3.0
Checking out the master branch.
Updating 337df3e..f8aaaf1
Fast-forward
 user/.gitignore                            |    2 +
 user/.travis.yml                           |    6 +
 user/CHANGELOG.md                          |   95 ++++++++++++
 user/README.md                             |  391 ++++++++++++++++++++++++++++++++++++++++++++++++
 user/Rakefile                              |   33 ++++
 user/attributes/default.rb                 |   42 ++++++
 user/metadata.json                         |   35 +++++
 user/metadata.rb                           |   14 ++
 user/providers/account.rb                  |  173 +++++++++++++++++++++
 user/recipes/data_bag.rb                   |   52 +++++++
 user/recipes/default.rb                    |   18 +++
 user/resources/account.rb                  |   40 +++++
 user/templates/default/authorized_keys.erb |    7 +
 13 files changed, 908 insertions(+)
 create mode 100644 user/.gitignore
 create mode 100644 user/.travis.yml
 create mode 100644 user/CHANGELOG.md
 create mode 100644 user/README.md
 create mode 100644 user/Rakefile
 create mode 100644 user/attributes/default.rb
 create mode 100644 user/metadata.json
 create mode 100644 user/metadata.rb
 create mode 100644 user/providers/account.rb
 create mode 100644 user/recipes/data_bag.rb
 create mode 100644 user/recipes/default.rb
 create mode 100644 user/resources/account.rb
 create mode 100644 user/templates/default/authorized_keys.erb
Cookbook user version 0.3.0 successfully installed
完成安裝”user”這個cookbook之後,會在cookbook目錄下多出一個user的目錄...

上傳Cookbook到Chef Server

上面Cookbook下載之後,需要依需求客製化欲建立的user及相關設定...當設定完成後,即可上傳該cookbook至Chef Server...
[root@Chef-Server-01 /opt/chef-local]# sudo knife cookbook upload user
Uploading user         [0.3.0]
Uploaded 1 cookbook.
此部份開始為user cookbook需進行的一些設定,詳細部分請參考user cookbook的github page:http://fnichol.github.io/chef-user/ 
設定server role...
[root@Chef-Server-01 /opt/chef-local]# cat -> /opt/chef-local/roles/ubuntu_servers.rb <<EOF
name "ubuntu_servers"
description "The base role applied to all nodes."
run_list(
    "recipe[user]",
    "recipe[user::data_bag]"
)
override_attributes(
    "users" => [ "ubuntu" ]
)
EOF
將server role上傳至Chef Server...
[root@Chef-Server-01 /opt/chef-local]# knife role from file roles/ubuntu_servers.rb
Updated Role ubuntu_servers!
建立user cookbook設定上需要的data_bag...
[root@Chef-Server-01 /opt/chef-local]# cd data_bags/
[root@Chef-Server-01 /opt/chef-local/data_bags]# mkdir users
[root@Chef-Server-01 /opt/chef-local/data_bags]# vim users/ubuntu.json
{
    "id"       : "ubuntu",
    "gid": "admin",
    "comment"  : "ubuntu",
    "home"     : "/home/ubuntu",
    "create_user_group":"false",
    "ssh_keygen": "false",
    "ssh_keys" : "ssh-rsa AAAAB3NzaC1yc2EAAAADAQASSAABAQDU+3VXGUANuuLVp3jeM3nQi23Odgz4mhiFI4PfDdlO3DtefSduikKC++PhvG2sdUxb+c7nxq+fq2GD8AEWltuUn3fYl0zdujr4zBbid/Blylp65PlT5ok5QwLuSwS8RtfQkyipocbAe3Jbtab13YLq/iUBlAkgD/GxfyZOaqs4tPdcn7DKfEuq81NjoKDkTV9GOUUpKi8ves4in/sB7eyFlrpm0+7mpwNh3cXNBRL8b3theIG3BLK7zbrUwTT6vcNETXt/wJAoO7cKQecFudhpqr8N/S?pSbflcRrw4GTjvkCZ58PylOXY1cUOfeeSvcxtNqsGPq8ZgtxRATlCBIAN simonsu@SimonAIR.local"
}
在data_bag下建立users databag...
[root@Chef-Server-01 /opt/chef-local/data_bags]# knife data bag create users
Created data_bag[users]
將上面建立的ubuntu.json文件上傳至databag...
[root@Chef-Server-01 /opt/chef-local/data_bags]# knife data bag from file users users/ubuntu.json
Updated data_bag_item[users::ubuntu]
在node run_list中建立ubuntu_servers的role...
[root@Chef-Server-01 /opt/chef-local/data_bags]# knife node run_list add Ubuntu-Chef-Client-01 "role[ubuntu_servers]"
run_list:  [role[ubuntu_servers]]

上述動作就完成Chef與Cookbook上之相關設定。

Client端拉取配置

當Chef Server上的設定完備後,client需要做的只有執行chef-client,將部屬設定拉下執行...
root@Ubuntu-Chef-Client-01:/etc/chef# chef-client
[2013-04-11T09:30:46+00:00] INFO: *** Chef 10.18.2 ***
[2013-04-11T09:30:47+00:00] INFO: Run List is [role[ubuntu_servers]]
[2013-04-11T09:30:47+00:00] INFO: Run List expands to [user, user::data_bag]
[2013-04-11T09:30:47+00:00] INFO: HTTP Request Returned 404 Not Found: No routes match the request: /reports/nodes/Ubuntu-Chef-Client-01/runs
[2013-04-11T09:30:47+00:00] INFO: Starting Chef Run for Ubuntu-Chef-Client-01
[2013-04-11T09:30:47+00:00] INFO: Running start handlers
[2013-04-11T09:30:47+00:00] INFO: Start handlers complete.
[2013-04-11T09:30:47+00:00] INFO: Loading cookbooks [user]
[2013-04-11T09:30:47+00:00] INFO: Storing updated cookbooks/user/resources/account.rb in the cache.
[2013-04-11T09:30:47+00:00] INFO: Storing updated cookbooks/user/providers/account.rb in the cache.
[2013-04-11T09:30:47+00:00] INFO: Storing updated cookbooks/user/recipes/default.rb in the cache.
[2013-04-11T09:30:47+00:00] INFO: Storing updated cookbooks/user/recipes/data_bag.rb in the cache.
[2013-04-11T09:30:47+00:00] INFO: Storing updated cookbooks/user/attributes/default.rb in the cache.
[2013-04-11T09:30:47+00:00] INFO: Storing updated cookbooks/user/metadata.json in the cache.
[2013-04-11T09:30:47+00:00] INFO: Storing updated cookbooks/user/README.md in the cache.
[2013-04-11T09:30:48+00:00] INFO: Storing updated cookbooks/user/metadata.rb in the cache.
[2013-04-11T09:30:48+00:00] INFO: Storing updated cookbooks/user/.gitignore in the cache.
[2013-04-11T09:30:48+00:00] INFO: Storing updated cookbooks/user/CHANGELOG.md in the cache.
[2013-04-11T09:30:48+00:00] INFO: Storing updated cookbooks/user/Rakefile in the cache.
[2013-04-11T09:30:48+00:00] INFO: Storing updated cookbooks/user/.travis.yml in the cache.
[2013-04-11T09:30:48+00:00] INFO: Processing user_account[ubuntu] action create (user::data_bag line 36)
...(Skip)

附註:

或許您會期望使用Cookbooks來部屬您的SmartOS或其他Linux Distribution,但是Cookbooks畢竟有支援上的限制,建議您詳細讀取Cookbook的提供說明與限制後,再實作於您的架構之上。