2013年4月8日 星期一

Yes, you are at a CLOUD... Initialize MiCloud Hadoop with 1 line...

  with



參考Hadoop官網之Single Node Setup,您可以透過一行指令碼簡單啟動Hadoop服務...

啟動Standalne模式,並直接執行預設範例(需要的話,可以直接下載Script做修改):

啟動Pseudo-Distributed Operation模式:

Standalone模式屬於範例的執行測試,在這邊就不執行測試...
下面顯示Pseudo-Distributed Operation模式...

===============Pseudo-Distributed Operation===============
Create config files ...
Edit .bashrc ...
===============Setup passphraseless ssh===============
Start to sync ssh key to local...
===============Execution===============
13/04/08 22:42:29 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = bddbe1f4-8a8b-4d6c-bab7-7421e62d605b.local/211.78.245.28
STARTUP_MSG:   args = [-format]
STARTUP_MSG:   version = 1.0.4
STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r 1393290; compiled by 'hortonfo' on Wed Oct  3 05:13:58 UTC 2012
************************************************************/
Re-format filesystem in /tmp/hadoop-root/dfs/name ? (Y or N) 13/04/08 22:42:29 INFO util.GSet: VM type       = 32-bit
13/04/08 22:42:29 INFO util.GSet: 2% max memory = 19.33375 MB
13/04/08 22:42:29 INFO util.GSet: capacity      = 2^22 = 4194304 entries
13/04/08 22:42:29 INFO util.GSet: recommended=4194304, actual=4194304
13/04/08 22:42:29 INFO namenode.FSNamesystem: fsOwner=root
13/04/08 22:42:29 INFO namenode.FSNamesystem: supergroup=supergroup
13/04/08 22:42:29 INFO namenode.FSNamesystem: isPermissionEnabled=true
13/04/08 22:42:29 INFO namenode.FSNamesystem: dfs.block.invalidate.limit=100
13/04/08 22:42:29 INFO namenode.FSNamesystem: isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s)
13/04/08 22:42:29 INFO namenode.NameNode: Caching file names occuring more than 10 times
13/04/08 22:42:29 INFO common.Storage: Image file of size 110 saved in 0 seconds.
13/04/08 22:42:29 INFO common.Storage: Storage directory /tmp/hadoop-root/dfs/name has been successfully formatted.
13/04/08 22:42:29 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at bddbe1f4-8a8b-4d6c-bab7-7421e62d605b.local/211.78.245.28
************************************************************/
starting namenode, logging to /usr/local/hadoop-1.0.4/libexec/../logs/hadoop-root-namenode-bddbe1f4-8a8b-4d6c-bab7-7421e62d605b.local.out
localhost: starting datanode, logging to /usr/local/hadoop-1.0.4/libexec/../logs/hadoop-root-datanode-bddbe1f4-8a8b-4d6c-bab7-7421e62d605b.local.out
localhost: starting secondarynamenode, logging to /usr/local/hadoop-1.0.4/libexec/../logs/hadoop-root-secondarynamenode-bddbe1f4-8a8b-4d6c-bab7-7421e62d605b.local.out
starting jobtracker, logging to /usr/local/hadoop-1.0.4/libexec/../logs/hadoop-root-jobtracker-bddbe1f4-8a8b-4d6c-bab7-7421e62d605b.local.out
localhost: starting tasktracker, logging to /usr/local/hadoop-1.0.4/libexec/../logs/hadoop-root-tasktracker-bddbe1f4-8a8b-4d6c-bab7-7421e62d605b.local.out
[root@bddbe1f4-8a8b-4d6c-bab7-7421e62d605b /usr/local/hadoop]#


同樣可透過ps來檢視服務啟動的狀況...

[root@bddbe1f4 /usr/local/hadoop]# ps -ef | grep java
    root 19887 91198   0 22:42:36 pts/1       0:03 /opt/local/java/sun6/bin/java -Dproc_jobtracker -Xmx1000m -Dcom.sun.management.
    root 19814 91198   0 22:42:35 ?           0:02 /opt/local/java/sun6/bin/java -Dproc_secondarynamenode -Xmx1000m -Dcom.sun.mana
    root 19988 91198   0 22:42:38 ?           0:02 /opt/local/java/sun6/bin/java -Dproc_tasktracker -Xmx1000m -Dhadoop.log.dir=/us
    root 20408 92006   0 22:45:16 pts/1       0:00 grep java
    root 19612 91198   0 22:42:31 pts/1       0:03 /opt/local/java/sun6/bin/java -Dproc_namenode -Xmx1000m -Dcom.sun.management.jm
[root@bddbe1f4 /usr/local/hadoop]#

註:上面執行腳本host於Github上,可依照需要做修改,修改玩之後,直接執行該script即可...