Fclose.com feeds

Converting Video Files for iPod/iPhone/iPad

Fri, 03 Feb 2012 20:40:21 PST

We usually have video files in .wmv .mpg .rmvb or .mkv formats. The iPod, iPhone or iPad only accept a limited type of video files, such as .mov and .mp4. To play these video files in .wmv .mpg or .rmvb format, we should first convert them to .mov or .mp4 files that can played by iTune or other video players on these Apple devices that use the same codec engine. We may possibly convert video files for iPod or iPhone with mplayer/mencoder but it may have some problems, such as the audio and video do not synchronize very well for some videos.ffmpeg can convert video files from various formats to others and most of the time ffmpeg works very well. This post introduces how to convert the video files in common formats (.wmv, .mpg, .rmvb, .mkv, .etc.) to .mp4 format for iPod/iPhone/iPad.

Install ffmpeg

First, we should install the ffmpeg package. The Fedora Linux does not include ffmpeg in its repository for some reasons but the RPM Fusion repository provides it for us. To install the ffmpeg package, add RPM Fusion repository first.

The install the ffmpeg package:

# yum install ffmpeg

Convert video file format with ffmpeg

The basic command to convert video file input.wmv to onput.mp4 is as follows.

# ffmpeg -i input.wmv output.mp4

Or other format (such as .mov) by:

# ffmpeg -i input.wmv output.mov

ffmpeg determines the output video’s format by its file name extersion.

Convert video file format with ffmpeg for iPod/iPhone/iPad

The files in .mp4 format can be played on Apples devices. The iPod/iPhone/iPad have specific resolution, so we need to to convert the video files to a very high resolution since the devices can only display a limited one. We use the iPod generation 2 as the example, its screen resolution is 480×320. Hence, we can only convert the video to this resolution for smaller video file size. Similarly, the bitrate can also be set accordingly. The default one of ffmpeg (200 kb/s) is too low. To improve the quality of the converted video, we can set it to a larger one.

One good configuration for me is:

# ffmpeg -i input.wmv -s 480x320 -b 1000k output.mp4

-s sets the frame size of the video. -b sets the bitrate of the video.

Convert video file format with ffmpeg for iPod/iPhone/iPad in one command

More conveniently, we can form this command to a script convert2mp4.sh:

#!/bin/bash
ffmpeg -i $1 -s 480x320 -b 1000k /tmp/$1.mp4

Each time to convert a video file video.wmv, we can simply run:

$ convert2mp4.sh video.wmv

and the converted file is /tmp/video.wmv.mp4 after the script finishes.

Hadoop Default Ports

Sat, 14 Jan 2012 23:18:22 PST

Hadoop’s namenode and datanodes expose a bunch of TCP ports used by Hadoop’s daemons to communicate to each other or listen directly to users’ requests. These ports information are needed by both the Hadoop users and cluster administrators to write programs or configure firewalls/gateways accordingly.

A post written by Philip Zeyliger from Cloudera’s blog summarizes the default ports used by Hadoop:

Hadoop Default Ports Quick Reference.

Please note that these ports are also configuration through Hadoop’s configuration, such as the DFS and jobtracker’s ports in conf/core-site.xml conf/mapred-site.xml as used in the Hadoop Installation Tutorial.

Simple Sort Benchmark on Hadoop

Fri, 06 Jan 2012 08:55:00 PST

After installing Hadoop, we usually run some benchmark programs to test whether the system works well. In the post of Hadoop install tutorial, we show a very simple to grep strings from a simple sets of files. In this post, we introduce the Sort for testing and benchmarking Hadoop. The Sort program is also included in the Hadoop distribution package, and the package also includes a input data generator which generate 10 GB * (number of slave nodes) input data to sort. This program processes larger a datasets, which gives some strength to Hadoop including the execution engine and HDFS.

The Sort example program simply uses the MapReduce framework to sort the input directory into the output directory. The mapper is the predefined IdentityMapper and the reducer is the predefined IdentityReducer, both of which just pass their inputs directly to the output. The inputs and outputs must be Sequence files where the keys and values are BytesWritable.

The RandomWriter example program writes 10 GB (by default) of random data per host to HDFS using MapReduce. Each map takes a single file name as input and writes random BytesWritable keys and values to the DFS sequence file. The maps do not emit any output and the reduce phase is not used.

For a quick test of the Sort benchmark, just execute these two commands after setting up and starting the Hadoop (here we are in the Hadoop directory. If run the commands outside the Hadoop directory, simply use the full/relative path for the jar file):

# hadoop jar hadoop-*-examples.jar randomwriter rand
# hadoop jar hadoop-*-examples.jar sort rand rand-sort

The first command generates the random data into rand and the second commands sorts the generated data in rand and the result is put into rand-sort.

For more details and more options of the Sort and RandomWriter example programs, please refer to the Hadoop Wiki: Sort and RandomWriter.

Guangdong Mahjong Scoring Rules

Fri, 30 Dec 2011 03:03:13 PST

广东麻将规则

本文整理介绍广东麻将的鸡平胡规则及积分规则。广东麻将的规则体系发展历史源远流长，如今已经很难考证。其中鸡平胡规则是广东珠三角地区普及广且流行的一种打法，讲究鸡胡、平胡、爆胡，打发简单节奏快速。此文仅供参考，并不保证一定准确。

另：推荐Mahjong Time的香港麻将积分规则（注：与本文中规则略有不同）。

基本规则

牌数

共一百三十六张：筒、索、万，东、南、西、北风，中、发、白，没有梅、兰、竹、菊、春、夏、秋、冬。

摸牌

游戏在一开始，只有庄家可得到十四张牌，其余的人十三张。庄家从牌中选出一张牌丢出。此时，其它三家都有权力要那张丢出的牌。庄家的下家（右手边的玩者），有权力吃或碰那张牌，其它两家则只可碰或杠那张牌。 “ 碰 ” 比 “ 吃 ” 优先。

和牌

即流局、荒牌。按照既定的规则将手中的牌凑成相应的组合，获得胜利。和牌规则在下文计分规则中同有介绍。

执位

在四位玩家开始游戏时，随机或商定分配风位。按照东、南、西、北的次序逆时针排列。

庄家

1、东风位玩家为庄家，第一盘可随机或商定分配风位。
2、如果庄家和牌（流局）则下盘依然由该庄家做庄。
3、如果非庄家胡牌则下一盘由庄家的下家玩家做庄。

风圈

1、当最后一位玩家做庄之牌局完结而没有连庄情况出现，则牌局的一个循环完结，称为“一圈”。
2、第一圈风圈为东。
3、每打完一圈，则按照东、南、西、北的次序转换风圈。

漏胡

若玩家A为叫胡的状态下，如下家B打出一张玩家A可以胡的牌而玩家A却放弃不胡，那若对家C或上家D打出相同的一张牌时，玩家A是不能胡那一张的。除非玩家A有进行动牌的情况，动牌的意思在于碰、上、杠、模牌的。当有动牌后，那不能胡那一张的规则便解除。

一炮多响

允许一炮多响，如果一炮多响包括庄家在内胡牌，下一轮庄家继续当庄。

番

广东麻将的特色就在“番”，番是2倍的意思

爆胡（封顶）

意思即为多少番为顶，不再叠加番数进行计算，但特殊牌除外，比如十三幺，可突破爆胡限制，这些特殊牌型不再叠加其他番种。爆胡为3番。

记分规则

术语名称

刻子：三张相同的组在一起。

将牌：一对牌的意思，任意两张相同的牌组在一起都可以成为将牌。

幺九牌：指序数牌的1、9牌和字牌。

积分规则

一、爆胡（封顶）以内的牌型番种

牌型	番数	分值	说明
鸡胡	0	1	什么牌都可以胡，可吃碰杠
平胡	1	2	全部都是顺子没有刻子
碰碰胡	2	4	全部是刻子没有顺子
混一色	2	4	整副牌由字牌及另外单一花色（筒、条或万）组成

二、爆胡以外的牌型（特殊牌型）

牌型	番数	分值	说明
清一色	4	16	整副牌由同一花色组成
混碰	4	16	混一色 + 碰碰胡
清碰	5	32	清一色 + 碰碰胡
混幺九	5	32	由幺九牌和字牌组成的牌型
小三元	5	32	拿齐中、发、白三种三元牌，但其中一种是将
小四喜	5	32	胡牌者完成东、南、西、北其中三组刻子，一组对子
字一色	6	64	由字牌组合成的刻子牌型
清幺九	6	64	只由幺九两种牌组成的刻子牌型
大三元	6	64	胡牌时，有中、发、白三组刻子
大四喜	6	64	胡牌者完成东、南、西、北四组刻子
九莲宝灯	6	64	同种牌形成 1112345678999 ，再摸到该种牌任何一张即可胡牌，不计清一色
十三幺	6	64	1 、 9 万筒索，东、南、西、北、中、发、白；以上牌型任意一张牌作将

附加情况

1 、自摸

在爆胡以内自摸，+1番，爆胡以外不加番。

2 、风位

拿到本盘门风的刻子，+1番，爆胡以外不加番。

3 、风圈

拿到该风圈局的刻子，+1番，爆胡以外不加番。

3、三元牌

红中、白板、发财，任意一个刻子为1番，两个刻子为2番，爆胡以外不加番。附加情况只对爆胡内牌型有效，而且必须在爆胡内牌型胡了后才能以加番牌型累计番数，爆胡内牌型加附加牌型不能冲破爆胡的上限。

特殊规则

1、一炮三响

同一张牌点三个炮，点炮者包三家。胡牌者不满3番牌型，按照3番满计算，大于3番的牌型，按照实际牌型计算。

2、杠上开花

杠牌后在正常摸牌的牌墙的另一端补牌时，刚好摸到自摸需要的牌从而胡牌。不满3番牌型，按照3番满计算，大于3番的牌型，按照实际牌型计算。

3、杠上开花包杠

开暗刻明杠补牌后胡牌，计自摸，点杠者包三家牌。不满3番牌型，按照3番满计算，大于3番的牌型，按照实际牌型计算。

4、海底捞月

摸最后一张牌胡牌。不满3番牌型，按照3番满计算，大于3番的牌型，按照实际牌型计算。

5、包自摸规则

十二张落地包自模，包先不包后。例一：若玩家A打出一张牌让其中一名玩家做成十二张落地（即已碰/吃/明杠出的牌共有12张牌）而做成该玩家为单吊一张叫胡的情况时，该名打出者A必须承担包自模的责任，即若该名玩家在12张落地的情况时自模胡牌，该打出让玩家成立12张落地的玩家A便需要为其余两家代付该笔自模的筹码。例二：玩家A放一张让玩家B 12张落地达成，及后玩家A再放一张让玩家C或 D 12张落地达成，此时玩家A只需要承担包玩家B自模的责任，不需要承包玩家C和D。

6、特殊牌型包自摸

大四喜：玩家已碰出东、南、西、北，任意3款。若有人打出余下一款者，那人则需承担包自模的责任。
大三元：玩家已碰出红中、白板、发财，任意2款．若有人打出余下一款者，那人则需承担包自模的责任。

7、抢杠

明刻开明杠，可以被抢杠胡，抢杠胡等同杠开，当自摸，被抢杠者（欲开杠者）包三家牌，暗杠不可抢杠胡。抢杠者牌型不满3番牌型，按照3番满计算，大于3番的牌型，按照实际牌型计算。

暗刻开明杠，不算抢杠胡，胡牌者只能胡点杠那张牌，只算普通胡牌。

8、天和

只有庄家独有的权利，庄家起牌后即成和牌牌型。按6番计算。

9、人和

起牌后，庄家打出的第一张牌，闲家吃和。按6番计算。

10、地和

闲家抓到最初的13张牌时已经听牌，第一圈里闲家自摸的第一张牌和牌。按6番计算。注：第一圈里如有人吃、碰、杠牌，已不生效，不成地和。

Hiking Ngong Ping Trail – Ngong Ping Village to Tung Chung City Center

Sun, 11 Dec 2011 06:12:21 PST

My wife and I went for a hiking on Nov. 16, 2011. We walked downhill from Ngong Ping (昂坪) Village to Tung Chung (東涌) city center along the Ngong Ping Trail (昂坪棧道). It took us about 3 hours to reach the Tung Chung city center (also the starting point of the Ngong Ping 360). The trail is not hard and the we can have sight view along the way. The Ngong Ping Trail is used as the access for maintenance and rescue trail for Ngong Ping 360 with much of it directly under the path of the cable cars. Although it is not designed for hiking, the Ngong Ping trail is actually a nice hiking path especially for those who do not want to walk too long.

We first take the Ngong Ping 360 cable car from Tung Chung to Ngong Ping. After having a sight view and lunch at the Ngong Ping Village, we started our walk along the Ngong Ping Trail. (We walked out of the Ngong Ping Village across a gate opposite to the Starbucks and got to the ending point of Ngong Ping Trail after a short walk. It is close to the Ngong Ping 360 cable car station).

After 3 hours’ walk, we went off the last maintain and went across the “river” (or sea) by a bridge (the one in the photo that appears at the beginning of the post). Then we walk to the Yat Tung Estate (東涌逸東鸷) and took a bus to the Tung Chung MTR station.

For more information about the trail: this page provides a map and some photos along the way (it follows a reverse direction and a different route from us for Ngong Ping to Tung Chung); the Ngong Ping 360 official website provides some suggested routes.

Some photos that I took along the way are as follows.

Hosting from Dreamhost

Thu, 24 Nov 2011 08:19:19 PST

I just registered an account on dreamhost with this year’s Black Friday promotion (if it does not appear automatically, try the code 777thanks11).

One-year hosting plus one free .com domain costs me only $9.24. Super good!

Just created a new domain name GoPeta.com with this one-year hosting plan. Will report more details about my experience with dreamhost.

Upgrading Fedora to Newer Version with Yum

Mon, 21 Nov 2011 05:16:21 PST

Please consider PreUpgrade for upgrading Fedora. This is the recommended method.

This post makes a list of actions should be done to upgrade Fedora using yum. Please note that this is for experienced users only, if you are not sure what you are doing and what do these commands mean, please stop here and read the manuals first, or go to Fedora’s Wiki about upgrading Fediora using yum.

This post aims to be a general method, although we use Fedora 15 to Fedora 16 as the example. We configure and install grub2 as the bootloader.

Clean Stuff

It is a good time to remove packages we don’t use. These packages could be candidates for removal, but check to see whether you use them directly or if they are used by applications not backed by rpm packages.

# package-cleanup --leaves

You can find orphaned packages (i.e. packages not in the repositories anymore). This will also show packages which have been partially uninstalled but where the “%postun” script failed.

# package-cleanup --orphans

Do the upgrade

Go to a text console and run in level 3

ctrl + alt + F2

init 3

Update yum first

yum update yum

Clean all cached meta data and packages

yum clean all

Import the key for the newer version’s repository

For Fedora 16, it is

rpm --import https://fedoraproject.org/static/A82BA4B7.txt

Synchronizes the installed package set with the latest packages available

yum --releasever=<release_number_you_want_to_sync_to> distro-sync

For upgrading Fedora 15 to Fedora 16:

yum --releasever=16 --disableplugin=presto  distro-sync

Double check the updated repositories after distro-sync

yum repolist

Update groups

yum groupupdate Base

List all other groups

yum grouplist

and also update them

yum groupupdate ALL GROUPS LISTED

Use grub2 instead of grub

Make configuration for grub2

/sbin/grub2-mkconfig -o /boot/grub2/grub.cfg

Install grub2

/sbin/grub2-install BOOTDEVICE

For example, to install grub2 on sda:

/sbin/grub2-install /dev/sda

Reset services start/stop priorities

cd /etc/rc.d/init.d; for f in *; do /sbin/chkconfig $f resetpriorities; done

Reboot the system now

After reboot, you may see new kernel version on new Fedora version. For example, kernel 3.1.1 on Fedora 16

[root@localhost ~]# uname -a
Linux localhost.localdomain 3.1.1-1.fc16.x86_64 #1 SMP Fri Nov 11 21:47:56 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux

Additional Repositories for Fedora Linux

Thu, 17 Nov 2011 19:32:19 PST

Fedora’s repositories provide most of the software packages needed when using Linux. However, for various consideration, they do not provide some packages, such as MPlayer, ffmpeg. Fortunately, some community maintained repository provides these software. And they may also package propriety software, , such as ATI fglrx dive, NVIDIA driver, into rpm packages. In this post, we introduce theses additional repositories and how to install them into Fedora.

RPM Fusion

“RPM Fusion provides software that the Fedora Project or Red Hat doesn’t want to ship. That software is provided as precompiled RPMs for all current Fedora versions and Red Hat Enterprise Linux 5 and 6; you can use the RPM Fusion repositories with tools like yum and PackageKit.”

RPM Fusion has two separate software repositories. The free for Open Source Software (as defined by the Fedora Licensing Guidelines) which the Fedora project cannot ship due to other reasons. The nonfree for redistributable software that is not Open Source Software (as defined by the Fedora Licensing Guidelines); this includes software with publicly available source-code that has “no commercial use”-like restrictions.

To enable access to both the free and the nonfree repository use the following command.

Released and currently supported Fedora versions:

# yum localinstall --nogpgcheck http://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-stable.noarch.rpm
# yum localinstall --nogpgcheck http://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-stable.noarch.rpm

Fedora Rawhide and what will become the next Fedora version (Alpha, Beta and snapshots):

# yum localinstall --nogpgcheck http://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-rawhide.noarch.rpm
# yum localinstall --nogpgcheck http://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-rawhide.noarch.rpm

How to Install, Run and Uninstall VMware Player and VirtualBox on Fedora Linux

Thu, 17 Nov 2011 16:46:13 PST

VMware Player and VirtualBox are two cool and free full virtualization solutions and both can run on top of a Linux host. In this post, we introduces how to install, run, and uninstall VMware Player and VirtualBox on Fedora Linux.

VMware Player

Install VMware Player

Download the installation bundle from VMware’s website. For example, the file we download is: VMware-Player-4.0.0-471780.x86_64.bundle .

Install needed kernel header and devel packages

# yum install kernel-headers kernel-devel

Run the bundle file

# sh VMware-Player-4.0.0-471780.x86_64.bundle

Then just follow the instruction of the GUI of the VMware installer, and you can finish the installation.

For Fedora 16 with kernel 3.1.0 or 3.1.1 with VMware Player 4.0.0, the vmnet module can not compile vmnet module. The VMware-Player-4.0.1-528992.x86_64 fixes this problem and works well with 3.1 kernel. If you want to run VMware Player 4.0.0, the community give patches to it so that it works with kernel 3.1. The patch and script given by Stefano Angeleri works well for me. You may use it quickly by the following commands.

$ wget http://weltall.heliohost.org/wordpress/wp-content/uploads/2011/09/vmware8linux31fix.tar.gz
$ tar xf vmware8linux31fix.tar.gz
# sh ./patch-modules_3.1.0.sh

systemctl prints a “[FAILED]” message. But you can ignore it and VMware Player works.

Run VMware Player

Just execute

$ vmplayer

Start a virtual machine just by one command

$ vmplayer /path/to/virtual/machine/config.vmx &

Give the path to the virtual machine configuration file (a .vmx file under the virtual machine’s directory) to vmplayer as its parameter.

Uninstall VMware Player

VMware installer provides the method to uninstall VMware products. We can use this tool

# vmware-installer  --uninstall-product vmware-player

It has a GUI and just follow its instruction to finish the uninstallation.

VirtualBox

Install VirtualBox

We use VirtualBox’s repository for Fedora here. We use VirtualBox-4.1 as the example.

Download the repository file.

# cd /etc/yum.repos.d
# wget http://download.virtualbox.org/virtualbox/rpm/fedora/virtualbox.repo

Install VirtualBox using yum

# yum install VirtualBox-4.1

Run VirtualBox

Just execute

$ virtualbox

Start a virtual machine just by one command

# VBoxManage startvm name_of_the_virtual_machine

VBoxManager use the virtual machine’s name as its parameter. The name can be found from VirtualBox’s virtual machine list.

Uninstall VirtualBox

As installing VirtualBox, we can uninstall (erase) it using yum

# yum erase VirtualBox-4.1

A Tutorial on Ns-2

Wed, 16 Nov 2011 23:47:34 PST

This is a tutorial on ns-2 which gives a overview of ns-2 and introduces how to define a simple topology and the network flow. This tutorial is based on the slides for Lab session of COMP 4622 at HKUST in the fall semester of 2011.

The slides of tutorial on ns-2 can be download from here:

Tutorial on ns-2

Installing ns-2 and ns-3 on Fedora Linux

Mon, 14 Nov 2011 23:53:22 PST

ns is a discrete-event network simulator targeted primarily for research and educational use. There are two version of ns, ns-2 and ns-3, which are quite different with each other in the implementation and method to use them. In this post, we introduce how to install ns-2 and ns-3 on Fedora Linux 15.

Install ns-2.34 on Fedora

Download the allinone package

$ wget http://downloads.sourceforge.net/project/nsnam/allinone/ns-allinone-2.34/ns-allinone-2.34.tar.gz

Uncompress the pacakge

$ tar xf ns-allinone-2.34.tar.gz

Install needed pacakges

# yum install gcc make libX11-devel libXt-devel libXmu-devel

Ns-2 requires older version of gcc. So we install gcc-34 and gcc-34-c++ for it

# yum install compat-gcc-34 compat-gcc-34-c++

Install the allinon package of ns-2. During the install process, specify the CXX compiler we use

$ cd ns-allinone-2.34
$ CXX=g++34 ./install

Configure the environmental variables for ns-2 and nam, and add the executables to the PATH so that we can use ns and nam directly.

Add to ~/.bashrc if you use bash

NS_HOME=/full/path/to/ns-allinone-2.34
PATH=$NS_HOME/bin:$NS_HOME/tcl8.4.18/unix:$NS_HOME/tk8.4.18/unix:$PATH
export PATH

Add to ~/.cshrc_user if you use c shell

setenv NS_HOME "/full/path/to/ns-allinone-2.34"
setenv PATH "${PATH}:${NS_HOME}/bin:${NS_HOME}/tcl8.4.18/unix:${NS_HOME}/tk8.4.18/unix"
setenv LD_LIBRARY_PATH "${NS_HOME}/otcl-1.13:${NS_HOME}/ns-2.34/lib:/usr/local/lib"
setenv TCL_LIBRARY "${NS_HOME}/tcl8.4.18/library"

The installation is done by this step. Open another shell and try our installation:

$ nam

and

$ ns

Install ns-3.12.1 on Fedora

Install needed packages

# yum install libxml2 libxml2-devel gcc gcc-c++ make automake \
autoconf binutils openssh-server openssh-clients openssl python \
python-devel mercurial bzr scons flex bison tcpdump valgrind gdb

Download the allinone pacakge for ns-3

$ mkdir tarballs
$ cd tarballs
$ wget http://www.nsnam.org/release/ns-allinone-3.12.1.tar.bz2
$ tar xjf ns-allinone-3.12.1.tar.bz2

Build ns-3

$ cd ns-allinone-3.12.1/
$ ./build.py

We will now interact directly with Waf in the ns-3.12.1 directory

$ cd ns-3.12.1

Configuration with Waf

$ ./waf -d optimized configure --enable-examples; ./waf

Validate our installation by running one example

$ ./waf shell
$ cd build/optimized/examples/
$ udp/udp-echo

Use tcpdump to display the simulation result

$ tcpdump -tt -r udp-echo-0-1.pcap

Hiking: MacLehose Trail Stage 5 – Tate’s Cairn to Tai Po Road

Sun, 13 Nov 2011 08:52:12 PST

We went hiking MacLehose Trail Stage 5 (10.6 km in total) on Nov. 13, and climbed to the Lion Rock along MacLehose Trail. It took us about 6 hours including around 1 hour to climb the Lion Rock and lots time for sight view.

My suggested route from our experience:

Take a taxi to Gilwell Camp Site (基維爾營地) from the Choi Hung (彩虹) MTR Station (or HKUST if you starts there as us).
Go through the Camp Site’s gate (pass the CampSite Office, a very small gate) to the MacLehose Trail.
Start from the Tate’s Cairn (大老山; Tai Lo Shan).
Reaches the west end of MacLehose Trail Section 5: Tai Po Road (大埔公路).
Take bus 81 at Shek Lei Pui Reservoir (石梨貝水塘) station to Price Edward or Mong Kok MTR station.

More about MacLehose Trail Stage 5:

Hong Kong Tourism Board

郊野樂行 Enjoy Hiking (with the map)

Photos taken:

Inline Assembly with GCC on Linux

Tue, 08 Nov 2011 20:06:30 PST

One cool feature of gcc is that it can inline assembly into C code. With inline assembly, the programmer can precisely control the execution of the processor, such as forcing variables to use registers, getting special processor state efficiently, and writing critical efficient code in assembly by hand.

I compile a list of tutorials from the Internet about inline assembly with gcc on Linux. These tutorials use AT&T syntax for assembly on x86 system. For x86-64, the difference with x86 is small. The method is similar, just the type of instructions and registers, such as movq, %rax, %r15.

GCC-Inline-Assembly-HOWTO by Sandeep.S

“This HOWTO explains the use and usage of the inline assembly feature provided by GCC. There are only two prerequisites for reading this article, and that’s obviously a basic knowledge of x86 assembly language and C.”

Brennan’s Guide to Inline Assembly by Brennan “Bas” Underwood

“This is meant to be an introduction to inline assembly under DJGPP. DJGPP is based on GCC, so it uses the AT&T/UNIX syntax and has a somewhat unique method of inline assembly.”

Using Assembly Language in Linux. by Phillip

“This article will describe assembly language programming under Linux. Contained within the bounds of the article is a comparison between Intel and AT&T syntax asm, a guide to using syscalls and a introductory guide to using inline asm in gcc.

This article was written due to the lack of (good) info on this field of programming (inline asm section in particular), in which case i should remind thee that this is not a shellcode writing tutorial because there is no lack of info in this field.”

Ordered Apple iPhone 4S 16GB Black at 3Shop of Three Hong Kong

Tue, 08 Nov 2011 00:38:11 PST

Update: Picked up the Apple iPhone 4S 16GB Black from 3 Hong Kong at 3Shop on the afternoon of Nov. 11, 2011. Everything is good. Also updated iOS to iOS 5.0.1.

Update: My $28 VAS is waived since I use the $56 1966 IDD package.

I ordered an Apple iPhone 4S 16GB Black from 3 Hong Kong at 3Shop. The main benefit of ordering from 3 Hong Kong is that I can get the iPhone 4S before Nov. 12, that is, the same day or one day later of the release of iPhone 4S by Apple in Hong Kong.

I registered at 3 Hong Kong one week before and a sales of 3 Hong Kong called me the day before yesterday and told me that I am in the head part of the queue and can get the iPhone 4S in the first several days after Nov. 11. When I make the order today, the staff of 3Shop confirmed with me that I can get the iPhone 4S on or before Nov. 12.

The iPhone 4S from 3 Hong Kong must go with a plan. I choose the 298/mon plan, which is suitable for me and also the best price for me compared to my current plan with 3 Hong Kong (88/mon, I use around 1400 min per month). I pay around extra ~~$1083.84~~ $411.84 totally in two years compared to my current one (deduct the price for iPhone 4S) with some benefit (I show the details later).

I pay by the Manhattan Credit Card, so that I can enjoy additional offer: 1) $0 prepay ($4300 for a normal one. This is very nice, really.), 2) pay handset fee monthly ($19.16 each month) for 24 months, 3) waive of the HK$12 MTR/tunnel fee (nice, too), 4) VAS of $28 ($36 for a normal one) ($0 if subscribe the $56 1966 IDD package), 5) Unlimited local data usage for first 6 months.

A summary of my plan and a comparison to my old plan are as follows.

Old plan with 3 Hong Kong:

Month contract: $88 (1300 min basic + Environmental Friendly Plan).

For 24 months in total: $88 x 24 = $2112

iPhone 4S plan with 3 Hong Kong:

The package: 1800 min basic + unlimited Wi-FI + 500MB (unlimited for the first 6 months) Local Data + others.

Month contract: $298

VAS packages: ~~$28~~ $0 (if subscribe the $56 1966 IDD package)

MTR/tunnel fee: $0 (Waived)

Handset (iPhone 4S 16G) price: $19.16

Monthly fee in total: $298 ~~+ $28~~ + $19.16 = ~~$345.16~~ $317.16

Fee for 24 months: ~~$345.16~~ $317.16 x 24 = ~~$8283.84~~ $7611.84

The Comparison:

Fee of iPhone 4S plan, deduct the price for iPhose 4S: ~~$8283.84~~ $7611.84 – $5088 (Apple’s price) = ~~$3195.84~~ $2523.84

The difference (deduct my old plan): ~~$3195.84~~ $2523.84 – $2112 = ~~$1083.84~~ $411.84

For each month: ~~$1083.84~~ $411.84 / 24 = ~~$45.16~~ $17.16

So I pay extra ~~$45.16~~ $17.16 each month after subscribing the iPhone 4S plan. But I can get the benefit of 1) using iPhone 4S 16G just after Apple’s release, 2) pay the iPhone 4S’s price monthly for 24 months, 3) unlimited Wi-Fi + 500MB local data, 4) 3Screen one year (worth $3,380 according to 3), 5) ~~$28 VAS package and~~ $0 MTR/tunnel fee.

Although 3′s signal is not very good according to others, I use 3 for more than 1 year and I am happy with it signal and the $56 unlimited IDD plan (I call to mainland China a lot), especially that only 3 has signal in the Lab where I stay in HKUST.

The iPhone 4S plan is a good choice for those who want to use iPhone 4S but can not get into the pre-order phase and do not want to wait in the IFC mall before Apple Store for a very long time.

Installing and Configuring 32-bit Firefox on 64-bit Linux (Fedora)

Sun, 30 Oct 2011 20:56:20 PDT

On a x86-64 64 bit Linux system, sometimes we may want to use the 32 bit Firefox, although the support to 64 bit Firefox is getting much better. Just for needed ones, this post introduces how to install 32 bit Firefox on 64 bit Linux (Fedora as the example).

Install 32-bit firefox

First, install firfox.i686 package.

# linux32
# yum install firefox.i686

Firefox can simply run now. But we still need to fix some issues.

When run firefox, we may get these warnings:

(firefox:4860): Gtk-WARNING **: Unable to locate theme engine in module_path: “clearlooks”,
Gtk-Message: Failed to load module “pk-gtk-module”: libpk-gtk-module.so: cannot open shared object file: No such file or directory
Gtk-Message: Failed to load module “canberra-gtk-module”: libcanberra-gtk-module.so: cannot open shared object file: No such file or directory

To fix these warnings, install needed packages:

# yum install libcanberra-gtk2.i686 \
 gtk2-engines.i686 \
 PackageKit-gtk-module.i686

However, there is no input method. To enable ibus-gtk2 input method in 32bit Firefox, install the ibus-gtk2 package

# yum install ibus-gtk2.i686

Install the flash-plugin

Enable rpmfusion repository: Enable RPM Fusion repositories

Install flash plugin

# yum install flash-plugin.i386

Set up flash plugin

# /usr/lib/flash-plugin/setup

There is no sound in flash. For sound support and curl support, install

# yum install alsa-plugins-pulseaudio.i686 curl.i686

You should have sound. flash-plugin works.

Firefox should complain nothing and work well now. Enjoy it.

Additional gift:

Force to run firefox in 32-bit mode:

#!/bin/bash

setarch i686 firefox -new-tab $*

How to Set Up and Configure NFS Server and Clients

Sun, 30 Oct 2011 08:42:09 PDT

NFS is widely deployed and used even after more than twenty year. NFS is easy to set up and easy to use. This introduces how to set up the NFS server and clients. We use the Fedora system as the example.

Set up a NFS Server

Configure /etc/exports to give clients the permission to use the NFS directories.

Edit /etc/exports

For example, to allow the servers inside subnet 10.0.0.1/24 to mount the /home directory with read/write permission. Add this line to /etc/exports:

/home 10.0.0.1/24(rw)

For details of the exports functions, please refer to export manual.

Start up the NFS service

Enable nfs service on the NFS server so that the NFS service daemon automatically starts each time the server starts:

# /sbin/chkconfig nfs on

You may also manually start it

# service nfs start

Client-slide configuration

Package installation

# yum nfs-utils

Start the rpcbind service

# service rpcbind restart

You may also set it to start automatically

# chkconfig rpcbind on

Mount the NFS directory

# mount NFS_SERVER:/lhome/userdir MOUNT_POINT

where NFS_SERVER is the NFS server’s address, and MOUNT_POINT is the local mount point on the client side for the NFS directory.

You may also consider using autofs on top of NFS as described in Unified Linux Login and Home Directory Using OpenLDAP and NFS/automount.

x86-64 ISA / Assembly Programming References

Tue, 25 Oct 2011 10:24:03 PDT

This post collect the reference resource for x86-64 (also know as Intel 64, AMD 64) ISA / assembly language programming. x86-64 is a 64-bit extension of the Intel x86 instruction set.

x86-64 Assembly Programming

Introduction to Computer Systems Resources (15-213 Introduction to Computer Systems Resources from CMU) Lots materials for learning machine-level programming on the x86-64 machines.

x86-64 Machine-Level Programming This document supplements Chapter 3 of the CS:APP book.

Gentle Introduction to x86-64 Assembly This document summarizes differences between x86-64 and i386 assembly. The readers should already know the i386 gas syntax.

x86-64 (and x86) ISA Reference

Intel® 64 and IA-32 Architectures Software Developer Manuals Official reference from Intel.

AMD64 Architecture Programmer’s Manual Official reference from AMD.

Others

Software Optimization Resources

Software optimization resources. Optimization manual for advanced assembly language programmers and compiler makers, details about the internal working of various microprocessors, instruction latencies, execution unit throughputs, micro-operation breakdown and other details for all application instructions of most microprocessors from Intel, AMD and VIA.

i386 References and Learning Resource

Computer Systems: A Programmer’s Perspective, 2/E (CS:APP2e) This text book introduces assembly programming on x86 machines.

80386 Dependent Features of as

PC Assembly Tutorial

x86 processor information

Binary interface

System V Application Binary Interface

Update history: Nov. 11, 2011. Add optimization resource.

Howto for New Git Users

Thu, 13 Oct 2011 00:54:03 PDT

This post is a tutorial for new users to set up git and clone and use the first repository. This post introduces how to start using git for new users. This post does not introduce details of how to use git commands. Please refer to the git manual or other tutorials for how to commit, push, etc.

In this post, we introduce how to set up keys for git, how to configure non-standard SSH port for git, and how to use the first git repository.

Users’ SSH private/public key pairs

Every user of git, administrator or a normal user, need to have a private/public SSH key pairs in ~/.ssh.

New users need to generate the key pairs (if they don’t have one) by executing this command:

$ ssh-keygen -t rsa

After generating the key pairs, the user can give the public key (~/.ssh/id_rsa.pub) to the git server administrator and ask he/she to create a account and repository.

After the git server administrator adds the new user by its public key to the git server, the user can starts to use git.

Non-standard SSH port

This section only for the git servers that use non-stardard SSH port, i.e. other than 22.

If sshd is listening on non-standard port(s) on the git server, for example 22111, the user should set SSH to use the special port. We use example.org:22111 as the example here.

Put these two lines in your ~/.ssh/config file:

Host example.org
  Port 22111

If this ~/.ssh/config file doesn’t exist, you should create it first and set it’s mod to 744:

$ chmod 744 ~/.ssh/config

Create the new repository

This section only for users to create new repository. If the user is to use an already exist repository, please skip this section. Besides, the steps in this section only need to be done once for one repository.

We use the example that the administrator create repository repo1 for user1 and give user1 write privilege to this repository. We assume the address for this repository is git@example.org:repo1.git . Now we introduce how does user1 create the new repository repo1.

As user1 has the write privilege on the repo1 repository, it can create this repository on it’s local machine first and then push it to the git server. After pushing it to the git server.

These operations are done by user1 on its local machine:

$ mkdir repo1
$ cd repo1
$ git init
$ touch README
$ git add README
$ git commit -a -m 'first commit'
$ git remote add origin git@example.org:repo1.git
$ git push origin master

If it successes, the new repository is created on the git server.

Use the repository

Please note that step 1 only need to be done for the first time. After getting the repository to a local directory on the user’s local machine, the user only need to follow step 2 to step 5.

1. Clone the repository for the first-time. You only need to do it once for the first time, for the later usage of this repository, you continue use this local directory.

$ git clone git@example.org:repo1.git

A directory named repo1 will be created in the current directory.

If you want to use the repository directory in the previous section, just add these line to the end of .git/config:

[branch "master"]
remote = origin
merge = refs/heads/master

2. Pull the updates made before by you or the others. Use “git pull” command in the repository’s directory.

$ cd repo1
$ git pull

Then you can work on this updated copy of the repository by editing the files.

If you add a file or directory (for example, add directory a/b and file a/t.txt), you can add it to the repository by:

$ git add a/b a/t.txt

3. Commit the changes made by you with a message

$ git commit -a -m 'update the files'

4. Push it to the git server

$ git push

By now, your changes have been pushed to the git server. When the others use “git pull” command, they will get the new version of the source codes that have changed by you.

You can use “git log” to see the logs of all the commits of this repository.

You may be also interested in more git solutions.

Installing the Flash Plugin for 64-bit Firefox in Linux x86-64

Wed, 28 Sep 2011 10:20:49 PDT

This post introduces how to install flash plugin to 64-bit firefox on a x86-64 Linux (Fedora as the example). Both 64-bit and 32-bit plugin are available.

64-bit flash plugin for Firefox on Linux

First, download Flash Player Release for 64-bit Linux from Adobe Labs.

Then, unpack the package:

$ tar xf flashplayer.tar.gz

Check whether all needed library is ready for it. If not, install the needed packages.

$ ldd libflashplayer.so

Give it executable attribute:

$ chmod +x libflashplayer.so

Copy the plugin to the plugin directory so that Firefox can use it:

$ cp libflashplayer.so ~/.mozilla/plugins/

32-bit flash plugin for Firefox on Linux

32-bit flash plugin can also work well for 64-bit firefox on 64-bit Linux. We can use the rpm packages from adobe’s repository.

First, install adobe release repository:

# rpm -Uvh http://linuxdownload.adobe.com/adobe-release/adobe-release-i386-1.0-1.noarch.rpm

Install nspluginwrapper so that 32-bit plugins can run on 64-bit system, and 32-bit alsa plugin for pulseaudio so that the 32-bit flash plugin can play sound on the 64-bit system.

# yum install nspluginwrapper.{x86_64,i686} \
 alsa-plugins-pulseaudio.i686 --disablerepo=adobe-linux-i386

Install the 32-bit flash plugin:

# su -c 'yum install flash-plugin'

New Blog Address

Fri, 19 Aug 2011 20:41:27 PDT

The blog domain is changed to fclose.com on Oct 20, 2010.

The latest address is pkill.info

The blog is moved to zhiqiangma.info from Jan. 22 2010.

A Free Personal WordPress Blog Solution

Fri, 19 Aug 2011 20:39:49 PDT

If you want to have a wordpress blog for FREE co.cc + x10hosting is the best choice. I will introduce both in this post.

I will briefly introduce how to set up a wordpress blog like my blog. The overall that I paid for this blog is only $0.89. I can say that it is nearly free. And the 5G traffic is enough for my blog. There are many free blog service providers such as wordpress.com and blogger. But I prefer to set up it by myself because I can control the domain name, the program and the content of my site and I can bakup the site very easily.

1. Domain registration:

Ff you do not want to pay for the domain you can also get a CO.CC domain for FREE from www.co.cc. You will have full dns or forwarding control of the co.cc domain you have. Dns server, CNAME, MX record, forwarding are all available. They provides an easy configuration choice for google application.

I register my domain at the low price of $0.89 from GoDaddy.com .info domain is very cheap for registration. And the annual renew fee is $7.9. If the .com domain is more convenient, you can register one at the price about $10/year.

2. Hosting service:

I am now using hostso’s HK$48/y plan. I decide to pay for a good hosting service to host my blog. With a hosting service that I pay I can get a better service.

If a free hosting service is needed, I still give my recommendations:

When considering the free hosting service, I have several rules:
* It should have some way for earning money. I don’t like to see that my site disappear because the site runs out of their money.
* It should provides PHP + MySQL solution. I mainly using WordPress.
* It should allow ad-free. I don’t like to have a big banner on top or at the bottom of my site.
* It should have a good availability rate and performance.
* It will be better if my friends in China can also browse the site on there servers.

x10Hosting.com is the best free hosting service I have ever found. My blog is running on their servers now. Their service can fulfil all my requirements listed above ;) . They provides 500MB disk space 10G bandwidth and 3 MySQL database, which is far enough for my personal blog. They provides .htaccess support and FTP account. And what’s best is that my friends in China can also browse my blog directly now. Their control panel and web design are much professional, I like it very much. Compared to the other free hosting service, I highly recommend x10hosting.

Before x10hosting, I used the free hosting plan of freehostia.com. I don’t recommend the service of 000webhost.com now even they offer a “1500 MB Disk Space, 100 GB Data Transfer” plan at the price of $0.00. I like the 000webhost’s service plan and the user interface better. But it seems they don’t accept new application now. I thought I wouldn’t spend time on transferring my site because their free 5G traffic plan is enough for me. If more disk space or traffic is needed for your site, you can consider the paid service plan. But the x10hosting’s service is good enough for persuading me to transfer ;)

So, with overall fee of $0.89, I get a personal blog. And I thank the guys that make wordpress and it’s useful plugins available to us for free! WordPress is my favourite blog. I can choose any theme that I like and the most beautiful thing is that there are all kinds of plugins ready for using!

The theme and plugins I am using are:
Theme: iBlog2 (A MacOS X style theme)
Plugins: Akismet (Anti spam), All in One SEO Pack (SEO plugin), KB Robots.txt (Edit robots.txt file), Sociable (Add share and enjoy), WP Super Cache (Save CPU usage of the server), Yet Another Related Posts Plugin (Recommend related posts).

This Blog Passes W3C XHTML 1.0 Transitional Validation

Fri, 19 Aug 2011 20:38:38 PDT

I have made this blog pass the W3C XHTML 1.0 transitional validation! I changed the theme code and made sure the theme have pass it. I also test the homepage and most of the links that can find on the homepage. Some of the post may not pass the validation because of the post content. If you find some post that can not pass it, please kindly notify me by comments or email :) .

Cheers!

CSS level 3 validation is also passed after changing the CSS file of the theme and the plugins.

Updated on 5 Mar. 2010. CSS3 validation passed.

Links

Fri, 19 Aug 2011 20:35:21 PDT

Frequently used links

婚纱照

Tue, 16 Aug 2011 21:21:39 PDT

摄于2010年.

一个段子

Tue, 16 Aug 2011 06:22:56 PDT

上大学时, 刚刚交了女朋友. 有天在寝室, 记得是中午, 跟室友玩扑克三张牌. 开始时我一直输, 都笑说 “情场得意, 赌场失意”. 我们想验证一下, 就一直玩下去. 最后结果是我连续输了30手 (如果没有记错).

后来证明连输30手牌是值得的: 当时的女朋友现在已经是我老婆.

Photos Taken in Kota Kinabalu and Kapalai, Sabah, Malaysia

Fri, 08 Jul 2011 19:37:00 PDT

Photos taken in Kota Kinabalu and Kapalai, Sabah, Malaysia, during our honeymoon travel in the summer of 2011.

The photo gallery:

Google PageRank Update 2011

Mon, 27 Jun 2011 21:11:55 PDT

Update 3: Fclose.com sees minor PageRank updates on November 8, 2011. For example, some tooles (Web Proxy, Lyrics, Gold Price, Fclose.com Search, etc.) and some posts (Howto for New Git Users, Installing the Flash Plugin for 64-bit Firefox in Linux x86-64, etc.) get PageRank of 3 from 0. Update: Confirmed that Google updates the PageRank in toolbar on November 8 2011.

Update 2: Google updates the PageRank in toolbar on August 05 2011.

Update 1: Google updates the PageRank in toolbar on July 18 2011.

Google updates the PageRank displayed in toolbar on Jun. 27, 2011. Google PageRank is just one of factors that determine the position of the page in the search result page. But seeing the growth of Google PageRank is fun for webmasters. The fclose.com’s PageRank is 0 from its birth after the site moves from pkill.info and the changing is done after the first Google PageRank update in January 2011. However, I believe the PageRanks of fclose.com should be quite good after it updates since the PageRank of pkill.info and original one at blog.ust.hk is around 3 and 4.

Today, when I open my site, I am still surprised, the Linux homepage and my homepage‘s PageRank changes to 4 and the other for blogs (Cloud Computing, Web, Programming, Software) has PageRank of 3 in homepage. In addition, and most important, a large number of posts (especially the ones that are linked to from the homepage) get the PageRank of 3 or 2.

That’s a fun thing (although it is not so important) which is a prize of writing blogs. And I also thank for my readers support to this site.

Summary of the Google PageRank updates in 2011:

PageRank updated on November 08 2011
PageRank updated on August 05 2011
PageRank updated on July 18 2011
PageRank updated on June 27 2011
PageRank updated on January 20 2011

GNU glibc Manual

Wed, 08 Jun 2011 20:13:43 PDT

“The C language provides no built-in facilities for performing such common operations as input/output, memory management, string manipulation, and the like. Instead, these facilities are defined in a standard library, which you compile and link with your programs. The GNU C library, described in this document, defines all of the library functions that are specified by the ISO C standard, as well as additional features specific to POSIX and other derivatives of the Unix operating system, and extensions specific to the GNU system.”

“The GNU C library includes several header files, each of which provides definitions and declarations for a group of related facilities; this information is used by the C compiler when processing your program. For example, the header filestdio.hdeclares facilities for performing input and output, and the header filestring.hdeclares string processing utilities. The organization of this manual generally follows the same division as the header files.”

The manual for glibc on gnu.org:

HTML version with entirely on one web page.

HTML version with one web page per node.

PDF version

Conferences on Cloud Computing 2012

Wed, 11 May 2011 00:50:38 PDT

This post lists important conferences on Cloud Computing in year 2012. This post is kept updating.

OSDI 2012

10th USENIX Symposium on Operating Systems Design and Implementation (OSDI ’12)

October 8–10, 2012, Hollywood, CA

“The tenth OSDI seeks to present innovative, exciting research in computer systems. OSDI brings together professionals from academic and industrial backgrounds in what has become a premier forum for discussing the design, implementation, and implications of systems software.”

Important Dates

Complete paper submissions: Thursday, May 3, 2012, 9:00 p.m. PDT

Notification of acceptance: Friday, July 20, 2012

Final papers: Wednesday, September 5, 2012

EuroSys 2012

April 11–13, 2012. Bern, Switzerland.

“It has a strong focus on systems research and development: operating systems, data base systems, real-time systems and middleware for networked, distributed, parallel, or embedded computing systems.”

Important Dates

Paper pre-submission: September 20, 2011

Abstract submission: October 14, 2011

Full paper submission: October 20, 2011

USENIX ATC ’12

2012 USENIX Annual Technical Conference (USENIX ATC ’12)

June 13–15, 2012, Boston, MA

“USENIX ATC has always been the place to present groundbreaking research and cutting-edge practices in a wide variety of technologies and environments. USENIX ATC ’12 will be no exception. ”

Important Dates

Paper titles and abstracts: January 10, 2012, 11:59 p.m. EST

Complete paper submissions: January 17, 2012, 11:59 p.m. EST

Notification to authors: March 26, 2012

Final papers due: May 2, 2012

ASPLOS 2012

Architectural Support for Programming Languages and Operating Systems.

“ASPLOS is a multi-disciplinary conference for research that spans the boundaries of hardware, computer architecture, compilers, languages, operating systems, networking, and applications. ASPLOS provides a high quality forum for scientists and engineers to present their latest research findings in these rapidly changing fields. It has captured some of the major computer systems innovations of the past two decades (e.g., RISC and VLIW processors, small and large-scale multiprocessors, clusters and networks-of-workstations, optimizing compilers, RAID, and network-storage system designs).”

Important Dates

London, UK, March 3–7 2012

Abstract deadline: Monday, July 18, 2011 (noon, EDT)

Full paper deadline: Monday, July 25, 2011 (noon, EDT)

VEE 2012

Eighth Annual International Conference on Virtual Execution Environments (VEE 2012)

“Virtualization has a central role on modern systems, nowadays being a key aspect of how systems are developed, executed, and managed in environments ranging from mobile computing devices to large-scale data centers. Virtualization techniques are applied at many interfaces, from hardware, to OS system calls, to high-level language run times, to cloud management stacks. While these approaches differ dramatically in implementation, they target similar benefits and often must tackle related challenges.

The 2012 ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments brings together researchers across the many applications of virtualization in today’s systems. We invite original papers on topics relating to virtualization – especially those that will have broad appeal across these approaches.”

Important Dates

London, UK, March 3–4 2012

Full paper deadline: Wednesday, November 2, 2011 (midnight, PST)

Author Rebuttal Period: December 8-9, 2011

Notification of acceptance: December 19, 2011

Final copy deadline: January 11, 2012

NSDI 2012

9th USENIX Symposium on Networked Systems Design and Implementation

“NSDI ’12 focuses on the design principles, implementation, and practical evaluation of large-scale networked and distributed systems.”

Important Dates

April 25–27, 2012, San Jose, CA

Paper titles and abstracts: September 27, 2011, 5:00 p.m. PDT

Complete paper submissions: October 4, 2011, 5:00 p.m. PDT

Notification of acceptance: December 20, 2011

HotPar ’12

4th USENIX Workshop on Hot Topics in Parallelism (HotPar ’12)

June 7–8, 2012, Berkeley, CA

“HotPar ’12 will bring together researchers and practitioners doing innovative work in the area of parallel computing. HotPar recognizes the broad impact of multicore computing and seeks relevant contributions in all fields, including application design, languages and compilers, systems, and architecture.”

Important Dates

Paper registration (abstract submission): Tuesday, January 24, 2012, 11:59 p.m. PST

Paper submissions: Tuesday, January 31, 2012, 11:59 p.m. PST

Notification to authors: Late March 2012

Electronic files of final papers due: Monday, April 30, 2012

Pitfalls and Lessons on Configuing and Tuning Hadoop

Tue, 26 Apr 2011 01:41:12 PDT

This post lists pitfalls and lessons learning when configuring and tuning Hadoop.

Hadoop with IPv6

Hadoo doesn’t support IPv6 currently (up to 0.20.2 and 0.21.0): Hadoop and IPv6. The performance of the cluster may suffer from turning IPv6 on in clusters: mail archive.

One good practice is to disable IPv6 on servers in the Hadoop cluster.

Hostname vs. IP

Hostname is preferred. Using hostname instead of IP address may possible solve some problem magically.

Hostgator Coupon

Wed, 06 Apr 2011 08:43:24 PDT

This post contains collection of coupons for Hostgator hosting service.

Enter Code FCLOSECOM25OFF and save 25% on all plans.

This coupon applies only to your first billing cycle. For more details about the Hosting packages, please visit HostGator’s website.

25% off Hostgator coupon:

FCLOSECOM25OFF

The price is $3.71/month which is much lower than Hostgator’s default price 3.96/month.

Setting Up Standalone (Local) Hadoop

Tue, 05 Apr 2011 09:56:02 PDT

Hadoop is designed to run on hundreds to thousands of computers inside cluster. However, Hadoop is configured to run things in a non-distributed mode as a single Java process by default. This is specially useful for debugging since distributed debugging is really a nightmare. This post introduces how to set up a standalone Hadoop environment.

1. Hadoop package and software installation

Follow the instruction of “1. Install needed packages” part in Hadoop Installation Tutorial to install packages. Fllow “4. Hadoop Concigurations” to configure hadoop-env.sh (this file only).

2. Just run Hadoop!

Just run hadoop jobs whose input and output is in local directories. We use a simple example to show how to start a Hadoop job.

The example finds and displays every match of the given regular expression. Output is written to the given output directory.

$ mkdir input
$ cp conf/*.xml input
$ bin/hadoop jar hadoop-mapred-examples-0.21.0.jar grep input output '[a-z.]+'
$ cat output/*

The jar file’s name may be different depending on the Hadoop distribution’s version.

Is it simple? Enjoy it and go further to play Fully-distributed Hadoop Installation.

Conferences on Cloud Computing 2011

Mon, 28 Mar 2011 11:15:39 PDT

This post lists important conferences on Cloud Computing in year 2011.

ACM Symposium on Cloud Computing

October 27 and 28, 2011, Cascais, Portugal

Submission Deadline: April 30, 2011

23rd ACM Symposium on Operating Systems Principles (SOSP)

October 23-26, 2011, Cascais, Portugal

Submission deadline: March 18, 2011, 11:59 PM GMT

EuroSys 2011

April 10-13, 2011. Salzburg, Austria.

CLOUD COMPUTING 2011

The Second International Conference on Cloud Computing, GRIDs, and Virtualization

September 25-30, 2011 – Rome, Italy

Submission (full paper): April 20, 2011

Printing Notes on a PDF File

Sat, 19 Mar 2011 00:39:00 PDT

I tried several software on Windows and Linux to print notes, especially sticky notes on a PDF file. Finally, I find the only working software is Adobe Acrobat. Unfortunately, Adobe Reader doesn’t provide a way to print sticky notes.

The detailed method is posted on Adobe’s blog: Printing Sticky Notes on a PDF

GDB Cheat Sheet

Sun, 13 Mar 2011 06:36:15 PDT

Here is a collection of good GDB Cheat Sheets from the Internet.

GDB QUICK REFERENCE GDB Version 5

GDB Cheat Sheet

gdb Cheatsheet

GDB commands by function – simple guide

复旦大学2005级CS专业毕业典礼照片 (FD05CS)

Thu, 10 Mar 2011 02:14:34 PST

以此留念我的大学生活.

一些毕业时照的照片:

如果你不想你的照片出现在这里,请跟我联系.

HP Compaq BIOS Crisis Recovery from a Failed BIOS Flashing

Thu, 03 Feb 2011 05:07:29 PST

I tried to flash my HP Compaq CQ35-240TX’s BIOS to F.15 but unfortunately failed. The computer turned to white screen. When I restarted (powered off then boot again), this is no response any more: the hard disk and caps lock’s LED keep blinding. I thought “Oh my God! I must send it to HP to repair it!”.

But after searching on the Internet and find there is a “crisis recovery” mechanism to recovery the BIOS even the CMOS don’t work. I finally managed to recovery the BIOS with a help of USB flash disk. I will introduce the procedure to crisis recovery my HP Compaq laptop. You need to prepare a USB flash disk and another computer that is connected to the Internet (we need to download the BIOS image from HP).

The steps to recovery the BIOS are as follows.

1. Format the USB flash disk’s filesystem to FAT16 on another computer.

2. Get the BIOS image from HP’s website (I use F15 version).

Download the BIOS update file from HP’s website (an exe file). Execute the BIOS update file on another computer. It will decompress the files. Select a directory to store the files and you can close the program. In the directory selected, we can get WinFlash.exe. Unzip WinFlash.exe using a program like WinRAR. We can find two .FD files. For my computer and the BIOS version I use, the two files are 306DF15.FD and 306EF15.FD. There are two model for Compaq CQ35 computers, so there are two .FD files. You may try one by one to see which model your computer use. Actually this model can be found in “BIOS Setup” (we can’t start it as the CMOS is bad)

3. Copy the file 306DF15.FD to USB disk’s root directory and rename it to 306D.bin. (the file may be different depending on the model)

4. Remove the AC power and battery from the lapttop.

5. Plug in the USB flash disk to the laptop.

6. Hold two keys: WinKey + B.

7. Plug in the AC power.

8. Hold the Power Button for 5 seconds, then release.

9. Push the Power button as normal to power on the computer.

10. After the laptop starts short beeping, release WinKey+B.

11. The laptop will keep short beeping for about one minute, it will power off itself after recovering the BIOS.

If the laptop keeps long beeping or beeping for more than 3 minutes, the recovery fails, possibly because of the wrong image file or the USB flash disk doesn’t work on the laptop. In this case, you may try another image file or another USB flash disk.

12. Normally power on the laptop after recovering the BIOS and you will see the Compaq logo appearing again.

Latex Cheat Sheet

Thu, 23 Dec 2010 21:44:50 PST

Latex cheat sheet by Winston Chang (www.stdout.org/~winston/):

Download latex cheat sheet

fclose – Close a Stream

Mon, 15 Nov 2010 13:33:54 PST

fclose is a frequently used C standard library which closes the file associated with the stream and disassociates it.

NAME

fclose – close a stream

SYNOPSIS

#include <stdio.h>
int fclose(FILE *fp);

DESCRIPTION

The fclose() function will flushes the stream pointed to by fp (writing any buffered output data using fflush()) and closes the underlying file descriptor.

RETURN VALUE

Upon successful completion 0 is returned. Otherwise, EOF is returned and errno is set to indicate the error. In either case any further access (including another call to fclose()) to the stream results in undefined behaviour.

ERRORS

EBADF The file descriptor underlying fp is not valid.

The fclose() function may also fail and set errno for any of the errors specified for the routines close(),
write() or fflush().

CONFORMING TO

C89, C99.

NOTES

Note that fclose() only flushes the user space buffers provided by the C library. To ensure that the data is physically stored on disk the kernel buffers must be flushed too, for example, with sync() or fsync().

From Linux Programmer’s Manual

EXAMPLE

#include <stdio.h>
int main(int argc, char* args[])
{
  FILE * fp;
  pFile = fopen ("file_to_write.txt","wt");
  fprintf (fp, "fclose example");
  fclose (fp);
  return 0;
}

Speeding Up the Site With Apache GZIP Compression

Tue, 14 Sep 2010 09:01:56 PDT

We can speed up the site with compression while save bandwidth at the same time. As most of the modern browsers support gzip encoding, we can set it up to let the users enjoy faster speed. The Apache mod_deflate is easy to set up and standard. It compress the content on the fly.

We can enable output compression in Apache by adding the following into the .htaccess file in the root directory of the website:

# Compress html, text, javascript, css and xml content:
AddOutputFilterByType DEFLATE text/plain
AddOutputFilterByType DEFLATE text/html
AddOutputFilterByType DEFLATE text/xml
AddOutputFilterByType DEFLATE text/css
AddOutputFilterByType DEFLATE application/xml
AddOutputFilterByType DEFLATE application/xhtml+xml
AddOutputFilterByType DEFLATE application/rss+xml
AddOutputFilterByType DEFLATE application/javascript
AddOutputFilterByType DEFLATE text/javascript
AddOutputFilterByType DEFLATE application/x-javascript
AddOutputFilterByType DEFLATE text/x-javascript

If the .htaccess file can’t be edited or the mode isn’t enabled, there is another alternative solution for php supported environment:

Speeding Up the Site by Using PHP GZIP Compression

Information for Living and Travelling in Hong Kong

Sat, 28 Aug 2010 23:22:33 PDT

This is a collection of useful online information for living and travelling in Hong Kong. This collection is kept updating.

Map

Hong Kong map: CentaMap

Public transportation

The train roadmap: MTR journey planner. A PDF version.

Road search and road info: Public Transport Enquiry Service.

Snapshots and live webcast of transportation condition from GovHK.

Public transportation in Hong Kong from GovHK.

Weather

Current weather and 7-day weather forecast: Hong Kong Observatory.

Beach

Beach water quality from EPD.

Dinning

Hong Kong restaurants guide: OpenRice.

Dim Sum menu with Cantonese pronunciation and names in Chinese and English: Dim Sum guide.

Movie and cinema

Hong Kong film guide: wmoov

Travel guide

Official travel guide from the Hong Kong Tourism Board: Discover Hong Kong.

Hong Kong National Geopark.

The Peak.

Victoria Harbour.

Hiking in Hong Kong

Enjoy hiking

Disable Caller ID

Hide the caller’s phone number by prefixing the number 133. For example, I want to call 88888888 but I don’t want them know my phone number, I can call 13388888888.

VISA

For non-permanent residents (such as Student VISA): Online Application for Extension of Stay (for Other Non-Permanent Residents)

HKID

Change the address in HKID card: Notification of Change of Particulars Previously Registered

Changing the Font Size of WordPress’s Visual Editor

Sat, 07 Aug 2010 00:17:04 PDT

The font size of WordPress’s visual editor is too small for me on my screen display. The default font size is 13px which is hard for my eye on my screen. There are two ways to change the default font size in the visual editor. One is changing WordPress’s css file for the editor while the other is using a plugin.

Method 1: Change WordPress’s CSS file

This is the most direct way. This method will apply to all the contributors’ editors.

The CSS file for the visual editor’s font is: wp-includes/js/tinymce/themes/advanced/skins/wp_theme/content.css in WordPress’s installation directory.

First download this file and open if with an text editor that you like. Then find these lines:

body.mceContentBody {
    font: 13px/19px Georgia, "Times New Roman", "Bitstream Charter", Times, serif;
    padding: 0.6em;
    margin: 0;
}

The font here specifies the font size of WordPress’s visual editor. If we want to change the font size to 16px, just change that line to:

font: 16px/19px Georgia, "Times New Roman", "Bitstream Charter", Times, serif;

Then save this CSS file and uploaded to the hosting server. It’s done!

Method 2: Use WordPress plugin

This method will not modify WordPress’s files. But it requires us to install a plugin: Visual Editor Font Size

After installing this plugin, you will find a “Visual Editor Font Size” tool bar under the editor (maybe a little far away from it). You can click the “+” and the “-” button to make the font larger or smaller.

RGBlite WordPress Theme

Sat, 29 May 2010 07:39:49 PDT

About RGBlite Theme

RGBlite Theme is a simple and light-weight theme for WordPress. RGBlite Theme provides a green-red-blue mixed style, bright view and fast rendering experience to the users.

RGBlite Theme passes W3C XHTML 1.0 and W3C CSS level 3 validation. This theme works well on WordPress 2.9.x and WordPress 3.0.

RGBlite theme is among the top 10 most fastest WordPress themes in one measurement: 430 WordPress themes: Page load time measures. Here is the detailed measurement result and the overall rank image from the result:

How to install

Download the theme from the link below. Then upload it to the /wp-content/themes/ directory and select it in WordPress console will make this theme work.

Build in support for plugins

It is recommended to install these plugins while using this theme and the page template:
WordPress Popular Posts
Sitemap Generator for WordPress

Please install these plugins when you using this theme. These plugins will make this theme function well as the blog it is originally designed to.

Preview, download and Licence

Free to download, free to use, free to customize.
Released under the GPL License (What is the GPL?)

Preview is available: RGBlite Theme Demo or on WordPress.org.

Download: RGBlite WP Theme.(Version 1.3, Feb. 3, 2012)

How to get the homepage like the demo screenshot

If you like the homepage style in the demo screenshot which is different from the default recent posts list, you can easily follow these steps:

1. Create a page for the homepage. The content of this page will disappear on the top side. Select “Homepage” for the template on the right side of the page editing page.

2. In “Settings->Reading” in WordPress’s control panel, select “A static page” in the “Front page displays” section. Select the page you have created in step 1 for the “front page”.

3. The “Featured posts” part in the homepage requires “WordPress Popular Posts” plugin. After install this plugin, the “Featured posts” will appear.

The top part of the right sidebar is actually a “Text widget” which can be designed by yourself.

Translation

Thanks to the people that translate this theme to other languages:

Romanian, Alexander Ovsov (Web Geek Sciense).

Change Logs

Feb. 3, 2012. Update to version 1.3.
Feb. 1, 2011. Add page load speed measurement result.
Version 1.1.3. Fix lots bugs; delete archive.php, missing.php; use add_theme_support(‘automatic-feed-links’); smaller footer font;
Jul. 22, 2010 Add “How to get the homepage like the demo screenshot” part.
Version 1.1.2. Prettied default sidebar; Search form always stays on top of the sidebar; Search functions run now.
Version 1.1.1 Add nav menu support; Add body_class(); add wp_link_page(); Add default sidebar; Add search form in default sidebar;
Version 1.1.0 Change post meta in post to two lines and add the right part of first line; Add text before content; Move post meta from file to function; No border between content and sidebar; Homepage has page number; no border-bottom for header; post font 12px; h2 28px; h3 22px; h4 16px; Change title format. Head logo and the menu is on two lines; Change tag cloud line-height to 2em;
May 31, 2010. Change index.php.
May 29, 2010. The first version 1.0.0 released.

https://docs.google.com/leaf?id=0BxXe2zOqYbxmNmIxZDhmZmEtMWY3Mi00MWVhLThlMjgtMzU5NmZlZmVmZDQ4&hl=en

Setting Up POP3/IMAP Email Box on Nokia E71

Wed, 12 May 2010 10:27:39 PDT

The software in Nokia E71 already support POP3 and IMAP email box. But we may lead to install Nokia messaging and setting up account of Nokia messaging. Storing our emails in remote servers isn’t a secure way. And why not use IMAP directly if it is supported by the system software? How to set up POP3 or IMAP email box on Nokia E71 will be introduced in this post.

First create a new mailbox by select:

Menu -> Communic. -> Messaging -> Optioins -> Settings -> E-mail -> Options -> New mailbox -> Start

Allow connection? select NO. This is the most important part. Otherwise, we will be lead to install Nokia Messaging again.

E-mail address: your email address

Password: password

User name: user name

Then you will get a message that the account can’t be defined. It will ask us for the email client:

Select POP3/IMAP in the next screen:

You will get a wizard after that:

Then just follow the wizard and you will set up a email account using POP3 or IMAP.

Calling Three Hong Kong’s Customer Service

Mon, 12 Apr 2010 05:09:18 PDT

3′s hot line number for 3G service is 31623333. 8 in the main menu is the default key to connect to the customer service manager, but I always get “Sorry, there is error in you input, please input again.” After some tries, I find out the quick way to the customer service:

Main menu ->

Press 6 (Service Application & Value-Added Service Inquiry) ->

Press 1 (Services application.) ->

Press 5 (The other services application.)

You will get the message “Please wait for a while, our customer service manager will answer your call…”. Wait for a while and then you will get serviced :)

Update history:
Jun. 29, 2011. Update to current selection #.
Oct. 29, 2011. Update to current selection # 6 first.

Vim Formatting C/C++ Code

Sat, 13 Mar 2010 05:55:14 PST

Vim provides some basic formatting commands. A combination of these commands with the editing commands will make the programmer happy.

A list of basic text formatting commands in Vim:

= is an operator that formats/indents text.
i{ is a text object that specifies the surrounding code block. It should be used together with v, =, etc.

Some basic but useful operation that are combination of the two basic ones above:

vi{ visually selects the inner code block around the cursor.
=i{ formats the code block.
=2i{ formats the current block and the block around it.
== formats current line.

The method to format a block of C/C++ or Java code:

First, go to the start point of codes to be formatted, then press v to start selection.
Second, go to the end point.
Third, press = to format the codes that have been selected. All braces and indentations will be aligned.

A short way to format the whole buffer:

gg=G

The C Programming Style that I Follow

Thu, 04 Mar 2010 23:54:53 PST

C programming language allow any style the programmer like. A good style can make the code easy to understand and maintain, while a bad style will make the project a nightmare.

This is the C programming styles that I follows:

1. For each source file (*.c), there is a header file (*.h) that have the same name with it as it’s interface.

2. The header file starts with

#ifndef _HEADER_NAME_H
#  define  _HEADER_NAME_H

And end with:

#endif // _HEADER_NAME_H

3. #include is only used in source file usually. ONLY when it is really necessary, #include can be used in header file.

4. There is NO definition in header file.

5. Global variable defined in source file that is visible outside the module is declared as extern variable in the relevant header file. Global variable is used only when it is really necessary.

An example that follow this style is mrcc.

Keep Firefox Open After Closing the Last Tab

Mon, 22 Feb 2010 07:49:29 PST

Firefox should always be ready there for us, right? Even after we closing the last tab, we would like firefox to be still open. At least I like to treat it in this way.

But unfortunately, Firefox will close after we closing the last tab as default in version 3.5. But the good news is that this behaviour is configurable through Firefox’s configuration editor. Now, let’s open it:

Type in this into the address bar and hit enter:

about:config

Firefox will warn us changing these configuration may be harmful. Okay, tell it we’ll be careful by clicking the button.

The type in this in to the filter field:

LastTab

You will find this item with name:

browser.tabs.closeWindowWithLastTab

That’s exactly what we are looking for! Double click it to change it’s value to “false”.
Okay, that’s it. Close this tab now and try closing all the tabs by using lots of “CTRL + w”. See what? Firefox is still there open even after we closing the last tab.

mrcc – A Distributed C Compiler System on MapReduce

Sat, 16 Jan 2010 07:18:29 PST

The mrcc project’s homepage is here:

mrcc project

Abstract

mrcc^[3] is an open source compilation system that uses MapReduce^[1] to distribute C code compilation across the servers of the cloud computing platform. mrcc is built to use Hadoop^[2]by default, but it is easy to port it to other could computing platforms, such as MRlite^[8], by only changing the interface to the platform. The mrcc project homepage is http://fclose.com/p/mrcc/
and the source code is public available.

Introduction

Compiling a big project such as Linux may cost much time. Usually, a big project consists of many small source code files and many of these source files can be compiled simultaneously. Parallel compilation on distributed servers is a way to accelerate the compilation.

In this project, we build a compilation system on top of MapReduce for distributed compilation.

Design and architecture

The system of mrcc consists of one master server that controls the job and many slave servers that do the compilation work as shown in Figure 1. The master and the slaves can be one of the jobtracker or the tasktracker of the MapReduce system.

Figure 1: The architecture of mrcc

mrcc runs as a front end of cc. When compiling one project, the user only need to add “-jN” parameter and “CC=mrcc” at the end of the normal “make” arguments and make the project on the master server. make will build a dependency tree of the project source files and then fork multiple mrcc instances when several source files can be compiled in parallel.

mrcc will first do preprocessing such as expanding the options and inserting the header files into the source file. Then mrcc will determines whether the job can be done on MapReduce depending on the compiler’s options. For ensuring the correctness of the compilation, mrcc only distributes the jobs to slaves when they are safe. The other jobs are done on the master server locally.

When doing remote compilation, mrcc first put the preprocessed source file into the network file system which is HDFS when using Hadoop, then it runs the job on MapReduce. Only the mapper which is called mrcc-map is used in this system. mrcc-map first gets the source file from the network file system, and then it calls the compiler locally on the slave. After the compilation finished, mrcc-map puts the object file which is the result of the compilation back into the network file system. After mrcc-map finished, mrcc on the master server get the object file from network file system and puts it into the local file system on the master node. By now, the compilation of one source file is finished. make can go on to compile the other files that depends on the completed one.

Implementation

The mrcc compilation system consists of two core parts: the main program mrcc and the mapper for MapReduce called mrcc-map. This project is open source and all the source code can be downloaded from [3].

The overall system consists of seven parts:

The main program mrcc (mrcc.c, compile.c, remote.c, safeguard.c, exec.c);
The mapper mrcc-map (mrcc-map.c);
The compiler options processing part (args.c);
The logging part (trace.c, traceenv.c);
The network file system and MapReduce interface part (netfsutils.c, mrutils.c);
The temporary files cleaning up part (tempfile.c, cleanup.c);
Some other utilities for processing string, file path and name, and I/O operation (utils.c, files.c, io.c).

In this report, we only introduce the two core parts of this system: mrcc and mrcc-map.

mrcc

The work flow is shown as Figure 2. There are five time lines in the process: mrcc is called, begins to compile locally or remotely while source file is being preprocessed, the preprocessing finishes, mrcc returns and mrcc finishes. During this process one or two new process is created by mrcc for source preprocessing or compiling. And some other works such as logging are done which we will not introduce. In the work flow graph, only the core functions are shown.

Figure 2: Work flow of mrcc

When mrcc is called, it first preprocesses the compiler options by functions such as expand_preprocessor_options() which expands the arguments passed to preprocessor. Then make_tmpnam() is called to creates a temporary file inside the temporary directory and registers it for later cleanup. In make_tmpnam() a random number is generated for the temporary file. We use this random number as the id of the job and the object file name. The temporary output directory of the MapReduce job will be generated from this id to avoid name conflict ion. For example, if the temporary file name for storing a preprocessed source file is “/tmp/mrcc_36994a30.i”, the object file name and the output directory name are “/tmp/mrcc_36994a30.o” and “/tmp/mrcc_36994a30.odir” on the network file system.

mrcc then scans the options to determine whether this job can be done on remote server by calling scan_args(). While scanning the arguments, mrcc should also determine the default output file name according to the input source name. It is a little hard because the cc option rules are pretty complex. After that, mrcc calls the cpp_maybe() function to fork another process to run the preprocessor. The preprocessor insert header files into the source file. So the compiler version on the master and slaves needn’t to be identical since the remote server needn’t to do preprocessing.

Then mrcc compiles the source remotely or locally depending on the compiler options. Local compilation is straight forward. mrcc wait for the preprocessor process to complete and fork another process to do the compilation.

If the source file can be compiled on remote server, mrcc calls the function compile_remote(). mrcc should first wait for preprocessor to complete. Then mrcc put the preprocessed file into the network file system. After that, mrcc calls call_mapper() to generate relevant options and arguments from the id of the temporary file and the options and arguments of the compiler. The it runs the job on MapReduce. In this project, we use Hadoop Streaming to run the mrcc-map on the slaves. The options for mrcc-map are passed with the streaming job options.

After mrcc-map successfully finishes the job, the object file now exists on the network file system. mrcc generates the output file name from the id of the temporary file and calls get_result_fs() to retrieve the object file from the network file system.

As soon as mrcc gets the object file, it returns so that make can continue to compile other source files that may depend on the completed one. But there are still several temporary files and directories on the network file system and local disk that needed to be cleaned up. We use a tricky function atexit() to do the cleaning up work after mrcc returns. By using atexit() some of the clean up I/O operation are done while the next compilation is running to improve the overall performance.

mrcc-map

mrcc-map is put in local directories on all the slaves servers. We doing this to avoid distributing the mrcc-map when the job is called because the mrcc-map does not changed during compiling one project. The flow of mrcc-map is simpler than mrcc as shown in Figure 3.

mrcc-map first parse it’s arguments and finds out the file id generated by mrcc and the compilation options and arguments. Then mrcc-map gets the preprocessed file from net file system according to the id. After getting the preprocessed file, mrcc-map calls the local gcc/cc compiler and passes the compilation options and arguments to it. Then mrcc-map puts the object file to the netwrok file system and returns immediately. The clean up work is done after mrcc-map finishes to parallel I/O operation and the compilation in the same way that mrcc does.

Figure 3: mrcc-map work flow

Evaluation

Please refer to this paper for the evaluation experiments and results:

Z. Ma and L. Gu.The Limitation of MapReduce: A Probing Case and a Lightweight Solution. In Proc. of the 1st Intl. Conf. on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2010). Nov. 21-26, 2010, Lisbon, Portugal. (PDF from one author’s homepage)

Installation and configuration

Install hadoop

Install Hadoop into a directory, such as /lhome/zma/hadoop and start Hadoop.

Please refer to this tutorial for how to install and start Hadoop: Hadoop Installation Tutorial

Configuration

In netfsutils.c:

As an example, we assume hadoop command’s full path is: /lhome/zma/hadoop/bin/hadoop

Set the variables as follows (We use the full path to avoid bad PATH environment).

const char* put_file_fs_cmd = "/lhome/zma/hadoop/bin/hadoop dfs -put";
const char* get_file_fs_cmd = "/lhome/zma/hadoop/bin/hadoop dfs -get";
const char* del_file_fs_cmd = "/lhome/zma/hadoop/bin/hadoop dfs -rm";
const char* del_dir_fs_cmd = "/lhome/zma/hadoop/bin/hadoop dfs -rmr";

In mrutils.c

Set the variables and We also use the full path.

const char* mr_exec_cmd_prefix = "/lhome/zma/hadoop/bin/hadoop jar /lhome/zma/hadoop/contrib/streaming/hadoop-0.20.2-streaming.jar -mapper ";
const char* mr_exec_cmd_mapper = "/home/zma/bin/mrcc-map ";
const char* mr_exec_cmd_parameter = "-numReduceTasks 0 -input null -output ";

Compile and install:

$ make
# make install

How to use it

Compile the project using mrcc on MapReduce

As building a project:

$ make -j16 CC=mrcc

-jN: N is the maximum parallel compilation number. N may be set to the number of CPUs in the cluster.

Enable detailed logging if needed

On master node in the same shell where mrcc runs:

$ . docs/mrcc.export

Or set environment variables:

MRCC_LOG=./mrcc.log
MRCC_VERBOSE=1
export MRCC_LOG
export MRCC_VERBOSE

Related work

GNU make^[7] can execute many commands simultaneously by using “-j” option. But make can only execute many commands on local machine. mrcc makes use of this feature of make and distributed these commands (compilation) across servers on MapReduce.

distcc^[6] is a program to distribute compilation of C/C++ Objective C/C++ code between several machines on a network. distcc must use a daemon on every slaves and the master must controls all the jobs details such as load balance and failure tolerance. distcc is easy to set up and fast in a cluster with a small amount of machines, but it will be hard to manage this system when distcc is running on a computing platform with thousands of machines.

References

[1] Dean, J. and Ghemawat, S. 2004. MapReduce: simplified data processing on large clusters. In Proceedings of the 6th Conference on Symposium on Operating Systems Design & Implementation – Volume 6 (San Francisco, CA, December 06 – 08, 2004).
[2] Hadoop project. http://hadoop.apache.org/
[3] mrcc project. http://fclose.com/p/mrcc/
[4] Hadoop 0.20.1. http://hadoop.apache.org/common/releases.html#Download
[5] Linux. http://www.kernel.org
[6] distcc project. http://code.google.com/p/distcc/
[7] GNU make. http://www.gnu.org/software/make/
[8] Zhiqiang Ma and Lin Gu.The Limitation of MapReduce: A Probing Case and a Lightweight Solution. In Proc. of the 1st Intl. Conf. on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2010). Nov. 21-26, 2010, Lisbon, Portugal.

Update history:
Jun. 1, 2010. Revise the report;
Jun. 2, 2010. Change this to mrcc’s documents.;
Nov. 19, 2010. Change mrcc project homepage URL;
Feb. 25, 2011. Add paper reference for evaluation result;
Aug. 6, 2011. Add MRlite.
Spe. 8, 2011. Add installation, configuration and how to use it.

Vim + cgdb

Tue, 15 Dec 2009 04:33:17 PST

I begin to use vim for programming. I like the tools that focus on one function. I used to use emacs. But I think I like vim more.

For debugging, I use gdb. But I use the front end cgdb. I can see the codes with the cursor when debugging. I can use F8 for next instead of n + enter. And F10 for step instead of s + enter. i is the key for enter gdb command mode. And ESC can change the user to command mode. But ESC is too far away from my finger, so I prefer “Ctrl + [” to ESC ;)

My configuration files can be found here if they are useful for someone:
http://github.com/zma/config_files

In the config file of cgdb:
“set winsplit=top_big: is for letting the code window bigger than the gdb command windows, which is convenient for debugging.

A list of useful cgdb shortcuts:

`F5'
     Send a run command to GDB.
`F6'
     Send a continue command to GDB.
`F7'
     Send a finish command to GDB.
`F8'
     Send a next command to GDB.
`F10'
     Send a step command to GDB.
`spacebar'
     Sets a breakpoint at the current line number.
`t'
     Sets a temporary breakpoint at the current line number.
`i'
     Puts the user into "GDB mode".
`I'
     Puts the user into "TTY mode".

These short cuts is available in cgdb 0.6.5.

Updated on Apr. 2 2010. Short cuts are added.

Default Descending Sort Order and Inline Quote When Reply in Thunderbird

Thu, 03 Dec 2009 20:53:57 PST

I like to descending sort order when read mails. And I like inline quote when reply.

This is the configuration:

mail.reply_quote_inline = true
mailnews.default_sort_order = 2

Here: mailnews.default_sort_order: ascending 1, descending 2

Hadoop Installation Tutorial

Tue, 24 Nov 2009 01:48:37 PST

Hadoop mainly consists of two parts: Hadoop MapReduce and HDFS. Hadoop MapReduce is a programming model and software framework for writing applications, which is an open-source variant of MapReduce that is initially designed and implemented by Google for processing and generating large data sets [1]. HDFS is Hadoop’s underlying data persistency layer, which is loosely modelled after Google file system GFS [2]. Hadoop has seen active development activities and increasing adoption. Many cloud computing services, such as Amazon EC2, provide MapReduce functions, and the research community uses MapReduce and Hadoop to solve data-intensive problems in bioinformatics, computational finance, chemistry, and environmental science [3]. Although MapReduce has its limitations [3], it is an important framework to process large data sets.

How to set up a Hadoop environment in a cluster is introduced in this tutorial. In this tutorial, we set up a Hadoop cluster, one node runs as the NameNode, one node runs as the JobTracker and many nodes runs as the TaskTracker (slaves).

First we assume we have created a Linux user “hadoop” on each nodes that we use and the “hadoop” user’s home directory is “/home/hadoop/”.

Enable “hadoop” user to password-less SSH login to slaves

Just for our convenience, make sure the “hadoop” user from NameNode and JobTracker can ssh to the slaves without password so that we need not to input the password every time.

Details about password-less SSH login can be found Enabling Password-less ssh Login.

Install softwared needed by Hadoop

Java JDK:

Java JDK can be downloaded form: http://java.sun.com/. Then we can install (actually copy the jdk directory) Java JDK on all nodes of the Hadoop cluster.

As an example in this tutorial, the JDK is installed into

/home/hadoop/jdk1.6.0_24

I provide a simple bash script to duplicate the JDK directory to all nodes:

$ for i in `cat nodes`; do scp -rq /home/hadoop/jdk1.6.0_24 hadoop@$i:/home/hadoop/; done;

‘nodes’ is a file that contains all the nodes IPs or host names, one in one line.

Hadoop

Hadoop softwar can be downloaded from here. In this tutorial, we use Hadoop 0.20.203.0.

Then we can install Hadoop on all nodes of the Hadoop cluster.

We can directly unpack it to a directory. In this example, we store it in

/home/hadoop/hadoop/

which is a directory under the hadoop Linux user’s home directory.

The hadoop directory can also be duplicated to all nodes using the script above.

Configure environment variables of “hadoop” user

We assume the “hadoop” user use bash as its shell.

Add these two lines at the bottom of ~/.bashrc on all nodes:

export HADOOP_COMMON_HOME="/home/hadoop/hadoop/"
export PATH=$HADOOP_COMMON_HOME/bin/:$PATH

The HADOOP_COMMON_HOME environment variable is used by Hadoop’s utility scripts, and it must be set, otherwise the scripts may report an error message “Hadoop common not found”.

The second line adds hadoop’s bin directory to the PATH sothat we can directly run hadoop’s commands without specifying the full path to it.

Configure Hadoop

conf/hadoop-env.h

Add or change these lines to specify the JAVA_HOME and directory to store the logs:

export JAVA_HOME=/home/hadoop/jdk1.6.0_24
export HADOOP_LOG_DIR=/home/hadoop/data/logs

conf/core-site.xml

Here the NameNode runs on 10.1.1.30.

<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://10.1.1.30:9000</value>
</property>
</configuration>

conf/hdfs-site.xml

<configuration>

<property>
<name>dfs.replication</name>
<value>3</value>
</property>

<property>
<name>dfs.name.dir</name>
<value>/lhome/hadoop/data/dfs/name/</value>
</property>

<property>
<name>dfs.data.dir</name>
<value>/lhome/hadoop/data/dfs/data/</value>
</property>

</configuration>

dfs.replication is the number of replicas of each block. dfs.name.dir is the path on the local filesystem where the NameNode stores the namespace and transactions logs persistently. dfs.data.dir is comma-separated list of paths on the local filesystem of a DataNode where it stores its blocks.

conf/mapred-site.xml

Here the JobTracker runs on 10.1.1.2.

<configuration>

<property>
<name>mapred.job.tracker</name>
<value>10.1.1.2:9001</value>
</property>

<property>
<name>mapred.system.dir</name>
<value>/hadoop/data/mapred/system/</value>
</property>

<property>
<name>mapred.local.dir</name>
<value>/lhome/hadoop/data/mapred/local/</value>
</property>

</configuration>

mapreduce.jobtracker.address is host or IP and port of JobTracker. mapreduce.jobtracker.system.dir is the path on the HDFS where where the Map/Reduce framework stores system files. mapreduce.cluster.local.dir is comma-separated list of paths on the local filesystem where temporary MapReduce data is written.

conf/slaves

Delete localhost and add all the names of the TaskTrackers, each in on line. For example:

jobtrackname1
jobtrackname2
jobtrackname3
jobtrackname4
jobtrackname5
jobtrackname6

Duplicate Hadoop configuration files to all nodes

We may duplicate the configuration files under conf diretory to all nodes. The script mentioned above can be used.

By now, we have finished copying Hadoop softwares and configuring the Hadoop. Now let’s have some fun with Hadoop.

Start Hadoop

We need to start both the HDFS and MapReduce to start Hadoop.

Format a new HDFS

On NameNode (10.1.1.30):

$ hadoop namenode -format

Remember to delete HDFS’s local files on all nodes before re-formating it:

$ rm /home/hadoop/data /tmp/hadoop-hadoop -rf

Start HDFS

On NameNode (10.1.1.30):

$ start-dfs.sh

Check the HDFS status:

On NameNode (10.1.1.30):

$ hadoop dfsadmin -report

There may be less nodes listed in the report than we actually have. We can try it again.

Start mapred:

On JobTracker (10.1.1.2):

$ start-mapred.sh

Check job status:

$ hadoop job -list

Run Hadoop jobs

A simple example

We run a simple example built in Hadoop’s distribution. For easy-to-run and more larger tests, please consider the Simple Sort Benchmark on Hadoop.

Copy the input files into the distributed filesystem:

$ hadoop fs -put /home/hadoop/hadoop/conf input

Run some of the examples:

$ hadoop jar /home/hadoop/hadoop/hadoop-examples-*.jar grep input output 'dfs[a-z.]+'

Examine the output files:

Copy the output files from the distributed filesystem to the local
filesytem and examine them:

$ hadoop fs -get output output
$ cat output/*

View the output files on the distributed filesystem:

$ hadoop fs -cat output/*

Shut down Hadoop cluster

We can stop Hadoop when we no long use it.

Stop HDFS on NameNode (10.1.1.30):

$ stop-dfs.sh

Stop JobTracker and TaskTrackers on JobTracker (10.1.1.2):

$ stop-mapred.sh

Some possible problems

Firewall blocks connections

Configure iptables: We can configure iptables to allow all connections, if these nodes are in a secure local area network which is most of the situation, by this command on all nodes:

# iptables -F
# service iptables save

For a list of the default ports used by Hadoop, please refer to: Hadoop Default Ports.

Pitfalls and Lessons

References

[1] J. Dean and S. Ghemawat, “MapReduce: simplified data processing on large clusters.” in the 6th Conference on Symposium on Operating Systems Design & Implementation, vol. 6, San Francisco, CA, 2004, pp. 137–150.
[2] S. Ghemawat, H. Gobioff, and S.-T. Leung, “The Google filesystem,” in Proc. of the 9th ACM Symposium on Operating Systems Principles (SOSP’03), 2003, pp. 29–43.
[3] Z. Ma and L. Gu. The limitation of MapReduce: A probing case and a lightweight solution. In CLOUD COMPUTING 2010: Proc. of the 1st Intl. Conf. on Cloud Computing, GRIDs, and Virtualization, pages 68–73, 2010.

Generating a Mixed Source and Assembly List Using GCC

Mon, 02 Nov 2009 06:53:40 PST

When do debugging and optimization, we sometimes need to generate and investigate into the assembly generated by the compiler. Generating a mixed source and assembly list will help the programmer a lot for debugging and optimization. gcc can achieve this by working the the assembler.

Generate assembly list mixed with the source code

Just add these gcc compile options:

-Wa,-adhln -g

The command:

$ gcc -Wa,-adhln -g source_code.c > assembly_list.s

The options:

 -g: Produce debugging information
 -Wa,option: Pass option as an option to the assembler
 -adhln:
 a: turn on listings
 d: omit debugging directives; n: omit forms processing
 h: include high-level source
 l: include assembly

One example

The source code:

helloworld.c:

#include <stdio.h>
int main()
{
  printf("Helloworld!n");
  return 0;
}

The command:

$ gcc -Wa,-adhln -g helloworld.c > helloworld.s

helloworld.s:

...

0:helloworld.c  **** #include <stdio.h>
1:helloworld.c  ****
2:helloworld.c  **** int main()
3:helloworld.c  **** {
19                            .loc 1 4 0
20                            .cfi_startproc
21 0000 55                    pushq   %rbp
22                    .LCFI0:
23                            .cfi_def_cfa_offset 16
24 0001 4889E5                movq    %rsp, %rbp
25                            .cfi_offset 6, -16
26                    .LCFI1:
27                            .cfi_def_cfa_register 6
4:helloworld.c  ****   printf("Helloworld!n");
28                            .loc 1 5 0
29 0004 BF000000              movl    $.LC0, %edi
29      00
30 0009 E8000000              call    puts
30      00
5:helloworld.c  ****   return 0;
31                            .loc 1 6 0
32 000e B8000000              movl    $0, %eax
32      00
6:helloworld.c  **** }

...

Updated history
17 Mar. 2010. Add example source code.
24 Feb. 2010.
Oct. 19, 2011. revise and make the writting better.

Set Up SSH Server on Windows XP

Fri, 18 Sep 2009 00:18:05 PDT

It is useful sometimes to set up a sshd server on a Windows. One example is using remote forwards tunnel to pass through the firewall.

My platform is Windows XP SP3.

The sshd server is sshwindows. It can be dowloaded from here: http://sshwindows.sourceforge.net/. The version I use here is 3.8p1-1 20040709.

It is really easy:

1) Run sshwindows installer and click OK and OK…

2) Run cmd.exe:

3) cd C:Program FilesOpenSSHbin (it depends on the sshd’s install location)

4) mkgroup -l >> ..etcgroup

5) mkpasswd -l >> ..etcpasswd
Now all local users are added to the sshd users list. More details can be found in the readme.txt file.

6) Configuration the firewall and let it allow the sshd service listening on port 22

7) Start the sshd service: “net start “OpenSSH Server”"

Now it is OK ;) Try to ssh to the windows and have fun.

There are some problems such as the default home is “/home/USERNAME”, but you just get a warning and you can change to the directory by yourself. At least, it works.

Updated on 7 Mar. 2010.

Firefox Add-ons For Easier and Happier Web Life

Fri, 04 Sep 2009 02:59:27 PDT

Firefox is highly configurable and there are lots add-ons for it. I compile a list of the add-ons that make my life easier and happier.

FireGestures

“A customizable mouse gestures extension which enables to execute various commands and user scripts with five types of gestures.”

Adbock Plus

“Annoyed by adverts? Troubled by tracking? Bothered by banners? Install Adblock Plus now to regain control of the internet and change the way that you view the web.”

Flashblock

“Never be annoyed by a Flash animation again! Blocks Flash so it won’t get in your way, but if you want to see it, just click on…”

FoxyProxy

“FoxyProxy is an advanced proxy management tool that completely replaces Firefox’s limited proxying capabilities. It offers more features than SwitchProxy, ProxyButton, QuickProxy, xyzproxy, ProxyTex, TorButton, etc.”

Firebug

“Firebug integrates with Firefox to put a wealth of development tools at your fingertips while you browse. You can edit, debug, and monitor CSS, HTML, and JavaScript live in any web page…”

Screenshoter

“Pretty simple and sexy screen capturing tool”

Obsolete ones

I list some add-ons that I ever used but not using now.

Google Toolbar: I use it to check the site’s PageRank (PR) and bookmark my favourite site to Google bookmark.

Hide Menubar: Hide the menubar which are not used most of the time.
or Compact Menu 2: Duplicates the menubar on the toolbar
The newer version of Firefox can do this natively.

Thunderbird中使用Google Calendar及它的SMS功能

Tue, 25 Aug 2009 18:44:54 PDT

Google Calendar可以自动发短信提醒,这是一个非常棒的功能.为何不把它集成到Thunderbird中来呢.

首先安装Thunderbird的插件:
Lightning 0.9

然后下载:
Provider for Google Calendar 0.5.2

provider默认加入的提醒是pop-up.其实要把它改成sms非常简单:

将下载下来的.xpi文件unzip, 它就是一个zip包:)

然后修改其中的calGoogleUtils.js JavaScript文件, 将其中的两个pop-up修改为sms.

然后打包,再到Thunderbird下安装即可.

这样就把Google Calendar和Thunderbird集成到一直来啦.

Windows XP SP3 换号

Thu, 23 Jul 2009 23:49:07 PDT

运行

regedit.exe

找到

HKEY_LOCAL_MACHINE\Software\Microsoft\WindowsNT\CurrentVersion\WPA\Events

然后随便修改其中的一个字符运行

oobe/msoobe /a

激活，选择电话，换号然后重启即可。

MFC中屏蔽ESC和回车关闭对话框

Sat, 04 Jul 2009 04:31:42 PDT

解决方法是在CDialog::PreTranslateMessage() 的重载函数中将ESC和回车按键的消息处理掉.
直接上代码:

BOOL
CResultCollectorDlg::PreTranslateMessage(MSG* pMsg)
{
  if(pMsg->message == WM_KEYDOWN) {
    switch(pMsg->wParam) {
      case VK_RETURN: //回车
        return TRUE;
      case VK_ESCAPE: //ESC
        return TRUE;
    }
  }
  return CDialog::PreTranslateMessage(pMsg);
}

Fclose.com feeds

Converting Video Files for iPod/iPhone/iPad

Install ffmpeg

Convert video file format with ffmpeg for iPod/iPhone/iPad

Convert video file format with ffmpeg for iPod/iPhone/iPad in one command

Hadoop Default Ports

Simple Sort Benchmark on Hadoop

Guangdong Mahjong Scoring Rules

广东麻将规则

基本规则

记分规则

特殊规则

Hiking Ngong Ping Trail – Ngong Ping Village to Tung Chung City Center

Hosting from Dreamhost

Upgrading Fedora to Newer Version with Yum

Clean Stuff

Do the upgrade

Go to a text console and run in level 3

Update yum first

Clean all cached meta data and packages

Import the key for the newer version’s repository

Synchronizes the installed package set with the latest packages available

Update groups

Use grub2 instead of grub

Reset services start/stop priorities

Reboot the system now

Additional Repositories for Fedora Linux

RPM Fusion

How to Install, Run and Uninstall VMware Player and VirtualBox on Fedora Linux

VMware Player

Install VMware Player

Run VMware Player

Uninstall VMware Player

VirtualBox

Install VirtualBox

Run VirtualBox

Uninstall VirtualBox

A Tutorial on Ns-2

Installing ns-2 and ns-3 on Fedora Linux

Install ns-2.34 on Fedora

Install ns-3.12.1 on Fedora

Hiking: MacLehose Trail Stage 5 – Tate’s Cairn to Tai Po Road

Inline Assembly with GCC on Linux

Ordered Apple iPhone 4S 16GB Black at 3Shop of Three Hong Kong

Installing and Configuring 32-bit Firefox on 64-bit Linux (Fedora)

Install 32-bit firefox

Install the flash-plugin

How to Set Up and Configure NFS Server and Clients

Set up a NFS Server

Edit /etc/exports

Start up the NFS service

Client-slide configuration

Package installation

Start the rpcbind service

Mount the NFS directory

x86-64 ISA / Assembly Programming References

x86-64 Assembly Programming

x86-64 (and x86) ISA Reference

Others

i386 References and Learning Resource

Binary interface

Howto for New Git Users

Users’ SSH private/public key pairs

Non-standard SSH port

Create the new repository

Use the repository

Installing the Flash Plugin for 64-bit Firefox in Linux x86-64

64-bit flash plugin for Firefox on Linux

32-bit flash plugin for Firefox on Linux

New Blog Address

A Free Personal WordPress Blog Solution

This Blog Passes W3C XHTML 1.0 Transitional Validation

Links

Frequently used links

Social networks

Website tools

TV programs

Banks and online pay

Sites I like

Peoples and friends