Hadoop FileSystem (HDFS) Tutorial 1

In this tutorial I will show some common commands for HDFS operations.
If you don't have Hadoop setup in your linux, you can follow Hadoop Setup Guide

Log into Linux, "hduser" is the login used in following examples.

Start Hadoop If it's not running

$ start-dfs.sh
....
$ start-yarn.sh

Create someFile.txt in your home directory

hduser@ubuntu:~$ vi someFile.txt

Paste any text you want in to the file and save it.

Create Home Directory In HDFS (If it doesn't exist)

hduser@ubuntu:~$ hadoop fs -mkdir -p /user/hduser

Copy file someFile.txt from local disk to the user’s directory in HDFS.

hduser@ubuntu:~$ hadoop fs -copyFromLocal someFile.txt someFile.txt

Get a directory listing of the user’s home directory in HDFS

hduser@ubuntu:~$ hadoop fs –ls


Found 1 items
-rw-r--r--   1 hduser supergroup          5 2013-10-27 17:57 someFile.txt

Display the contents of the HDFS file /user/hduser/someFile.txt

hduser@ubuntu:~$ hadoop fs –cat /user/hduser/someFile.txt

Get a directory listing of the HDFS root directory

hduser@ubuntu:~$ hadoop fs –ls /

copy that file to the local disk, named as someFile2.txt

hduser@ubuntu:~$ hadoop fs –copyToLocal /user/hduser/someFile.txt someFile2.txt

Delete the file from hadoop hdfs

hduser@ubuntu:~$ hadoop fs –rm someFile.txt

Deleted someFile.txt

For a full list of commands, Please visit HDFS FileSystem Shell Commands. Please feel free to leave me any comments or suggestions.