File System Commands
- Linux look at everything as a file. Linux file system will have tens of thousands of files. If you write a program you add one more file to the system. When you compile it you are adding at least two more.
- Linux file is a storehouse of information. It is simply a sequence of charaters. linux places no restriction on the structure of the file. A file contains exactly those bytes that you out in to it, be it a source program executable code or anything else. It neither contains it's own size not it's attributes. It doesn't contain it's own name.
- Although everything is treated as a file by linux it's still divided files into various categories:
Ordinary files, device files, directory files etc.
Commands:
pwd: Checking current directory:
- When you log in you are placed in a specific directory of the file system. This directory is known as your current directory. At any time you should be able to know what your current directory. At any time, you should be able to know what your current directory is. For this we have to use the "pwd" command.
$pwd
/usr/ben
cd: changing directories:
- We can move around in the file system using cd command. It changes the current directory to the directory specified as the argument. For example:
$pwd
/user/username
$cd progs
$pwd
/user/username/progs
- The command cd progs means change your subdirectory to progs under the current directory. If we use absolute pathname that also the effect will be the same. You can use like this also
cd /user/username/progs
If we want to switch to the /bin directory where most of the Unix/Linux commands are kept in that case we have to use the absolute pathname.
If we use
cd without ant argument then it reverts to the home directory.
mkdir: Making Directories:
- We can create directories with mkdir command. This command is followed by the named of the directories to create. If we want to make the directory by the name patch.
$mkdir patch
We can create number of sub directories in a single shot.
$mkdir one two three
Unix/Linux systems lets we create directory chain with just one command. For example:
$mkdir foo foo/one foo/two
This command creates three subdirectories- foo & two subdirectories under foo.
But we can't enter like this-
$mkdir foo/one foo/two foo
This command failed to create two subdirectories one & two but still creates foo
directory.
rmdir: Removing Directories:
- We can remove directories with rmdir command. For example:
$rmdir patch
Like mkdir, rmdir can also delete more than one directory in a single shot. For example:
$rmdir one two tree
Also the tree directories & subdirectories that we just prepared using mkdir can be removed by using rmdir with reverse set of arguments.
$ rmdir foo/one foo/two foo
But we can't enter like this.
$rmdir foo foo/one foo/two
In this case, rmdir will silently delete the lowest levels of directories one & two. You should remember the following rules while deleting the directories.
- "you can't delete the directory unless it is empty. In this case foo directory can't be removed because of the existence of the directories one & two.
- You can't remove a subdirectory unless you are placed in a directory, which is hierarchically above the one you have chosen to remove.
To understand this look at the following example.
$cd progs
$pwd
/user/useraname/progs
$rmdir progs
rmdir:progs:Directory does not exit.
To remove this directory we must position ourselves in the directory above progs that is username & remove it from there.
$cd /user/username
$pwd
/user/username
$rmdir progs
The mkdir and rmdir commands work with only those directories, which are owned by the user.
ls: Listing Files:
This command is used to list all the files in the current directory.
$ls
We can see here there is a complete list of filenames in the current directory. Arranged in ASCII collating sequence with one filename in each line.
If you want a particular file then just use ls with particular filename.
$ls patch
patch
-x Option:Output In Multiple columns:
When we have several files then it is better to display the filename in multiple columns. So we need the -x option to produce the multi-columnar output.
-F option: Identifying Directories & Executables:
With the help of ls command we got only the filenames, but we didn't know how many of them are directories & executable files. To get this -F option should be used.
$ls -F
Note that the two symbols that tag some of the filenames the * indicates the file containing the executable code & the / refers to a directory.
-a Option:Showing Hidden Files:
ls command doesn't show all the files in a directory. Some of files are hidden files in every directory. This option lists all the files even hidden files also.
All these files beginning with a '.' are normally not listed out only with ls.
-r Option: Reversing the sort order:
We can reverse the order of the presentation of the list with the -r option. The sorting is done in the reverse ASCII collating order.
$ls -r
The ls command will work differently if we give the two directory names with it.
$ls directory1 directory2
Here is the list of important option:
| Options |
Description |
| -x |
Display multi-columnar output |
| -r |
Sorts files in reverse order |
| -R
|
Recursive listing of all files in subdirectories |
| -F
|
Makes executables with * & directories with / |
| -a |
Shows all files including hidden ones |
| -i |
Shows inode number of files |
| -l |
Long listing showing seven attributes of flies |
| -d |
Force listing of directories |
| -t |
Sorts files by modification time |
| -u |
Sorts files by access time |
cat : Displaying and Creating Files:
cat command is generally used to display the contents of a file.
$cat filename
cat command also accepts more than one filenames
$cat file1 file2
The contents of second file are shown immediately after the contents of first file. cat is normally used to display the text files only.
Using
cat to
create file:
cat is used for creating files. Look at the following examples:
$cat>one
This command shows the inserting of data into the files. ( Ctrl -d)
$ prompt returns
When we terminate the command line after pressing the enter key the prompt vanishes & now waits for input the user side. Finally press the control-d to signify the end of data input.
cp: Copying of files:
Copy the contents of one file to another with the cp command.
syntax
cp [options] old_filename new_filename
you now have two copies of the file, each with identical contents. They are completely independent of each other and you can edit and modify either as needed. They each have their own inode, data blocks and directory table entries.
We can also copy files to another directory. For example
cp one progs/two
cp can also be used to copy more than one file in a single shot. In that case the last filename must be a directory. For example : to copy files one, two, tree to the progs directory we have to write.
$cp one two tree progs
To work this command it is necessary that progs directory should be there.
you can also use * metacharacter if the names having same starting characters. For example: if filename are page1, page2, page3.
cp page* progs
Common Options
-i : interactive ( prompt and wait for confirmation before proceeding)
$cp -i one two
cp:overwrite two?y/n
-r: recursively copy a directory
It is possible to copy entire directory structure with this option. For example the following command copies all files & subdirectories from progs to newprogs.
cp -r progs newprogs
rm: Deleting files:
Remove a file with the rm, remove command.
syntax
rm [option] filename
common options
-i : interactive ( prompt and wait for confirmation before proceeding )
-r : recursively remove a directory, first removing the files and
subdirectories beneath it.
-f : don't prompt for confirmation (overrides -i)
Examples:
$rm old_filename
A listing of the directory will now show that the file no longer exists. Actually all you've done is to remove the directory table entry and mark the inode as unused. The file contents are still on the disk, but the system now has no way of identifying those data blocks with a file name. There is no command to
"unremove" a file that have been removed in this way. For this reason many novice users alias their remove command to be
"rm -i" where the -i option prompts them to answer yes or no before the file is removed.
rm dosen't normally remove directory but removes the files from it. You can remove two files from foo directory without having to "cd" to it.
$rm foo/one/ foo/two
sometimes it is required to delete all the files of directory then we can use the * metacharacter which represents all files. For example:
$rm *
mv: Renaming Files:
Rename a file with the move command,
mv.
Syntax:
mv [option] old_filename new_filename
Common Options:
-i interactive (prompt and wait for confirmation before preceeding)
-f don't prompt even when copying over and existing target file ( overrides -i )
Example:
$mv old_filename new_filename
You now have a file called new_filename and the file old_filename is gone. Actually all you've done is to update the directory table entry to give the file a new name. The contents of the file remain where they were.
mv can also be used to rename a directory.
more:Paging Output:
If the file too large to display it's contents on screen, then the contents will scroll off the screen when we view it with cat. So a pager is required more allows the user to view a file one screen at a time. To view the contents of a file page1 use the following command with the filename.
$more page1
You will see the contents on the screen one page at a time. At the bottom of the screen, you will see the filename with the percentage of file content that has been viewed.
more command also contains some internal commands-
q- this command is used to exit more.
f- to scroll forward one screen.
b- to scroll backward one screen.
to move even slowly only one line at a time use, j key for scrolling.
forward and k for scrolling backward.
we can use
more with multiple filenames.
more page1 page2 page3
you will first see the contents of the first file preceded by it's name after that the message "page1:ENDS(next file: page2)" and then the contents of the next file. In this way it display the contents of all the files. In between we can switch to previous or next files also using :n or :p for next and previous voyage.
lp (Line Printing): Printing A File:
Users are not allowed to directly access the printer. But users can spool ( or line up) a job with others in a print queue. With spooling one can orderly print the jobs relieve the user from the necessity of administering the usage of print resources. The spooling facility is provided by the
lp command. Take a look at following example:
$lp page1
request id is pr1-100(1 file )
This command will print a single copy of first page. This command also gives the request id. It is the combination of printer name & job number.
If there are more than one printer in a system, we have to use the -d option with the printer name if the printer name is dotmatrix.
$lp -ddotmatrix page1
The another option we can use with lp is -t ( title ) followed by the title string. this command prints the title on the first page.
$lp -t "first page" page1
We can cancel our jobs with cancel command which requires the request id of the job as an argument.
$ cancel pr1-100
file: To know the file types:
There are three of files. i.e. ordinary files directory files & device files. A regular file may contain simple text a C program or executable file. This file command is used to determine the type of file.
$file filename.lst
filename.lst: English text
This command you can use with more filename also. We can apply this command to all the files in the progs directory. Just look at the following example ( * to indicate all the files )
$file /proc/*
wc: Line, Words and Character Count:
This command counts lines, words and characters depending on the options used. It takes one or more file name as an argument. Before you use wc command take a look at the following example
$create a file type some lines.
$ cat > filename
$wc filename
This command offers three option to make specific count. This -l counts only number of lines, -w counts number of words and -c counts number of characters.
$wc -l filename
$wc -w filename
$wc -c filename
we can use this command with multiple file names, wc produces a separate line for each file.
$ wc file1 file2 file3
split: Splitting a file into multiple files:
Some file are so large that some times we found difficulties in editing them. So there's a need to split them. The split command breaks up its input into several equi-line segments and all these segments are created as separate file in the current directory. Split by default break up a file into 1000 line pieces.
$split file1
This command creates a group of files xaa, xab, xac like this up to xzz. If the file is not large enough to occupy 1000 line you can override this default figure of 1000 by specifying the size.
$split -101 filename
Each file will contain 101 lines.
cmp: Comparing Two files:
Some times two files are exactly identical to each other, in such cases you can delete one of them. There are three commands to do this comparison one of them is cmp command. this command requires two file names as an argument.
$cmp file1 file2
This command compares the two files byte by byte and location of the first mismatch gives on the screen. The -l option gives a detailed list of byte number and differing bytes in octal for each character, that differs in both files.
$cmp -l file1 file2
If the two files are identical the cmp does not display many message but it simple returns the $ prompt.
diff: Differences in files:
The diff command compares two files, directories, etc and reports all differences between the two. It deals only with ASCII files. It's output format is designed to report the changes necessary to convert the first file into the second.
Syntax:
diff [option] file1 file2
Common Options
-b ignore trailing blanks
-i ignore the case of letters
-w ignore and characters
-e produce and output formated for use with the editor, ed
-r apply diff recursively through common sub-directories
Examples:
For the file1 and file2 files, the difference between them:
Note that the output lists the differences as well as in which file the difference exists.
Lines in the first file are preceded by "<", and those in the second file are preceded by ">".
File Permissions:
Each file, directory and executable has permission set for who can rad, write, and/or execute it. To find the permissions assigned to a file, the ls command with the -l option should be used. Also using the -g option with "ls -l" will help when it is necessary to know the group for which the permissions are set ( BSD only ).
When using the "ls -lg" command on a file ( ls -l on sysV), the output will appear as follows.
-rwx-r-x--- user unixgroup size month nn hh:mm filename
The are above designated by letters and dashes ( -rwx-r-x---) is the area showing the file type and permissions. Therefore, a permission string for example of -rwx-r-x--- allows the use ( owner ) of the read, write and execute it; those in the unixgroup of the file can rad and execute it; others cannot access it at all.
Chmod - Change File Permissions:
The command to change permissions on an item ( file, directory, etc) is chmod ( changes mode). The syntax involves using the command with three digits ( representing the user ( owner, u) permission, the group (g) permissions, and other (o) user's permissions) followed by the argument (which may be a file name or list of files and directories). Or by using symbolic representation for the permissions and who they apply to.
Each of the permission types is represented by either a numeric equivalent:
read=4, write=2, execute=1
or single letter:
read-=r, write=w, execute=x
A permission of 4 or r would specify read permissions. If the permission desired are read and write, the 4 (representing rad) and the 2 (representing write) are added together to make a permission of 6. Therefore, a permission setting of 6 would allow read and write permissions.
Alternatively you could use symbolic notation which uses the one letter representation for who and for the permissions and an operator, where the operator can be:
+ add permissions
- remove permissions
= set permissions
So to set read and write the owner we could use "u=rw" in symbolic notation.
Syntax:
$ chmod nnn [argument list] numeric mode
$ chmod [ who]op[prem] [argument list] symbolic mode
Where nnn are the three numbers representing user, group and other permissions, who is any u, g, o or a (all) and perm is any of r, w, x . In symbolic natation you can separate permission specifications by commas as shown in the example below.
Common Options:
-f force (no error message is generated if the change is unsucessful)
-R recursively descend through the directory structure and change the modes.
Example:
If the permission desired for file1 is user: read, write, execute, group: read, execute, other: rad, execute the command to use would be
$ chmod 755 file2 or chmod u=rwx,go=rx file1
Reminder: When giving permissions to group and other to use a file, it is necessary to allow a least execute permission to the directories for the path in which the file is located. The easiest way to do this is to be in the directory for which permissions need to be granted.
chmod 7111. or chmod u=rw,+x. or chmod u=rwx.go=x.
where the dot (.) indicates this directory.
Shell
Shell is command interpreter that execute commands. A Unix shell is both a command interpreter, which provides the user interface to the rich set of unix utilities, and a programming language allowing these utilities to be combined. Files containing commands can be created and become commands themselves. These new commands have the same status as system command in directories like '/bin', allowing users or groups to establish custom environments.
A shell allows execution of unix command both synchronously and asynchronously. The shell waits for synchronous commands to complete before accepting more input; asynchronous commands continue to execute in parallel with the shell while it reads and executes additional commands. The redirection construct permit fine-grained control of the input and output of those commands and the shell allows control over the contents of their environment. Unix shells also provide a small set of built in commands ( builtins ) implementing functionality impossible ( e.g. cd, break, continue and exec ), or inconvenient ( history, getopts, kill or pwd for example ) to obtain via separate utilities. Shells may be used interactively or non-interactively they accept input typed from the keyboard or from a file.
While executing commands is essential most of the power ( and complexity ) of shells is due to their embedded programming languages. Like any high-level language, the shell provides variables flow control constructs, quoting and functions.
Shells have begun offering features geared specifically for interactive use rather than to augment the programming language. These interactive features include job control command line editing history and aliases.
Shell Operations:
The following is a brief description of the shell's operation when it reads and executes a command. Basically the shell does the following:
Reads it input from a file, from a string supplied as an argument to the '-c' invocation or from the user's terminal.
Breaks the input into words and operators, obeying the quoting rules. These tokens are separated by metacharacters. Alias expansion is performed by this step.
Parses the tokens into simple and compound commands.
Performs the various shell expansions, breaking the expanded tokens into lists of filenames and commands and arguments.
Performs any necessary redirections and removes the redirection operators and their operands from the argument list.
Executes the command.
Optionally waits for the command to complete and collects its exit status.
comments:
In a non-interactive shell or an interactive shell in which the interactive_comments option to the shops built in is enabled a word beginning with '#' causes that word and all remaining characters on that line to be ignored. An interactive shell without the interactive_comments option enable does not allow comments. The interactive_comments option is on by default in interactive shells.
Pipelines:
A pipeline is a sequence of simple commands separated by '|'. The formate for a pipeline is -
[time [-p]] [!] command1 [|command2..]
The output of each command in the pipeline is connected to the input of the next command. That is each command reads the previous command's output.
Pattern Matching : Wild cards:
Up to this point we used certain commands with more than one filename (e.g., page1 page2, page3) as arguments. Often you may require to enter a number of such filenames in the command line. To illustrate this look at the following examples:
$ls -l file1 file2 file3
Each filename has a common string viz., page. But this command line requires lot of typing work & doesn't look good. Fortunately shell offers solution to this.
The * And ?
this * is known as metacharacter. this character matches any number of characters. When this character is appended to the string page the pattern page * expands in to all files in which first four characters constitute to string page. The previous command line can now be shortened with this sequence.
$ ls -l file*
Now when shell encounters the above command line it looks for metacharacters & identifies the * immediately. It then matches all the files in the current directory with the pattern page start and replace the pattern with the list that matches the pattern. It then reconstructs the command line as below & passes it to the kernel for execution.
$ls -l file1 file2 file3
The next character is ? , which matches a single character. When used with the same string page (page?) the shell matches all five character filenames beginning with page. Placing another? at the end of this string creates the pattern page?? where page is followed by two characters. Look at the following example.
$ls -x file?
Pattern Matching
Any character that appears in a pattern other than the special pattern characters described below, matches itself. The NULL character may not occur in a pattern. The special pattern characters must be quoted if they are to be matched literally.
The special pattern characters have the following meanings:
* Matches any string including the null string.
? Matches any single character.
[...] Matches any one of the enclosed characters. A pair of characters separated by a minus sign denotes a range; any character lexically between those two character, inclusive is matched. If the first character following the `['is a ` !' or a `^' then any character not enclosed is matched. A `_' may be matched by including it as the first or last character in the set. A `}' may be matched by including it as the first character in the set. Within `[' and `]' , character classes can be specified using the syntax [:class:], where class is one of the following classes defined in the POSIX.2 standard:
alnum alpha ascii blank cntrl digit graph lower print punct space upper sdigit
A character class matches any character belonging to that class. Within `[' and `]', an equivalence class can be specified using the syntax [=c=], which matches all characters with the same collation weight ( as defined by the current locale ) as the character c. Within `[' and `]', the syntax [.symbol.] matches the collating symbol symbol.
If the extglob shell option is enable using the shopt built in, several extended pattern-matching operators are recognized. In the following description, a pattern-list is a list of one or more patterns separated by a `|'. Composite patterns may be formed using one or more of the following sub-patterns:
? ( pattern-list)
Matches zero or one occurrence of the given patterns.
* ( pattern-list)
Matches zero or more occurrences of the given patterns.
+ ( pattern-list)
Matches one or more occurrences of the given patterns.
@ ( pattern-list)
Matches exactly one of the given patterns.
! ( pattern-list)
Matches anything except one of the given patterns.
When Wild - Cards lose their meaning:
The metacharacters * and ? lose their meaning when used inside the class & are matched literally. Similarly - & ! also lose their significance when placed outside the class. Additionally ! loses it's meaning when placed anywhere but at the beginning of the class. The - loses it's meaning if it is not bounded properly on either side by a single character.
Matching the Dot:
The * doesn't match all the files beginning with a . (dot) or the / of a pathename. If you want to list all the hidden files in your directory having at least 3 characters after the dot, then the dot or the must be matched explicitly.
$ls -x .???*
When using rm with *:
We use these metacharacter to speed up our interaction with the system but there are great risks involved if you are not alert. When using shell metacharacters especially the *, you could be totally at sea if instead of typing
$rm file*
Which removes all file. You inadvertently introduces space.
$rm file * ( space between page and * )
Even if there is not file named file in the current directory, and you get an error message to indicate that the command will still removed all files in this directory because of singular existence of *. This happens because shell rightly treats the * as a separate argument.
Escaping: The Backslash (\):
It is generally accepted principle that file names should not contain the shell meta characters. What happens if we do that imagine a file named page* created with the >.
Symbol:
$>file* # you can create any empty file like this
How do we remove this file without deleting other files. ( If we have other files named file1, file2, file3). the shell uses another characters \(Backslash), to remove the special meaning of any metacharacter placed after it. Here the shell has to be told that the asterisk has to treated & matched literally, instead of being interpreted as metacharacter use the \ before * to solve the problem.
$ ls -x file\*
file*
$ rm file\*
Escaping The < ENTER> Key:
When you enter a long chain of commands you can split the command line by hitting enter key. But only after the \ escapes this key.
$wc -l file1 \ ( Enter)
> file2 file3 ( Press Enter )
Quoting:
What happens when we enter a shell metacharacter with echo command as below:
$echo *
showing files and directories
You simply see list of all files in the current directory. Why that happens? The shell uses the * meta character to match the files in the current directory. All files matched so you see all of them in the output. Suppose you intend to literally echo a * without permitting the shell to interfere. Escaping with a \, solves the problem.
$echo \*
There is another solution. When the argument to a command is enclosed in quotes the meaning of all special characters are turn off.
$echo '*' ( you can use double quotes also )
Quoting Preserves Spaces:
Whenever a shell finds contiguous spaces & tabs in the command line, it compresses them to a single space. Look at the following example:
$echo write one or two lines with so many space.
The above arguments to echo could have been preserved by escaping the space character, whenever it occurs at least twice.
$ echo dflfjdasl\ \ \sdafsda\ \ \sdfsad\ \ \sdfsad\.
In this situation quoting is recommended.
$echo "sdfklsajlfjsadl;"
grep: Searching For A Pattern:
grep is one of the most useful unix filter. It scans the file for a specific occurrence of a pattern, and displays the selected pattern the filenames where that pattern occurs or the line number in which they found. The syntax for it is as follows- grep option pattern filename (s)
Look at the following example
$ps -aux | gerp firefox
grep when unable to find the required pattern then it silently returns the prompt. Take a look at this -
$ grep production record1.lst
$- #No production person found
It is generally safe to quote the pattern though always it is not required. Quoting is essential if the search string consist of more than one word. Try the following example -
$grep baiju damoder record.lst
gerp:can't open damodar
gerp interprets damodar as a filename and so unable to open that file. but still searches the record1.lst and gives the result. To correct this problem you quote the pattern just like -
$ grep 'jiten' record1.lst
When we use grep with a series of string it interprets the first arguments as the pattern & the rest as filenames. It then displays along with the output.
$grep manager record1.lst record2.lst
grep options:
Option
|
Significance
|
-c
|
Displays count of number of occurances
|
-l
|
Displays list of filename only
|
-n
|
Display line number along with lines
|
-v
|
Displays all but lines matching expressions
|
-i
|
Ignores care for matching
|
-h
|
Omits filenamewhrn handling multiple files
|
-n
|
Display line & n lines above & below
|
