Below you can find some useful UNIX commands for exploring your next-gen data. You can find a more in depth UNIX tutorial here.

Clicking on the command will take you to its man page.

Getting Started | Man Pages | Navigating the File System | Mounting | Working With Text Files

Getting Started

clear or Ctrl-l: clear screen

up and down to navigate through previously used commands

Ctrl-r let's you search the command history of your shell
## Ctrl-r opens this query tool
## start typing part of the command you issued a while back and it will show
## up in the query tool -- I think the history gets deleted after logging out
(reverse-i-search)`':
use Tab to complete names of files and directories

exit or Ctrl-d: log out of shell

poweroff: one of several ways to shut down a computer from the shell prompt

Man Pages

man: displays the manual for the command you are interested in
To search a man page type / search term. To scroll through your search hits use n. To exit type q.
man man
## displays the man-page of man ;-)
man ssh
## displays man-page of the ssh client

Navigating the File System


Here is a nice introduction to the Unix file system. Reading this should clarify greatly how the file system on a typical Unix system looks like (Mac users may have to make some adjustments).

cd: move between directories
cd .. will take you back one directory in your path. You can use multiple .. arguments. For instance, if you are in /home/cartwrightlab/desktop and would like to get back to /home use cd ../..
pwd
/home/cartwrightlab/desktop
cd ../..
pwd
/home
cp: copy files
cp dir1/foo.txt dir2/
## copies foo.txt from dir1 to dir2
cp -r dir1/ dir2/
## copies dir1 and its contents to dir2
ls: list the contents in a directory
ls -l
## list files with detailed info
ls -a
## show all files including hidden files
ls -lh
## show a list of files including their sizes in human readable form
ls -lha
## well...
mkdir: make a new directory
mkdir foo
## makes directory foo
mv: move file or rename file
mv foo.txt newdirectory/foo.txt
## moves foo.txt into newdirectory
mv foo.txt cat.txt
## renames foo.txt to cat.txt
 
rename: rename multiple files according to a regular expression indicating a rule for renaming
rename 's/\.txt$/\.fasta/' *
## changes the file extension for all files (*) in a directory from .txt to .fasta
rm: remove files or directories
rm foo.txt
## delete foo.txt
rm -r foo
## delete the directory foo and its contents
find: search for files -- here's a great tutorial for harnessing the full power of find
find /home/user -name pattern
## searches for files in /home/user that match the pattern
df: report file system disk space usage
df -h
## reports the size, used space, and available
## space of every device available on the system
##
## useful if you want to know how much space you
## have left to work with -- Can I really extract this tarball?!
du: estimate file space usage / directory size
du -h /home/user
## estimates the size of the user's home directory
## and the contained directories
##
## can be used to estimate the size of any directory
pwd: print the path of the directory you are in

Note: All commands can be issued on the files you would like to work with from the directory they are in. They can also be issued from another directory. In this case use the path for your files relative to your working directory. Ex: Your file (file_1.txt) is in the directory home/cartwrightlab/Documents/TruSeq/ you are working in home/cartwrightlab/Documents and want to rename file_1 without leaving your current directory. The command would look like this:
mv TruSeq/file_1.txt TruSeq/newfile_1.txt
Notice that the entire path is not necessary since TruSeq is a subdirectory of Documents.

Mounting

mount: mount a device

A device can be a hard-drive. These are generally found in the directory /dev. The exact name differs from system to system. Let's say /dev/sdb1 is the USB drive you just plugged into your computer. You would mount it to the directory /mnt by doing the following:
mount /dev/sdb1 /mnt
umount: unmount file system

To unmount the drive from above example do this. The process may take a while to finish, as there may be data that has to be written to the device first!
umount /mnt

Working With Text Files

For additional information on working with text files see Editing text files.

less: print file contents to standard out; you can search patterns in less in the same way as the man pages
less foo.txt
head: print the first 10 lines of file to standard out
head foo.txt
to print the first 5 lines:
head -n 5 foo.txt
tail: print the last 10 lines of file to standard out
tail foo.txt
to print the last 5 lines:
tail -n 5 foo.txt
nano: text editor that is very easy to use
If all you would like to do is look at a file use less instead of nano. Nano is not ideal for editing very large files, see the editing text files post.
nano foo.txt
cat: concatenate files and print to standard out
cat foo.txt
## dump foo.txt to standard out (the screen)
cat foo1.txt foo2.txt foo3.txt
## dump foo1.txt through foo3.txt to standard out
cat *.txt
## dump all text files in current directory to standard out
grep: pattern search within file -- returns the lines in a file that contain the search pattern
## Simple pattern matching and full-fledged support for regular expressions.
## There are tons of how-to's on the web.
grep pattern foo.txt
## basic usage to look for pattern in file foo.txt
cat foo.txt | grep pattern
## same as above using cat and a pipe (see I/O Redirection)
cat *.txt | grep pattern
## search for a pattern in all text files in the present directory
uniq: report or omit repeated lines
cat foo.txt | uniq > foo-filtered.txt
## only retain uniq lines in foo.txt and save these in
## foo-filtered.txt
##
## think about filtering out duplicate sequences etc.
vi: powerful text editor with steep learning curve. Why would you learn the basics of vi? Because it's installed on pretty much any Unix(-like) computer you will encounter. Cheat sheet for reference.

wc: newline, word, and byte counts for each file
wc -l foo.txt
## counts the number of lines in foo.txt
cat foo.txt | wc -l
## same as above using a pipe (see I/O Redirection)