To count all the files by file extension recursively on the command line $ find . -type f | sed -n ’s/..*.//p’ | sort | uniq -c 40 3g2 5 AVI 13 DS_Store 28 JPG 30 MOV 133 MP4 64 THM 1 docx 18 jpg 1 json 4 m3u 89 m4a 2 m4r 156 m4v 41 mkv 112 mov 38 mp3 587 mp4 1 nfo 2 osp 30 png 1 sh 4 srt 6 svg 10 torrent 6 txt 5 webm 10 zip

Sometimes you may need to copy an entire directory structure to another system using the command line. Here is a quick way to do it using the tar command:

cat myfile.tgz | ssh user@host “tar xzf - -C /some/dir”

If you have a text file that has funny looking ^M characters at the end of each line, in most cases, you have to get rid of them before they can be used. This is especially the case when you’ve copied or transferred a file from a Windows-based system to a *nix-based one. If these files are shell scripts meant to run on the *nix-based system they more often than not won’t work. There are various solutions to this problem.

In my day job I find that I frequently need to remove those pesky ‘^M’ line endings from text files that are transferred from one system to the other. Most of the time it is just one file that needs to be fixed so going into vi and typing

:%s/^M//g

solves the problem, but on occasion I might be confronted with performing this process on multiple files and sometimes recursively through many subdirectories. When that happens it becomes necessary to bring out the “big guns” and do:

Sometimes you just need to hash something. Here are some command lines to do just that.

Windows PowerShell:

Get-FileHash C:\path\to\file.iso -Algorithm [MD5|SHA1|SHA256|SHA384|SHA512|MACTripleDES|RIPEMD160]

(SHA256 is the default if the parameter is omitted)

Linux:

md5sum /path/to/file sha1sum /path/to/file sha256sum /path/to/file

Sometimes you need to copy an entire directory structure to another location on the command line. Here is a quick way to do it using the tar command:

tar cf - * | ( cd /target; tar xfp -)

To sort files in a directory and to list them by the biggest file first:

$ du -a | sort -n -r | more

Alternatively you can use:

$ du -s * | sort -n -r | more

Sometimes I need to know how many times a specific character occurs in a string. The best tool to use for something like that is AWK. Let’s say we’re looking for the count of commas in a string: string=“text,text,text,text” char="," echo “${string}” | awk -F"${char}" ‘{print NF-1}’

To backup files from a source directory to a backup directory:

$ mkdir /path/to/backup/directory
$ cd $HOME/source/directory
$ tar cf - . | (cd /path/to/backup/directory && tar xBvf -)

Define the following alias:

tar cvf - . | ( cd \!* ; tar xvf - )

or as an alias:

alias cpbytar='tar cvf - . | ( cd \!* ; tar xvf - )'

(The alias definition above is for Bash)

To do a recursive copy of a directory to another location, preserving the PERMISSIONS and OWNERSHIP of the files. “cd” to the source location and invoke the following alias: