Basics of GnuPG (GPG) command in Linux

What is GPG ?

GnuPG (GPG)Gnu Privacy Guard or GnuPG or popularly known as GPG is a GPL Licensed alternative to PGP (Pretty Good Privacy) and its openPGP complaint program for *nix people based on rfc 4880. It is part of GNU software project started in 1991 by Werner Koch and majorly funded by German Government.  Download its pdf format from here.

Its basic use is to send encrypted mails or files  to the recipient who can decrypt these using its private key. It is based on public and private key mechanism for encryption/decryption. We can encrypt any of our data using our own key pair and send it to the person who can read the message if he has the proper key to decrypt it! Many people use public key generated by gpg to verify his email signature too!

 
It uses following algorithm for various purposes used for safe message communication:
Pubkey: RSA, RSA-E, RSA-S, ELG-E, DSA
Cipher: 3DES, CAST5, BLOWFISH, AES, AES192, AES256, TWOFISH
Hash: MD5, SHA1, RIPEMD160, SHA256, SHA384, SHA512, SHA224
Compression: Uncompressed, ZIP, ZLIB, BZIP2
 

Why GPG?

  • Because it is free and meant to be a replacement of PGP.
  • Gpg is a CLI program but there are many GUI also through which you can manage keys easily like seahorse for GNOME (yum install seahorse) and KGpg for KDE.
  • It allows you to encrypt and sign your data, includes a key management system as well as access modules for all kind of public key directories.
  •  If you wish to encrypt your message while sending mail to someone important, you may try this method.
  • You can share your public key and other users can download it to verify signature in mails/files sent by you for authenticity. It would stop social engineering through email even would stop spams send in the name of your friend’s id.

 Applications of GPG

  • GPG encryption has been added to graphical email client like Evolution for email security.
  • There is a GNOME front-end application for managing PGP and SSH keys called “Seahorse” which integrates with Nautilus, gedit and Evolution too for encryption, decryption etc.
  • PHP based email framework “horde” uses it too!
  • Enigmail is a data encryption/decryption extension for Mozilla Thunderbird and the SeaMonkey which uses GPG
  • Mozilla Firefox also gets GPG enabled using Enigform.
  • GnuPG is being used for Windows Explorer and Outlook through GPG4win tool which are wrapped in the standard Windows installer to make GnuPG easier to get installed and to be used in Windows systems.
  • There are many frontend softwares that support GPG.

 

How GPG works

It uses hybrid encryption techniques i.e. it uses a combination of symmetric key cryptography for speed and public-key cryptography for easy secured key exchange. By default GnuPG uses the CAST5 symmetrical algorithm.
 
As a matter of fact, GnuPG does not use patented or otherwise restricted software or algorithms. Instead, GnuPG uses a variety of other, non-patented algorithms.
 
It will be clearer that how GnuPG works once we see the working of gpg commands step by step:

  • Which version of gpg we are going to use?
  • gpg command to generate keys
  • Analysis of freshly created directory (.gnupg) and files inside it.
  • Once you get public and private key. You must keep private key safe, once you forget it then you will never be able to decrypt the data. So, better take private key backup.
  • Want to see the list of public and private keys?
  • Encrypt the message for specific recipient
  • Decrypt the encrypted message

 

GPG commands explained

Which version of gpg we are going to use?

[vim][sjaiswal@AlienCoders ~]$ gpg –version
gpg (GnuPG) 1.4.5
Copyright (C) 2006 Free Software Foundation, Inc.
This program comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it
under certain conditions. See the file COPYING for details.
 
Home: ~/.gnupg
Supported algorithms:
Pubkey: RSA, RSA-E, RSA-S, ELG-E, DSA
Cipher: 3DES, CAST5, BLOWFISH, AES, AES192, AES256, TWOFISH
Hash: MD5, SHA1, RIPEMD160, SHA256, SHA384, SHA512, SHA224
Compression: Uncompressed, ZIP, ZLIB, BZIP2
[sjaiswal@AlienCoders ~]$
[/vim]
 

gpg command to generate keys

 [vim][sjaiswal@AlienCoders ~/gpg_test]$ gpg –gen-key
gpg (GnuPG) 1.4.5; Copyright (C) 2006 Free Software Foundation, Inc.
This program comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it
under certain conditions. See the file COPYING for details.
 
Please select what kind of key you want:
   (1) DSA and Elgamal (default)
   (2) DSA (sign only)
   (5) RSA (sign only)
Your selection? 1
DSA keypair will have 1024 bits.
ELG-E keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 4094
Requested keysize is 4094 bits
rounded up to 4096 bits
Please specify how long the key should be valid.
         0 = key does not expire
      <n>  = key expires in n days
      <n>w = key expires in n weeks
      <n>m = key expires in n months
      <n>y = key expires in n years
Key is valid for? (0) 3m
Key expires at Mon 03 Feb 2014 04:46:09 AM MST
Is this correct? (y/N) y
 
You need a user ID to identify your key; the software constructs the user ID
from the Real Name, Comment and Email Address in this form:
    "Heinrich Heine (Der Dichter) <heinrichh@duesseldorf.de>"
 
Real name: Sanjeev Jaiswal
Email address: sjaiswal@gmail.com
Comment: "GPG Key Test"
You selected this USER-ID:
    "Sanjeev Jaiswal ("GPG Key Test") <sjaiswal@gmail.com>"
 
Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? O
You need a Passphrase to protect your secret key.
 
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
.++++++++++.+++++++++++++++..+++++..++++++++++++++++++++.+++++++++++++++++++++++++.
 
+gpg: /home/sjaiswal/.gnupg/trustdb.gpg: trustdb created
gpg: key CBE9BE42 marked as ultimately trusted
public and secret key created and signed.
 
gpg: checking the trustdb
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0  valid:   1  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 1u
gpg: next trustdb check due at 2014-02-03
pub   1024D/CBE9BE42 2013-11-05 [expires: 2014-02-03]
      Key fingerprint = 40B3 E709 81DA 43AF 1F64  117B DF03 A4AD CBE9 BE42
uid                  Sanjeev Jaiswal ("GPG Key Test") <sjaiswal@gmail.com>
sub   4096g/38765DB9 2013-11-05 [expires: 2014-02-03]
[/vim]
 
Note:  

  • To generate keys using gpg, it would ask which kind of key you wish to use; you can choose any of the given option. Type 1 or 2 or 5.
  • Type the keysize between the given range
  • Then provide the expiration date of key. You can use days,weeks, months, years.
  • Once you are done with expiration days, use the next option carefully. Type Real name, Email and comment appropriately as it will be used while encrypting the data and will ask the recipient name. it will match recipient name before matching the keys.
  • Then type anything using keyboard, do mouse activities etc to speed up random generation of keys else it may take lot of time.
  • Once it will get created,  .gnupg directory under your home directory will be there. Use ls to see what all files got created.

 

Analysis of freshly created directory (.gnupg) and files inside it

[vim] $ ls .gnupg/
gpg.conf  pubring.gpg  pubring.gpg~  random_seed  secring.gpg  trustdb.gpg
[/vim]

  • gpg.conf -> it contains all options set by you. Unless you specify which option file to use (with the command line option "–options filename"), GnuPG uses the file ~/.gnupg/gpg.conf  by default. Check strings gpg.conf for more details.
  • pubring.gpg -> public key stored here. You should export it in ASCII format to send it to others.
  • pubring.gpg~ -> backup of public key
  • random_seed -> it contains all random keys used for encryption that you might be typing while generating keys.

[vim][Sanjeev@AlienCoders]$ strings random_seed
u-~N
bqUk
9a<b
vxyv
H@W@
02H]
KC!%
^v9@'
,i~JZ8Y
T_3     P>
[/vim]

  • secring.gpg -> it’s the secret key ring and one should keep it safe. Better have its backup
  • trustdb.gpg -> its trusted db which contains signatures, expiration date etc. and from time to time the trust database must be updated so that expired keys or signatures and the resulting changes in the Web of Trust can be tracked.

Normally, GnuPG will calculate when this is required and do it automatically.
 
 

Getting fingerprint and KeyID

[vim][sjaiswal@AlienCoders ~/.gnupg]$ gpg –fingerprint sjaiswal@gmail.com
pub   1024D/CBE9BE42 2013-11-05 [expires: 2014-02-03]
      Key fingerprint = 40B3 E709 81DA 43AF 1F64  117B DF03 A4AD CBE9 BE42
uid                  Sanjeev Jaiswal ("GPG Key Test") <sjaiswal@gmail.com>
sub   4096g/38765DB9 2013-11-05 [expires: 2014-02-03]
[/vim]
 
Note: KeyID here is: 0xCBE9BE42  (always prepend 0x as it is 8 hex digits)
 

Taking backup of private key

[vim]gpg –export-secret-keys –armor sjaiswal@gmail.com > sjaiswal-privkey.asc
[/vim]
 

Listing public and private key(s)

[vim][sjaiswal@AlienCoders ~/.gnupg]$ gpg –list-keys
/home/sjaiswal/.gnupg/pubring.gpg
———————————
pub   1024D/CBE9BE42 2013-11-05 [expires: 2014-02-03]
uid                  Sanjeev Jaiswal ("GPG Key Test") <sjaiswal@gmail.com>
sub   4096g/38765DB9 2013-11-05 [expires: 2014-02-03]
 
[sjaiswal@AlienCoders ~/.gnupg]$ gpg –list-secret-keys
/home/sjaiswal/.gnupg/secring.gpg
———————————
sec   1024D/CBE9BE42 2013-11-05 [expires: 2014-02-03]
uid                  Sanjeev Jaiswal ("GPG Key Test") <sjaiswal@gmail.com>
ssb   4096g/38765DB9 2013-11-05
[/vim]
 

Encrypting Message for recipient Sanjeev Jaiswal

Type the message and save it in text file, let’s say message.txt
[vim][sjaiswal@AlienCoders ~/.gnupg]$gpg recipient Sanjeev Jaiswal  –encrypt message.txt
[/vim]
 
It will create message.txt.gpg , which is an encrypted file. To decrypt it, you need to type passphrase that you had typed while generating keys.
 Or
[vim]gpg  -r real-name –out secrets_to_aliencoders –encrypt secrets
[/vim]
 
which will have encrypted message in secrets_to_aliencoders

Decrypting the message

 
[vim][sjaiswal@AlienCoders ~/.gnupg]$ gpg –decrypt message.txt.gpg
[/vim]
 
You need a passphrase to unlock the secret key for
user: "Sanjeev Jaiswal ("GPG Key Test") <sjaiswal@gmail.com>"
4096-bit ELG-E key, ID 38765DB9, created 2013-11-05 (main key ID CBE9BE42)
 
gpg: encrypted with 4096-bit ELG-E key, ID 38765DB9, created 2013-11-05
      "Sanjeev Jaiswal ("GPG Key Test") <sjaiswal@gmail.com>"
Hi
This is Sabnjeev
 
Or
[vim][sjaiswal@AlienCoders ~/.gnupg]$ gpg  –output secrets_from_tom –decrypt secrets_to_aliencoders
[/vim]
 
 Which would save the decrypted message in secrets_from_sanjeev
 

Editing Key

[vim]gpg –edit-key sjaiswal@gmail.com
[/vim]
 

There is more:

Photo IDs

 GnuPG has the ability to add a photo ID to a public key, exactly as in recent Windows versions of PGP. A photo ID attached to a public key can help other users to identify the owner of the key. To add a photo ID to your own public key, use the command "gpg –edit-key <name>" and then enter "addphoto". GnuPG will ask for the filename of a suitable JPEG. No other types of image files can be used.
If you want to see a photo ID on a particular key, enter the command "–show-photos" before using the command "gpg –list-keys <name>". If <name> is omitted, GnuPG will display all the photos (if any) after listing all the keys in your public keyring. Alternatively, if you want photos to be displayed in all cases by default, you should uncomment the line "# show-photos" in the options file inside !GnuPGUser.
 

Output of trustdb

 [vim][Sanjeev@AlienCoders]$ gpg –update-trustdb
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0  valid:   1  signed:   7  trust: 0-, 0q, 0n, 0m, 0f, 1u
gpg: depth: 1  valid:   7  signed:   3  trust: 0-, 0q, 4n, 3m, 0f, 0u
gpg: the next trustdb check will be done on 2014-02-04
[/vim]
               
The first line shows you the actual trust policy used by your GnuPG installation, and which you can modify at your needs. It states that a key in your keyring is valid if it has been signed by at least 3 marginally trusted keys, or by at least one fully trusted key.
 
The second line describes the key of level 0, that is the key owned by you. It states that in your keyring you have one level zero key, which is signed by 7 keys. Furthermore among all the level zero keys, you have 0 of them for which you haven't yet evaluated the trust level. 0 of them are the keys for which you have no idea of which validity level to assign (q="I don't know or won't say"). You also have 0 keys that you do not trust at all (n="I do NOT trust"), 0 marginally trusted keys (m="I trust marginally"), 0 fully trusted keys (f="I trust fully") and 1 ultimately trusted keys (u="I trust ultimately").
 
The third line analyzes the keys of level 1 in your keyring. You have 7 fully valid keys, because you have personally signed them. Furthermore, among the keys that are stored in your keyring, you have 3 of them that are not signed directly by you, but are at least signed by one of the fully valid keys. The trust status counters have the same meaning of the ones in the second line. This time you have 4 keys signed by you but for which you do not trust at all the owner as signer of third party's keys. On the other side, 3 of the 7 keys that you have signed are marginally trusted. This means that you are only marginally confident that the owners of those keys can verify well the keys that they sign.
 
[vim][Sanjeev@AlienCoders]$ gpg –check-trustdb
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0  valid:   1  signed:   7  trust: 0-, 0q, 0n, 0m, 0f, 1u
gpg: next trustdb check due at 2014-02-04
[/vim]

If you wish to know other commands that you may use using gpg then try
[vim]man gpg
[/vim] or
[vim]gpg –help
[/vim]

We uploaded it in slideshare also:

 

Source: http://www.wikipedia.com
Reference: GnuPG Handbook

Read more

How to fix rpmfusion-(non)free repo error in CentOS 6.4

[Problem]: When I wanted to install any repo using yum command, I was getting error because of which nothing was getting installed properly through Yum command and I was using centos 6.4.
 
I was getting below errors.
[Error]:
  • Cannot find a valid baseurl for repo: rpmfusion-free
  • Cannot find a valid baseurl for repo: rpmfusion-nonfree
Read more

How to install VLC Media player in CentOS 6.x

How to install VLC Media player in CentOS 6.x step by step
I installed CentOS v6.4 in my system and started installing various applications to make my system a place for programmers with some entertainment applications. I tried to play mp3 songs and video songs but rhythm, Mplayer gave below errors for that I had to do lots of hard work just to listen a song, I was not in mood to explore and search and download those missing plugins.
 

So, I decided to install one of my all-time favorite player (KMPlayer when I am using Windows) VLC media player. It was not so easy to install it in CentOs as it seemed. I though downloading rpm package or tar file would do the things. But it required lots of steps before playing the first song in VLC.

You need to install listed four libraries before installing vlc in your local repository:

  • epel
  • remi
  • rpmfusion-free
  • rpmfusion-nonfree
Here are the working steps to install VLC Media player in CentOS 6.x:
 
Step 1
First you need to make a local repository of epel by using yum command (login as root to use yum command) below command:
 
# yum localinstall –nogpgcheck http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
 


 
Then type y to get epel installed properly. Then you will see the confirmation message like this:



Step 2
Now you need to install remi by using below command:
# yum localinstall –nogpgcheck http://rpms.famillecollet.com/enterprise/remi-release-6.rpm
 
 

Type y and complete the installation. You will see below image after successful completion.

 
 
Step 3
Now install rpmfusion free by typing below command
# yum localinstall –nogpgcheck
http://download1.rpmfusion.org/free/el/updates/6/i386/rpmfusion-free-release-6-1.noarch.rpm
 



Type y to complete the installation

 
 
 
Step 4
Now install rpmfusion nonfree library by using below command
# yum localinstall –nogpgcheck http://download1.rpmfusion.org/nonfree/el/updates/6/i386/rpmfusion-nonfree-release-6-1.noarch.rpm
 



Type y to complete the installation 
 

Step 5
You may skip this step. But just to confirm that all above libraries went well and you are able to find vlc info from the repository. If yes, then proceed to install VLC else you have some issue.
 
# yum –enablerepo=remi-test info vlc

 
 
Step 6
Now you can install VLC easily. Just type the below command:
# yum –enablerepo=remi-test install vlc
 



 
Step 7



Now it’s the time to play a song using VLC Media player. You may find it under application or use command prompt and type start vlc (be sure you are running the command as a non-root user else it will give error).
Read more

Whatever you knew about chmod is wrong

Something that you don’t know about Chmod under Linux

chmod in LinuxIf you are working on Linux environment and working on some programs/scripts then you might have used chmod often for sure. Many of us think that what’s there in chmod to remember. Why people write so many articles on one of the simplest command in Linux which even a newbie use it like anything.
Then my question to all Linux users is that “how many octets a full permissions set have?”. I know, many Linux users would say three within three seconds but it’s four literally. Stumped right?

Usually we write
[vim]
chmod 777 file
[/vim]
But what we are actually saying:
[vim]
chmod 0777 file
[/vim]

The first octet works the same way as the other three works, as it has the same 3 possible values that adds to make the octet :
4 – setuid (letter used for it: s)
2 – setgid (letter used for it: s)
1 – sticky bit (letter used for it: t)

Note: your first octet will always be reset to 0 when using chown or chgrp on files. We need not to worry when using chown and chgrp. It’s effective only on chmod then: D

Let’s see what those 3 values mean for the first octet.
Setuid (4 or s)
Using digit 4 or s for any file means you are setting a userid to that files i.e. setuid. If you do setuid on binary files, you're telling the operating system that you want this binary file to be executed as the owner of the binary file always.

So, let's say the permissions on a binary are set like this:
[vim]
# chmod 4750 some_binary
# ls -al some_binary
-rwsr-x— 1 root coders 0 Feb 13 21:43 some_binary
[/vim]

You'll notice the small 's' in the user permissions blocks instead of x  – this means that if a user on the system executes this binary, it will run as root with full root permissions. Anyone in the coders group can run this binary too! Because the execute bit is set for the group also. So, when the binary will get executed, it will run with root permissions. Isn’t it fascinating (Kind of masquerading without credentials 😛 )

Be cautious with setuid! Anything higher than 4750 can be proved dangerous else it would allow everyone to run the binary as the root user. Also, if you allow full access plus setuid, you will be opening yourself up for the real mess:

[vim]
# chmod 4777 some_binary
# ls -al some_binary
-rwsrwxrwx 1 root coders 0 Feb 13 21:43 some_binary
[/vim]

Not only can every user on the system execute this binary, but they can edit it before it runs at root! It goes without saying, but this could be used to beat up your system pretty badly. If you don’t allow enough user permissions for execution to anyone, Linux will throw the mysterious error as uppercase 'S' into your terminal:
[vim]
# chmod 4400 some_binary
# ls -al some_binary
-r-S—— 1 root coders 0 Feb 13 21:43 some_binary
[/vim]

Since no one can execute this script anyways (except root), you get the capital 'S' in output.

Setgid (2 or s -> little confusion right?)
Setgid is almost same as setuid, but the binary runs with the privileges of the owner group rather than the user's privileges. This isn't quite so useful in I guess, but in case you need to know what’s this so I wrote minimal amount of it :D, here's how the permissions come out:
[vim]
# chmod 2750 some_binary
# ls -al some_binary
-rwxr-s— 1 root coders 0 Feb 13 21:43 some_binary
[/vim]

And if you again neglect the execution permission to everyone you get ‘S’ but this time group’s execution place:
[vim]
# chmod 2400 some_binary
# ls -al some_binary
-r—-S— 1 root coders 0 Feb 13 21:43 some_binary
[/vim]

Sticky Bit (1 or t)
This one is the trickiest term for a Linux file permission, and very important too. Its best example is your world writable /tmp directory (or any other world writable location like /var/log/ etc). Since world writable locations allow users to do anything with creating, editing, appending, and deleting files, which can be a total haphazard if certain users share a common directory and users like me played a prank by deleting it: P.

Let's say users work in an organization and they work on a file in a world writeable directory. One user gets mad upon the false praise of other coders from his manager. So, he deleted all of the files that belong to that poor user L . Obviously, this could lead to a much more vulnerable situation. If you apply the sticky bit on the directory, the users can do anything they want to that files  but they can't write to or delete files which they didn't create (They can create new file though). Awesome feature, right? Now, how you can set the sticky bit:
[vim]
#chmod 1777 /tmp
# ls -ld /tmp
drwxrwxrwt 3 root root 4096 Feb 13 21:57 /tmp
[/vim]

If you want to see T instead of t in your output, please do the same mistake that I pointed in above two points i.e. don’t give execute permission to anyone. 😛
[vim]
#chmod 1744 /tmp
# ls -ld /tmp
drw-r–r-T 3 root root 4096 Feb 13 21:57 /tmp
[/vim]

How this setuid and setgid works on directory?
Setting the setgid bit on a directory means whatever files you have created in that directory will be owned by the group who owns the directory. No matter what your primary group is, any files you make will be owned by the group who owns the directory.

setuid is meant to implement on files especially binary files. So, setting the setuid bit on a directory has no effect in almost all Linux Distros. I didn’t test much on this to various *nix variants.

For your knowledge: The setuid bit was invented by Dennis Ritchie. His employer, AT&T, applied for a patent in 1972; the patent was granted in 1979 as patent number US 4135240 “Protection of data file contents”. The patent was later placed in the public domain.

So, if you are interested, please do some homework and let us know also what special stuffs you found there 😀

Read more

How to search files using the find command in Linux

How to search files using the “find” command in Linux
find command in LinuxIf you are at *nix system, whether you are System administrator, common user, programmer or whoever; you will surely need to find file using different criteria. For that *nix system has very powerful and efficient command called “find”.
The find command is a powerful *nix utility that allows the user to find files located in the file system through various criteria such as the file name, owner, group, size, inodes, when file was last accessed, when the file status was last changed, the file's permissions, even using regular expression pattern etc.
I will try to cover maximum usage of commands that you are gonna need for any such operations. I hope after reading those examples one will easily use find command for one's problem.

Syntax for find command:
[vim]
find where-to-look criteria what-to-do
[/vim]

Instead of explaining each options and then showing you its uses with examples, better to see the use of all options while going through different examples based on various requirements.

Examples
1.       Find a file "alien.coders" that exists somewhere in the file system
[vim]
$ find / -name alien.coders -print
[/vim]
 
If the file is found the path to the file will be printed as output. On most platforms the -print is optional, however, on some *NIX systems nothing will be printed without using print. If you don’t provide any arguments, find searches recursively through all the directories.

2.Find a file without searching network or mounted file systems
[vim]
$ find / -name alien.coders -print -xdev
[/vim]
I found it useful, when you have mounted network drives or file systems that you do not want searched (Like Windows box or other remote servers). This will surely increase the search speed greatly if the mounted file system is large or over a slow network. "-mount" does the same thing as "-xdev"  to make it compatible with other versions of find.

3.Find a file without showing "Permission Denied" messages
[vim]
$ find / -name alien.coders -print 2>/dev/null
[/vim]
When find tries to search a directory or file that you do not have permission to read the message "Permission Denied" will be output to the screen. The 2>/dev/null option sends these messages to /dev/null so that the found files are easily viewed.

4.Find a file, who's name ends with .coders, within the current directory and only search 3 directories deep
[vim]
$ find . -name *.coders -maxdepth 3 -print
[/vim]
-maxdepth option allows you to specify till how much deeper you want to search for a file by specifying n digit i.e. –max depth 3 in the above example.

5.Search directories "./dir1" and "./dir2" for a file "alien.coders"
[vim]
$ find ./dir1 ./dir2 -name alien.coders -print
[/vim]

6.Search for files that are owned by the user "aliens"
[vim]
$ find /alice/in/wonderland/ -user aliens -print
[/vim]
The files output will belong to the user "aliencs". Similar criteria are -uid to search for a user by their numerical id, -group to search by a group name, and -gid to search by a group id number.

7.Find a file that is a certain type. "-type l" searches for symbolic links ( manual page of find has lot more explanation on this)
[vim]
$ find /some/directory -type l -print
[/vim]
Several types of files can be searched for:

  • b    block (buffered) special
  • c    character (un-buffered) special
  • d    directory
  • p    named pipe (FIFO)
  • f     regular file
  • l     symbolic link
  • s    socket
  • D   door (Solaris)

8.Search for directories that contains the word "alien" but do not end with ".coders"
[vim]
$ find . -name '*alien*' ! -name '*.coders' -type d -print
[/vim]
The "!" allows you to exclude results that contain the phrases following it.
 
9.Search files which are modified between 10 and 60 minutes:
[vim]
find . -mmin +9 -mmin -61
[/vim]
 (-mmin  +or – n is for minutes and –mtime + or – n is for no of days ( n*24 hours format))
(-n means less than n and +n means more than n like -9 and +61 in that example)

10.find files n days older and above certain file size
We have already seen the use of mtime and we will combine with –size option to find files which are n days older and greater than some file size in *nix. This is very common scenario for system administrator people where they need to delete some large old files to free some space in the machine.
This example of find command will find which are more than 30 days old and size greater than 1MB (1024 Kilo bytes with k option or 1024*1024 Bytes with c option instead of M and G for GB i.e. –size +1G for greater than 1Gb file size).
[vim]
find . -mtime +10 -size +1M -exec ls -l {} \;
[/vim]

11.   Search files which are writable by both their owner and group:
[vim]
find . -perm -220
[/vim]
or
[vim]
find . -perm -g+w,u+w
[/vim]
 
The power of find
find becomes extremely useful when combined with other commands. One such combination would be using find and grep together or with xargs and awk etc.
[vim]
$ find dir-path -type f -name '*.txt'  -exec grep -s Aliens {} \; -print
[/vim]
This sequence uses find to look in give path for a file (-type f) with a name ending in .txt. It sends the files it finds to the grep command via the -exec option and grep searches the file found for any occurrences of the word "Aliens".
If the file is found it will be output to the screen and if the word "Aliens" is found, within one of the found files, the line that "Aliens" occurs in will also be output to the screen.

12.   Find files and print those files whose name contains core.two-or-more-numeric-digits :
Without Regular Expression
[vim]
find directory-path -name "core.[0-9][0-9]*[0-9]" | xargs ls -lRt | awk '{print $9}'
 [/vim]

With regular expression
[vim]
find directory-path   -type f -regex ".*/core\.[0-9]*" | xargs -r ls -lRt | awk '{print $9}'
[/vim]
 
13.   Find files using regular expression and avoid hard link count error
[vim]
find directory-path -noleaf -type f -regex  ".*/core\.[0-9]*" | xargs -r ls -lRt | awk '{print $9}'
[/vim]
 
Some points to remember while using find command in Linux:

  • All arguments are optional though, but what you will do without any argument? What I meant is, if you write just find, then also it will work.
[vim]
find
find .
find . -print
find -print
[/vim]

All commands would fetch you the same result. i.e it will display the pathnames of all files in the current directory and all sub directories.
 

  • If find command doesn’t locate any matching files then it will produce no output
  • You may specify as many place as you wish to search the file. Ex: find /etc/dev /home/aliencoders/ . –name test
  • -print action lists the names of files separated by a new line or even send find’s output to xrags though pipe which separates file names using white space. So, it may cause an error if space or new line is encountered in any file name.
    No issues, we have better solution. Use “-print0” instead 😀
  • For better format output, try to use printf similar to C programming language.
    ex: find . –name ‘[!.]*’ –printf ‘Name: %10f  Size: %5s \n’
  • Note: The ordering of find's options is important for getting the expected results as well as for performance reason.
What is the real difference between exec and xargs (Source: unix.com)?
find . -name H* -exec ls -l {} \; executes the command ls -l on each individual file.
find . -name H* | xargs ls -l constructs an argument list from the output of the find commend and passes it to ls.

Consider if the ouput of the find command produced:
H1
H2
H3

the first command would execute
ls -l H1
ls -l H2
ls -l H3

but the second would execute
ls -l H1 H2 H3
 

the second (xargs) is faster because xargs will collect file names and execute a command with as long as a length as possible. Often this will be just a single command.

If file name is having space then xargs will fail but exec will work because each filename is passes to exec as a single value, with special characters escaped, so it is as if the filename has been enclosed in single quotes.

 For more details about how to use different arguments and options type man find
 
Read more

How to remove ^M and other non-printable characters from the file in Linux

How to remove ^M and other non-printable characters from the file
If you are working or playing around with files on different O.S., and you are just copying/moving files from one O.S. based system to other. Unknowingly you are inviting non-printable character to come with! This specially happens when you open any file in Linux using any editor (vim in my case), which you got from windows based machines.  You will see ^M at the end of each line (It is usually treated as carriage return/linefeed in Linux environment which we got from its step brother Windows :p ).
Read more

Simple steps to verify email id existence

Email id exists or notOur team always face email existence problem when they try to:

  • Verify the  registered account holder’s email id or
  • While sending mail to anonymous user who posted comment with some fake email id or
  • To check, if that email address is still valid (ex: 2 years back emailed was valid but not now)

Well it’s not possible to know 100% whether given e-mail address exists in real or not but it will be handy in majority of the cases. You can do it manually also if you are in Linux box using host or nslookup and telnet command. I would prefer host command for this purpose.

How it works:

  • Check if domain name exists or not using ping command
  • If domain name exists then check if it has any mail handlers i.e. mx records in DNS. Use host or nslookup command for it
  • If mx record exists for that domain name use that mail handler using telnet command
  • Use telnet and smtp command to verify user email id. If you get 250/Ok for user email id. You may assume that it exists.

Here many things came into the picture while testing various mail ids from various mail handlers. I will discuss it under “possible ways of failures”

Steps to check the email existence manually

1.       Get the mail id to verify (we can get domain-name from this format to use it in our next step userid@domain-name . For ex:admin@aliencoders.org)

2.       Either use nslookup command or host command to get mail server address (Lists of mx records from DNS)

a.       Using nslookup:
nslookup –type=mx domain-name

b.      Using host :
host –t mx domain-name

3.       Get the least digital value’s mx record to use it in telnet

a.       telnet mxrecord port-no i.e. telnet mail.aliencoders.org 25

b.      Once connected use SMTP commands to know if email id exists or not. It will return 250 as a return code with OK or Accepted. Depending upon mail server configuration and Linux flavors too.

Let’s see it practically with all combination of commands, inputs and outputs 😀

Output from nslookup (it will show many more things which I have not shown here)

$ nslookup -type=mx gmail.com
Non-authoritative answer:
gmail.com       mail exchanger = 20 alt2.gmail-smtp-in.l.google.com.
gmail.com       mail exchanger = 30 alt3.gmail-smtp-in.l.google.com.
gmail.com       mail exchanger = 40 alt4.gmail-smtp-in.l.google.com.
gmail.com       mail exchanger = 5 gmail-smtp-in-v4v6.l.google.com.
gmail.com       mail exchanger = 10 alt1.gmail-smtp-in.l.google.com.

Output from host command ( I prefer this over nslookup )

$ host -t mx gmail.com
gmail.com mail is handled by 30 alt3.gmail-smtp-in.l.google.com.
gmail.com mail is handled by 40 alt4.gmail-smtp-in.l.google.com.
gmail.com mail is handled by 5 gmail-smtp-in-v4v6.l.google.com.
gmail.com mail is handled by 10 alt1.gmail-smtp-in.l.google.com.
gmail.com mail is handled by 20 alt2.gmail-smtp-in.l.google.com.

Get the value which has lowest ranked digit. In this case it is 5 so take gmail-smtp-in-v4v6.l.google.com for our purpose
Either use host or nslookup for the first stage.

Telnet outputs:

Step 1

telnet gmail-smtp-in-v4v6.l.google.com 25
Trying 173.194.77.27…
Connected to gmail-smtp-in-v4v6.l.google.com.
Escape character is '^]'.
220 mx.google.com ESMTP hk4si16050652obc.168

Step 2 (type helo, if it didn’t work try helo hi, if it didn’t work too try ehlo then )

helo
250 mx.google.com at your service

Step 3 (type any valid email id. Invalid email id will say ok but after rcpt to it will throw an error and don’t skip this step)

mail from: <sanjeev@aliencoders.org>
250 2.1.0 OK hk4si16050652obc.168

Step 4 (You need to provide email which needs to be verified.  Ok means it may exists. Not 1005 guaranteed on existence but we can infer that this email id may work)

rcpt to: <jassics@gmail.com>
250 2.1.5 OK hk4si16050652obc.168

Telnet outputs on failure (Various reasons are there)

1.       Network is unreachable. (sometimes it is reachable form one host but not from others)

telnet gmail-smtp-in-v4v6.l.google.com 25
Trying 74.125.127.26…

telnet: connect to address 74.125.127.26: Connection timed out
Trying 2001:4860:8005::1b…

telnet: connect to address 2001:4860:8005::1b: Network is unreachable

telnet: Unable to connect to remote host: Network is unreachable

2.       Error at mail from:<mail-id> :

  • Don’t forget to use <> while typing email id. jassics@gmail.com is wrong. jassics@gmail.com is correct
  • Never use space before or  after mail id
  • Don’t use single quote or double quote while typing email id

3.       If email id exists but it doesn’t show up 250/ok then below reasons may justify this:

  • Server response delayed
  • Email address black listed
  • Email does not exist  (even if it existed 2 years back )

Steps to check email existence using Perl

  1. Get Mail::CheckUser module from CPAN
  2. Install it either manually or using cpan command
  3. Write simple script using this module and check the output

Good thing with this script is:

  • You don’t need to use any command like host, nslookup telnet and don’t need to remember all those SMTP commands.
  • Just pass the email id rest thing it will do.
  • Internally it checks:
  • It checks the syntax of an email address.   
  • It checks if there any MX records or A records for the domain part of the email address.
  • It tries to connect to an email server directly via SMTP to check if mailbox is valid. It can detect bad mailboxes in many cases.

Here is the fully working Perl script

Why still this trick doesn’t assure 100% proof that email id exists or not?

There can be many reasons for not trusting on this script or the above said manual steps. Best way is to send an email to the given email id. If it bounced back, means it doesn’t exist or can’t be delivered. (I chat with a person in gtalk but can’t send an email to that person. Strange isn’t it).

  • There may be server delay
  • Email id may be black listed
  • It may say it exists if it has to bypass proxies
  • Final output will depend upon server settings which may be using  an algorithm to stop spammer like stuffs
  • For instance SMTP defines the VRFY command for this purpose but gmail doesn’t support it.
  • Never send bulk emails, your mail id will be flagged as a spammers by many hosting companies
  • You may not get proper result because network is unreachable or down by that time

Note: This was just for educational purpose and found helpful for web masters or admins. Don’t use it as spam stuffs, you IP address and your mail id will be blacklisted. Like admin@aliencoders.org has been blacklisted from various mail servers.

Read more