howto-adopt-gnus v2.3


Introduction

It is not for you if...

Okay, enough. Now, it is for you if...

To summarize, read on if you often receive more than 40 emails per day and therefore you need an efficient tool.


Prerequisites


What is it?

click to enlarge

Gnus is the most popular news/mail client for Emacs. It will allow you to send and receive news and emails.

Of course, with the help of regular expressions, all your emails will be sorted and located in different groups (for example, all the messages you receive from the mailing list info-gnu@gnu.org can go to a group named "info-gnu"), and you will browse through your messages in optional threaded mode, read them, forward some, write followups to others, etc.

An example is the best advocate, so on the right you can see my configuration when I initially wrote that documentation, showing 93 unread "articles" (article is a generic name to designate either a news message or an email message), sorted in different email groups (in purple) and newsgroups (in blue); those groups themselves are sorted under different topics (in grey). Click to enlarge.


How to set it up?

Receiving emails

The easiest will be to setup Gnus for reading your messages from a Unix mail spool (e.g. usually the file named /var/spool/<username> or /var/spool/mail/<username>) If you're on a network with the mail spools NFS-mounted from the mail server, end of discussion.

However, NFS is no longer very popular and you may have access to your email messages distantly, through the POP3 or IMAP protocols. To address this problem, fetchmail can grab the messages for you and deliver them locally in your mail spool with appropriate configuration. Here's the transcript of my ~/.fetchmailrc file to achieve this:

poll IP_MAILSERVER protocol pop3 username MY_LOGIN password MY_PASSWORD mda "/usr/bin/procmail -d %T"

If you need SSL

Some mail providers, for example gmail, use SSL for your privacy. In such a case, simply add options ssl to the end of the above configuration line.

If you can IMAP over SSH

For privacy, if the remote site supports IMAP and SSH, you can avoid using the POP3 protocol. First, be sure you can login via SSH without typing your password, by putting your key in the remote ~/.ssh/authorized_keys (see man ssh for more details), and then you can use a ~/.fetchmailrc like the following:

poll IP_MAILSERVER proto imap preauth ssh plugin "ssh MY_LOGIN@IP_MAILSERVER /usr/sbin/imapd" mda "/usr/bin/procmail -d %T"

Putting the IP of the mail server is good if it is static, of course; it will eventually address any temporary DNS problems. At the end of the line, we can see that the mda (mail delivery agent) is procmail, which will deliver the messages to your local spool file.

Start fetchmail in daemon mode, and just forget about it (for example: fetchmail -d 180 which will make fetchmail polling for new messages every 180 seconds).

Sending emails

For obscure reasons, gnus absolutely wanted to send the messages to the localhost for a long time - actually this is not really true anymore, but my configuration stayed like this for simplicity.

To do so, you need a mail server on the box you will be sending the messages.

If the machine you use already has a local mail server (for example, shared machine in your organization), end of discussion.

However, in many organizations, especially if you have priviledged access on your own machine, you might not have a mail server on your own machine. This also applies if you connect to the Internet directly or mostly directly (e.g. from home).

Installing a local mail server

Installing a local mail server is not difficult. Postfix is a good example. You only need to install the software, and add a few lines to the configuration file (usually /etc/postfix/main.cf). Here's what I had to add to my configuration file:

relayhost = mail.your-domain.com
masquerade_domains = $mydomain
myorigin = $mydomain

The first line is theoretically optional, but highly recommended, otherwise your machine will talk directly to the destination SMTP servers, and may be refused for antispam measures. Use the SMTP server adviced by the network admins or by the mail documentation of your provider, or use host -t MX orange.fr (if your provider is orange.fr) to know the name of your mail exchanger.

The masquerading and origin options are useful in order to produce email messages with a valid "From:" envelope; it will replace @your-machine.your-domain.com by @your-domain.com which will probably produce your valid email address.

If you're running a dialup box (ok at home there's not much remainings of that, but still if you do emails while commuting and you don't want to use a mobile data connection, you might be interested), the good thing is to setup your Postfix to defer the transport of the messages, e.g. the messages will be queued, and then only flushed when you connect to the Internet. To achieve this, you need to add this line to your configuration file: defer_transports = smtp. Then when you connect to the Internet, issue: sendmail -q.

Once your local mail server is set up, test it with the simple command line program mail (usually provided by nail or mailx) and monitor the mail log file (typically /var/log/syslog).

If you need STARTTLS

Some mail providers, for example gmail, use STARTTLS. In such a case, you need some additional configuration, though it is safer (and usually easier because no STARTTLS is needed) to stick with sending mail to the SMTP server of your ISP (for example, gmail has a list of banned IPs even when using STARTTLS, probably to fight abuse, whereas it will properly route @gmail.com emails coming from your ISP). To use STARTTLS with postfix:

smtp_use_tls = yes
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_tls_security_options = noanonymous

Have your mail and password in /etc/postfix/sasl_passwd:

[smtp.gmail.com] user@gmail.com:password

protect your file with chmod go-r /etc/postfix/sasl_passwd, and issue postmap /etc/postfix/sasl_passwd to hash the file. That is a simple configuration scheme, however it leads to warnings "certificate verification failed.." in the logs; you can go there if you prefer a complete and clean STARTTLS configuration.

This hasn't worked properly with postfix 2.5.1 though (don't know for sure if this is related to the version though). I had to add in /etc/postfix/transport:

gmail.com smtp:[smtp.gmail.com]:587

and postmap /etc/postfix/transport, else postfix logs tell that it's not possible to automatically trust Thawte signed certificate at gmail (don't understand fully the relationship of the error message actually).


Inside the .emacs

Now is the perfect time to tweak a little bit more your Emacs configuration file. Here's an excerpt of mine. The options are self-documented so I don't need to explain much more. Pick what you like. Credits go to Chmouel for the biggest part of this.

(require 'gnus-cite) 

(setq
 user-full-name "Myfirstname Mylastname"
 user-mail-address "mymail@your-domain.com"
 nnmail-spool-file "/var/spool/mail/mylogin"
 display-time-mail-file "/var/spool/mail/mylogin"
 nnml-directory "~/private/mail/"
 message-default-charset `iso-8859-15
 gnus-select-method '(nntp "news")
 gnus-secondary-select-methods '((nnml ""))
 gnus-signature-limit 500
 message-cite-function 'message-cite-original-without-signature
)

  ; this puts the messages in several groups according to regexp
(setq nnmail-split-methods
	  '(
	     ("Me" "^From.*gc.*andrake")

	     ("cooker" "^To.*cooker@.*andrake")
	     ("cooker" "^Cc.*cooker@.*andrake")
	     ("cooker" "^To.* cooker@")

	     ("Webmin" "webmin-l")

	     ("Info-gnu" "^To.*info-gnu@gnu.org");
	     ("Info-gnu" "^Cc.*info-gnu@gnu.org");

	     ("Other" "")))

(add-hook 'gnus-article-display-hook
	    '(lambda ()
	       (gnus-article-de-quoted-unreadable)
	       (gnus-article-emphasize)
	       (gnus-article-hide-boring-headers)
	       (gnus-article-hide-headers-if-wanted)
	       (gnus-article-hide-pgp)
	       (gnus-article-highlight)
	       (gnus-article-highlight-citation)
	       ))

(setq gnus-message-archive-method
	'(nnfolder "archive"
		   (nnfolder-inhibit-expiry t)
		   (nnfolder-active-file "~/Mail/sent-mail/active")
		   (nnfolder-directory "~/Mail/sent-mail/")
		   (nnfolder-get-new-mail nil)))
(setq gnus-message-archive-group
	'((if (message-news-p)
	      "sent-news" 
	    (concat "sent-mail-" (format-time-string 
				  "%Y-%m" (current-time))))))

The useful shortcuts to begin with

Okay, the last part is effectively using your new toy! The biggest problem of Emacs is the time when you still do not remember the necessary shortcuts. Here's the most needed ones. First, when you are reading the messages (e.g. when you are in the Group and Summary buffers, which show respectively the available groups, and the available articles in a given group) :

Editing the messages:


Advanced

Advanced shortcuts

In the Group buffer:

In the Summary buffer:

In the Article buffer, on a mime part:

Editing a new article:

Incorporating old mail

If you switch from an older (sucking) mail program to gnus, you may want to incorporate your old mail.

  1. Type 'G f' and give the path of the mbox file when prompted to create an `nndoc' group from the mbox file.
  2. Type 'SPACE' to enter the newly created group.
  3. Type 'M P b' to process-mark all articles in this group
  4. Type 'B r' to respool all the process-marked articles, and answer 'nnml' when prompted

Scoring

Create a ~/News/all.SCORE file and put things like that inside ('s' meaning substring and 'r' meaning regexp):

(("references"
  ("mymachine.mycompanynetwork.com" 50 nil s))
 ("subject"
  ("xmms" 50 nil s)
  ("XMMS" 50 nil s)
  ("[gG]rany" 500 nil r)
  ("install" 50 nil s)
  ("[dD]rak[xX]" 50 nil r)
  ("tage.*1" 50 nil r)
 )
 ("from"
  ("@redhat" 30 nil s)
  ("hmouel" 2 nil s)
  ("ixel" 2 nil s)
 )
 ("cc"
  ("gc@mandrakesoft.com" 100 nil s)
 )
 ("head"
  ("X-Mailer: Outlook" -3 nil s)
 )
 ("body"
  ("[gG]uillaume.*[cC]ottenceau" 25 nil r)
  ("gc@mandra" 25 nil s)
  (" gc " 25 nil s)
  )

  

Additionally, if you create a file named ~/News/groupname.SCORE, it will apply to any group called groupname. You may add the following to have a better Summary line:

(setq gnus-summary-line-format "%3i%U%R%I%[%4L: %-18,18n%] %s\n")
  

Footnotes

With the following in your ~/.emacs and footnote.el in your load path, you will type C-c ! a to insert a footnote in your message. Then C-u C-SPACE to come back to where you were in the message.

(require 'cl)
(require 'footnote)
(setq footnote-body-tag-spacing 1)
  

Handling sucking Ms-Word attachments

The following will enable the viewing of the text contents of Ms-Word documents directly inside the articles (thx to Pixel). It needs antiword.

(defun mm-inline-msword (handle)
  (let (text)
    (with-temp-buffer
      (mm-insert-part handle)
      (call-process-region (point-min) (point-max) "antiword" t t nil "-")
      (setq text (buffer-string)))
    (mm-insert-inline handle text)))

(add-to-list 'mm-automatic-display "application/msword")
(add-to-list 'mm-inlined-types "application/msword")
(add-to-list 'mm-inline-media-tests '("application/msword" mm-inline-msword identity))

; when the sucker provided the wrong mimetype, use the file extension
(add-to-list 'mm-inlined-types "application/octet-stream")
(add-to-list 'mm-inline-media-tests
	     '("application/octet-stream" mm-inline-msword
	       (lambda (handle)
		 (let ((name (mail-content-type-get (mm-handle-disposition handle) 'filename)))
		   (and name (equal ".doc" (substring name -4 nil)))
		   ))))
  

Using different signatures

To better show your 3l33t attitude, you *need* to use different alternate signatures, according to where you actually post your messages.

(setq gnus-posting-styles
       '(
         ("troll" (signature-file "~/.misc/signature.troll"))
         ("microsoft" (signature-file "~/.misc/signature.ms_sucks"))
	 ))
  

Better look

Notice: that unfortunately ceased to completely work when upgrading to recent versions of gnus; for some, it is possible to fix by removing the -face part of the face name, for some I was unable to understand why, even by using M-x describe-char :/

(require 'gnus-cite) 
(set-face-foreground 'gnus-summary-normal-read-face "LightBlue")
(set-face-foreground 'gnus-signature-face "lightsteelblue")
(copy-face 'default 'gnus-cite-face-1)
(set-face-foreground 'gnus-cite-face-1 "lightgray")
(copy-face 'default 'gnus-cite-face-2)
(set-face-foreground 'gnus-cite-face-2 "aquamarine")
(copy-face 'default 'gnus-cite-face-3)
(set-face-foreground 'gnus-cite-face-3 "lightsteelblue")
(copy-face 'default 'gnus-header-content-face)
(set-face-foreground 'gnus-header-content-face "pink")
(copy-face 'default 'gnus-header-from-face)
(set-face-foreground 'gnus-header-from-face "khaki")
(copy-face 'default 'gnus-header-name-face)
(set-face-foreground 'gnus-header-name-face "plum")
(copy-face 'italic 'gnus-header-subject-face)
(set-face-foreground 'gnus-header-subject-face "lightcoral")
(copy-face 'italic 'gnus-cite-attribution-face)
(set-face-foreground 'gnus-cite-attribution-face "tan")
(copy-face 'bold 'gnus-emphasis-bold)
(set-face-foreground 'gnus-emphasis-bold "mediumturquoise")
(set-face-foreground 'message-header-other-face "pink")
(set-face-foreground 'message-cited-text-face "lightgray")

(copy-face 'bold 'gnus-group-news-3-face)
(set-face-foreground 'gnus-group-news-3-face "LightBlue")
(set-face-foreground 'gnus-group-news-3-empty-face "LightBlue")
(copy-face 'bold 'gnus-group-mail-3-face)
(set-face-foreground 'gnus-group-mail-3-face "thistle")
(copy-face 'default 'gnus-group-mail-3-empty-face)
(set-face-foreground 'gnus-group-mail-3-empty-face "khaki")

(copy-face 'default 'gnus-cite-face-1)
(set-face-foreground 'gnus-cite-face-1 "lightgray")

(setq message-cite-function 'message-cite-original-without-signature)

(add-hook 'gnus-startup-hook 
	  '(lambda ()
	     (set-face-foreground 'bold "lightgray")
	     ))

(add-hook 'gnus-article-display-hook
          '(lambda ()
	     (gnus-article-de-quoted-unreadable)
	     (gnus-article-emphasize)
	     (gnus-article-hide-boring-headers)
	     (gnus-article-hide-headers-if-wanted)
	     (gnus-article-hide-pgp)
	     (gnus-article-highlight)
	     (gnus-article-highlight-citation)
	     (gnus-article-date-local)              ; will actually convert timestamp from other timezones to yours
             ))

Advanced splitting

Here's a more advanced splitting method (splitting = way to automatically split and sort the incoming mail).

The function you can see first is useful when you receive many mail: it allows to sort by month. That will much speed up gnus. It's also nice to remove or tarball the old mail.

The Fancy split allows you not seeing the crossposted mails in several groups.

(defun date (u) (concat (format-time-string "%Y-%m." (current-time)) u))

(setq nnmail-split-methods 'nnmail-split-fancy)
(setq nnmail-split-fancy 
      `(|
	("X-Spam-Status" "Yes.*" ,(date "Spam"))
	("X-Loop" "future.*" ,(date "future"))
	("X-Mailer" "phpGroupWare" "groupware")

	(any ".*gcottenceau@april.org.*" "april-perso")

	("To" ".*gc@mandrakesoft.com.*" ,(date "Other"))
	("Cc" ".*gc@mandrakesoft.com.*" ,(date "Other"))
	))
  

Advanced searching

Builtin search is slow. You can grab searching love by using an indexer. I personally had luck using swish-e and nnir.el, with some documentation found on Emacs website. Then G-G and feel the relief. The only caveat is swish-e is very memory hungry :/


Last update: Tue Mar 1 17:37:46 2011