Th.Oughts : Hacking Gnus expiry

When mail grows in size, one thing that hurts the most for Gnus users is mail expiration. The default rule, as far as I understand it, is to run the process for a group when you exit it. Some admins/users set gnus-group-expire-all-groups to run when you exit gnus. So, an easy out is

(remove-hook 'gnus-summary-prepare-exit-hook
             'gnus-summary-expiry-articles)

(remove-hook 'gnus-Exit-group-hook
             'gnus-group-expire-all-groups)

However, the problem now is that you need to remember to manually run expiry once in a while. Someone like me can easily forget that!

Another option is to fetch mail locally before serving to Gnus. I have used offlineimap with pretty good results. But note that the simplest backend to use - nnfolder is also terribly slow when handling large amounts of mail. So, you would have to end up running a local dovecot instance too. On resource starved hardware, this could be a problem.

Another way, I got around this is by timing by expiration process. Mail expiry is meant to be fire-and-forget, but you really don't need to run it more than say, once a day.

  1. On the first run, when exiting a group, create an assoc list of the following format

    ((group1 . time1) (group2 . time2) ...
    
  2. Once we have a list with some entries is when the fun starts. We probably don't need to run the expiry process on a group that was just processed. We could check that with this example code snippet

    (let ((entry gnus-newsgroup-name))
      (if (check-entry-in-ignore-list entry)
          ;;use the default behavior
          (gnus-summary-expire-articles)
        (progn
          (let ((last-expiry-time (cdr (assoc entry gnus-expiry-table))))
            (if (and last-expiry-time
                     ( > (- (float-time) (string-to-number last-expiry-time))
                         gnus-group-expiry-time-limit))
                (progn
                  (let ((time (number-to-string(float-time))))
                    (setq gnus-expiry-table
                          (acons entry time gnus-expiry-table)))
                  (gnus-summary-expire-articles)))))))
    
  3. And finally, when exiting a group, we check times for all groups. But we are not done yet! We need to store the times we last ran expiration to a file

    (defun write-string-to-file (string file)
      "Simple Function to write a string of the form - \"group value\""
       (if string
           (with-temp-buffer
             (insert string)
             (when (file-writable-p file)
               (write-region (point-min)
                             (point-max)
                             file))))
    
  4. Things are different the next time you fire up Gnus. We need to load up the last known entries from the file we just wrote (I will call it .gnus-expire)

    (let ((file gnus-expire-log-file))
      (if (file-exists-p file)
          (let ((entries (with-current-buffer
                             (find-file-noselect file)
                           (split-string
                            (save-restriction
                              (widen)
                              (buffer-substring-no-properties
                                                       (point-min)
                               (point-max)))
                            "\n" t))))
            (mapcar (lambda (tuple)
                      (let ((groupname (car (split-string tuple)))
                            (value (cadr (split-string tuple))))
                        ;(message "%s %s" groupname value)
                        (setq gnus-expiry-table
                              (acons groupname value gnus-expiry-table))))
                    entries)))))
    

And now, we have a populated associative list to perform step 2 above.

Here's the complete source

;;; expiry-hack.el - Don't expire often!
;; Author: Bandan Das <bsd@makefile.in>
;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; This assoc list will store the expiry times
;; associated with groups. If there's a .gnus-expiry-file,
;; this gets filled up with the last time expiry was
;; performed
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(defvar gnus-expiry-table nil
  "This is the run-time assoc list")
(defvar gnus-expire-log-file "~/.gnus-expire"
  "The file where the expire times are written")
(defvar gnus-group-ignore-list nil
  "Don't apply our expire logic for anything here")
(defvar gnus-group-expiry-time-limit 86400
  "When to run expiry")
;; remove some hooks first

(defun gnus-read-last-expiry-times()
  "Read the expiry times from the file, populates gnus-expiry-table"
  (let ((file gnus-expire-log-file))
    (if (file-exists-p file)
        (let ((entries (with-current-buffer
                           (find-file-noselect file)
                         (split-string
                          (save-restriction
                            (widen)
                            (buffer-substring-no-properties
                             (point-min)
                             (point-max)))
                          "\n" t))))
          (mapcar (lambda (tuple)
                    (let ((groupname (car (split-string tuple)))
                          (value (cadr (split-string tuple))))
                      ;(message "%s %s" groupname value)
                      (setq gnus-expiry-table
                            (acons groupname value gnus-expiry-table))))
                  entries)))))

(defun write-string-to-file (string file)
  "Simple Function to write a string of the form - \"group value\""
   (if string
       (with-temp-buffer
         (insert string)
         (when (file-writable-p file)
           (write-region (point-min)
                         (point-max)
                         file)))))

(defun check-entry-in-ignore-list (entry)
  "Just check an entry against the ignore list"
  (setq match nil)
  (setq count 0)
  (setq found nil)
  (while
      (and (not found) (< count (length gnus-group-ignore-list)))
    (when (string-match (elt gnus-group-ignore-list count) entry)
      (setq found t))
    (setq count (+ count  1)))
  found)

(defun gnus-group-custom-expire-single-group()
  "This is called when you exit a group, functionally
   similar to the function below to expire all groups"
  (let ((entry gnus-newsgroup-name))
    (if (check-entry-in-ignore-list entry)
        ;;use the default behavior
        (gnus-summary-expire-articles)
      (progn
        (let ((last-expiry-time (cdr (assoc entry gnus-expiry-table))))
          (if (and last-expiry-time
                   ( > (- (float-time) (string-to-number last-expiry-time))
                       gnus-group-expiry-time-limit))
              (progn
                (let ((time (number-to-string(float-time))))
                  (setq gnus-expiry-table
                        (acons entry time gnus-expiry-table)))
                (gnus-summary-expire-articles))))))))

;gnus-group-marked
(defun gnus-group-custom-expire-all-groups ()
  "This function gets the list of all groups,
  and creates a new list out of it based on -
   - if there is an ignore list
   - If we have matching entry in gnus-expiry-table
  and we need to process expiry"
  (interactive)
  (setq to-write nil)
  (save-excursion
    (gnus-message 5 "Running custom expire...")
    (let ((gnus-group-tmp-list (mapcar (lambda (info)
                                       (gnus-info-group info))
                                     (cdr gnus-newsrc-alist)))
          (to-write nil))
      (dolist (entry gnus-group-tmp-list)
        (if (check-entry-in-ignore-list entry)
            (add-to-list 'gnus-group-marked entry) ;;default behavior - add to expiry list
          (progn
            (let ((last-expiry-time (cdr (assoc entry gnus-expiry-table))))
              (if (and last-expiry-time
                       ( > (- (float-time) (string-to-number last-expiry-time))
                         gnus-group-expiry-time-limit))
                  (add-to-list 'gnus-group-marked entry))
              ;but write it so it gets accounted for the next expiry!
              (setq to-write (concat to-write
                                     (concat
                                      entry " "
                                      (number-to-string (float-time))
                                      "\n")))))))
      (write-string-to-file to-write gnus-expire-log-file))
    (gnus-group-expire-articles nil))
  (gnus-group-position-point)
  (gnus-message 5 "Expiring...done"))


(provide 'expiry-hack)

Comments

blog comments powered by Disqus