A new way to read Public-Inbox

Some time ago I canceled all my mailing list subscriptions and changed the way I consumed information contained therein that was not directly sent to me. This post tries to describe where I landed after that change.

Why change?

While trying to stay up to date with mailing lists, I received what seemed to me as an unmanageable amount of mail that just ended up unread and gathering dust in my inbox. When I finally did read them; they had already become irrelevant. This “new way” of reading lists is just me trying to avoid all the crud that I don’t read anyway.

Unsubscribe, Unsubscribe, Unsubscribe!!!

It is just too much to actually read; life is too short to start a 100 e-mail thread only to realize half way through that it is not interesting. Just let it all go and stop expecting to find the relevant bits by reading it all. This is specially relevant in high traffic mailing lists that have a high noise to information ratio.

But don’t stop reading

So how do you keep up to date? I’m currently experimenting with a few LEI queries that (I believe) give me a decent subset of what I need to be reading. The following are a list of the ones that I keep coming back to and a small explanations of the arguments and search strings. I use this as ad-hock queries that I run when I need them, instead of something that comes into my inbox regularly.

Query Base

lei q \
    -o ~/Mail/[lei-staging|lists] \
    -I https://lore.kernel.org/all \
    --dedup=mid \
    -t \
    --no-save \
    [-a] \
    "(rt:1.month.ago..)"

This is the part of the query that stays more or less the same. With a few changes depending on where the query results end up:

1. -o ~/Mail/[lei-staging|lists]: This is the directory where the query results will end up. I have setup two:

   • lei-staging : It servers to simply look at the current query. When I use this directory, I do not use the -a argument; this means that every time I execute a query the Mail Dir that is created replaces the previous one. I make sure not to track this directory within my mail reader (neomutt) nor my mail indexer (notmuch)

   • lists : This one servers to follow the interesting bits that I have found in my initial lei-staging query searches. I change the output dir to lists once I know that I’m intersted and I have a very specific query that returns just one thread (this usually means the subject string)

2. -I https://lore.kernel.org/all I always search all lore because as I have found that the search patterns that I usually use give a small enough result that I can assess without too much fuss.

3. --dedup=mid There is always a chance to have duplicates and I want LEI to de-duplicate them by looking at the mail identification (mid).

4. -t I’m interested in threads, single e-mails usually miss the relevant context that only the thread can provide.

5. --no-save By default LEI saves the query in the ‘-o’ directory. I do not need this as I’m not running lei up.

6. [-a] When I’m doing a lei-staging query, I do not use append

7. "rt:1.month.ago.." The lei-q command is designed in such a way that each individual search “TERM” are anded. My first search term is always a time range because I’m usually not interested in information that is very old. Here are some aspects that I consider when using the “rt” search term:

   • I run most of my lei queries weekly, so I use “rt:1.week.ago…” or “rt:2.week.ago…” if I see that I have not done it in a while. I don’t go over two weeks; my thought being that if that mail that I missed 3 weeks ago has not made it into my inbox, it was not that important to begin with. Remember, all this is for contextual information.

   • I sometimes run historical searches. For these I broaden my search range to something like “rt:1.year.ago…” and add on to that as I move forward. If I find that I don’t get lots of hits for a 1 year range, I try 5, 10 or simply remove the “rt” search term altogether.

   • Very seldom I just want to get a mail that was just sent. In these cases I use “rt:1.day.ago…”

Stalking People

lei q \
    -o ~/Mail/lei-staging \
    -I https://lore.kernel.org/all \
    --dedup=mid \
    -t \
    --no-save \
    "(rt:1.week.ago..)" \
    "f:torvalds"

There are several reasons in my case that I focus on mails sent from a certain person:

1. Certain people publish relevant (for what I’m doing) information to the lists. For example: I try to read mails from Linus when it gets close to the merge window to figure out when it is happening and if there are any special considerations.

2. When working on a patch set with other developers, I find that looking at their discussions adds to the my situational awareness.

Files that are interesting

lei q \
    -o ~/Mail/lei-staging \
    -I https://lore.kernel.org/all \
    --dedup=mid \
    -t \
    --no-save \
    "(rt:1.month.ago..)" \
    "dfn:proc_sysctl.c"

There are two reasons I search for discussions/patchsets based on files:

1. Getting to know if people are working on the same file that I’m touching and what they are doing can help me avoid adding merge conflicts. It also helps in knowing when to coordinate with other developers.

2. I want to know if I have missed anything relevant to what I maintain. These are the cases where someone sent something to the list, but forgot to send it to me.

Keep what is interesting updated

I put the mails that I find relevant/interesting in a “lists” directory. However, since I am not subscribed to the list, I do not get any subsequent mails to the thread. So how to get the mails that are sent after I have made my query?

I have a script that does that for me. It uses notmuch to find all the thread subjects inside my “lists” directory and then runs a lei query that updates each one. This script runs every time I sync may mail. And on the off chance that I don’t want to read any more updates from a thread I just remove it from the “lists” directory.

#!/usr/bin/env python3
import subprocess
import os.path
import textwrap

mail_path = "~/Mail/fastmail"
sync_dir_name="lists"
notmuch_cmd = "notmuch search --format=json folder:{} | jq -r '.[].subject'".format(sync_dir_name)

subjects = subprocess.run(notmuch_cmd, shell=True, capture_output=True).stdout.splitlines()

lei_output = os.path.join(mail_path, sync_dir_name)
lei_I = "https://lore.kernel.org/all"
lei_time_range = "\"(rt:1.year.ago..)\""
lei_base_cmd = "lei q -v -o {} -I {} --dedup=mid -t --no-save -a {}" \
  .format(lei_output, lei_I, lei_time_range)

print("Detected {} thread(s) in {}".format(len(subjects), lei_output))
for i in range(0,len(subjects)):
  s = subjects[i].decode('utf-8')
  if len(s)  1:
    print (textwrap.indent(lei_cmd_output, '    '))

new_cmd = ["notmuch", "new"]
print ("Executing {}".format(new_cmd))
new_cmd_out = str(subprocess.run(new_cmd, capture_output=True).stdout)
print ("{}".format(new_cmd_out))

Conclusion

Simplifying your inbox does not mean losing touch. LEI helps you stay informed without overwhelming your inbox. Try these queries, and share your own tips in the comments!