Differences between revisions 1 and 45 (spanning 44 versions)
Revision 1 as of 2016-06-14 06:32:08
Size: 32
Editor: alders
Comment:
Revision 45 as of 2016-09-14 09:00:20
Size: 8781
Editor: alders
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
Describe Email/Filtering here. #acl TrustedGroup:read,write,admin All:

= Email filtering =

Email filtering can be done through [[http://sieve.info/|Sieve]]. With Sieve, the filter rules are stored and applied on the mailserver. This is an advantage over client-side filtering, where emails are only filtered if you read them with a specific email client.

You have several options to manage your filters. You can use

 * Our webmailer https://email.ee.ethz.ch/
 * Mozilla Thunderbird with the Sieve plugin
 * A command line tool

We recommend to use our webmailer in the first place. Especially vacation messages can be defined in a very comfortable way through the webmailer. See [[Email/Vacation]] for further details. However, the Sieve interface of the webmailer only allows to define rather simple filter rules. For advanced rules, you have to use the Thunderbird plugin or the command line tool.

The following sections will explain how you get started to manage your email filters with these tools. Choose the one that fits you best. Be careful when editing filters from different clients, e.g. from the webmailer and Thunderbird. The clients may not work well together. The best approach is to always edit your filters with the same client.


== Manage email filtering rules with the webmailer ==

 1. Login on https://email.ee.ethz.ch/.
 1. Choose '''Settings → Filters'''.
 1. In the '''Filter sets''' list, you will see the entry '''managesieve'''. You may create other filter sets, but please note that only one filter set can be active at a time. We thus recommend that you select and use the '''managesieve''' filter set through the webmailer.
 1. In the '''Filters''' column you may now create new filters with the '''+''' Symbol at the bottom of the column. Or you can select existing filters and adapt them or enable/disable or delete them through the sprockets symbol. The filter form is self explanatory. You can create or change your filters with drop-down lists and text fields. Please note: If you created a vacation message through '''Settings → Vacation''', this filter will also be listed here. But if you intend to change that vacation message, it is best to do that through '''Settings → Vacation'''.

<<BR>>
{{attachment:roundcube-sieve-filters.png}}
<<BR>>

In the first section ''For incoming mail'' you define which emails are affected by this rule. With the '''+''' Symbol on the right you can add additional conditions. And on the top you define how these conditions are concatenated:

|| '''Option''' || '''Description''' ||
|| Match all of the following rules || All conditions have to be fulfilled before the action below is applied ||
|| Match any of the following rules || At least one condition has to be fulfilled before the action below is applied ||
|| All messages || The action is applied to any email you receive ||

Here are some more detailed explanations of expressions you might come across when defining new filter rules:

|| '''Comparator''' || '''Description''' ||
|| default || case insensitive comparison ||
|| strict (octet) || case sensitive comparison ||
|| case insensitive (ascii-casemap) || case insensitive comparison ||
|| numeric (ascii-numeric) || comparison of numbers ||

The '''header''' field specifies, against which header field the comparison should be applied. This allows you to e.g. compare a "Received" field for its date.

If there are multiple header fields of the type you specified, you can select the one that should be used with the '''index''' value.

|| '''Modifier''' || '''Description''' ||
|| none || Test against all header fields ||
|| address || Test against all header fields that contain addresses, i.e. From, To, Cc, Bcc, Sender, Resent-From, Resent-To, ... ||
|| envelope || Test against email envelope (don't use this one unless you are familiar with email envelopes) ||

If a '''address''' or '''envelope''' modifier is in effect, you may choose, against which part of the email address the comparison should be done:

|| '''Type''' || '''Description''' ||
|| all || Compare against the whole email address '''`user@domain.org`''' ||
|| domain || Only compare against the domain part `user@`'''`domain.org`''' ||
|| local part || Only compare against the local part '''`user`'''`@domain.org` ||
|| user || Compare against the user part of the local part '''`user`'''`+organisation@domain.org` ||
|| detail || Compare against the detail part of the local part `user+`'''`organisation`'''`@domain.org` ||

When one of your rules matches against the '''body''' of an email message, you can choose, how the email body should be transformed prior to the comparison:

|| '''Modifier''' || '''Description''' ||
|| text || Compare against all text elemenst of an email body (i.e. plain text and html elements). Same as ''content type &rarr; text''. ||
|| undecoded (raw) || Compare against the unmodified email body. Includes MIME part information. Don't use it unless you know what you are doing. ||
|| content type || Compare against all MIME parts of the message with the given content type. You would most often use content types like '''`text`''', '''`text/plain`''' or '''`text/html`'''. ||


== Manage email filtering rules with Mozilla Thunderbird ==

=== Setup ===

 1. First of all you need to install the '''Sieve''' plugin. Unfortunately, the version in the official Thunderbird plugin archive is too old (version 0.2.2). So you need to download the plugin directly from the [[https://github.com/thsmi/sieve/tree/master/nightly|Development Builds]] website. Choose the youngest release. At the moment of this writing this is '''sieve-0.2.3h.xpi'''. Once downloaded, you can install it in Thunderbird through '''Add-ons Manager &rarr; Install Add-on From File...'''

 1. Enable Sieve filtering by right-clicking on you D-ITET mailfolder and choosing '''Settings''' (see screenshot below).

 1. Under the '''Sieve Settings''' menu entry, enable Sieve with '''Yes, manage Sieve scripts for this account''' and then select '''Change Settings...'''
 <<BR>>
 Please make sure that you set at least the following options:
 . In the '''Server''' tab:
  . Server Name &rarr; IMAP Server: '''imap.ee.ethz.ch'''
  . Server Port &rarr; '''4190 (Default)'''
 . In the '''Security''' tab:
  . Connection Security &rarr; '''Force TLS'''
  . Authentication &rarr; '''IMAP Username and Password'''

 1. Select '''OK''' to save the settings and to enable Sieve management from Thunderbird.

=== Edit Sieve Filters ===

 1. In the options menu, select '''Message Filters &rarr; Sieve Message Filters...'''
 <<BR>>
 There is another interface to edit Sieve filters, available in the Options menu through ''Message Filters &rarr; Message Filters &rarr; Server side filters''. But we recommend the ''Sieve Message Filters'' mentioned before.

 1. The interface is self-explanatory. Once you start to create a new Sieve filter, the right hand side shows help for available Sieve commands. But you will have to write your filters all by yourself without the help of dropdown lists as in the Roundcube webmailer.
 <<BR>>
 Here is a Sieve tutorial to get you started: https://tty1.net/blog/2011/sieve-tutorial_en.html

== Sieve filtering examples ==

=== Catchall filter ===

A catchall domain is an email domain where emails to e.g. team@somename.ethz.ch are translated on our mailserver to somename+team, which in turn can then be redirected in the ''somename'' account through ''.forward+team'' files (if you maintain such a catchall domain, you will probably know, what we are speaking about here).

In order to do the same in Sieve, you will have to create a redirection rule for every email address. For example:

{{{
if address :is ["to", "cc", "bcc"] "team@somename.ethz.ch" {
    redirect "user1@ee.ethz.ch";
    redirect "user2@ee.ethz.ch";
    redirect "user3@ee.ethz.ch";
}
}}}


=== Deduplicate filter ===

With procmail it was possible to delete duplicate emails (i.e. if you get exactly the same email through different ways) with the following rule in ''.procmailrc'':

{{{
:0 Wh: $HOME/.procmail_msgid_lock
| formail -D 8192 $HOME/.procmail_msgid_cache
}}}

With Sieve you can get the same result with the following rule:

{{{
require ["duplicate"];

if duplicate {
    discard;
}
}}}


=== Header manipulations ===

Email headers can be edited through the '''editheader''' plugin. The following example replaces the subject for emails sent to {{{team@somename.ethz.ch}}} with "TEST".

{{{
require ["editheader"];

if address :is ["to", "bcc", "cc"] "team@somename.ethz.ch" {
    deleteheader :index 1 "Subject";
    addheader :last "Subject" "TEST";
}
}}}

Email filtering

Email filtering can be done through Sieve. With Sieve, the filter rules are stored and applied on the mailserver. This is an advantage over client-side filtering, where emails are only filtered if you read them with a specific email client.

You have several options to manage your filters. You can use

We recommend to use our webmailer in the first place. Especially vacation messages can be defined in a very comfortable way through the webmailer. See Email/Vacation for further details. However, the Sieve interface of the webmailer only allows to define rather simple filter rules. For advanced rules, you have to use the Thunderbird plugin or the command line tool.

The following sections will explain how you get started to manage your email filters with these tools. Choose the one that fits you best. Be careful when editing filters from different clients, e.g. from the webmailer and Thunderbird. The clients may not work well together. The best approach is to always edit your filters with the same client.

Manage email filtering rules with the webmailer

  1. Login on https://email.ee.ethz.ch/.

  2. Choose Settings → Filters.

  3. In the Filter sets list, you will see the entry managesieve. You may create other filter sets, but please note that only one filter set can be active at a time. We thus recommend that you select and use the managesieve filter set through the webmailer.

  4. In the Filters column you may now create new filters with the + Symbol at the bottom of the column. Or you can select existing filters and adapt them or enable/disable or delete them through the sprockets symbol. The filter form is self explanatory. You can create or change your filters with drop-down lists and text fields. Please note: If you created a vacation message through Settings → Vacation, this filter will also be listed here. But if you intend to change that vacation message, it is best to do that through Settings → Vacation.


roundcube-sieve-filters.png

In the first section For incoming mail you define which emails are affected by this rule. With the + Symbol on the right you can add additional conditions. And on the top you define how these conditions are concatenated:

Option

Description

Match all of the following rules

All conditions have to be fulfilled before the action below is applied

Match any of the following rules

At least one condition has to be fulfilled before the action below is applied

All messages

The action is applied to any email you receive

Here are some more detailed explanations of expressions you might come across when defining new filter rules:

Comparator

Description

default

case insensitive comparison

strict (octet)

case sensitive comparison

case insensitive (ascii-casemap)

case insensitive comparison

numeric (ascii-numeric)

comparison of numbers

The header field specifies, against which header field the comparison should be applied. This allows you to e.g. compare a "Received" field for its date.

If there are multiple header fields of the type you specified, you can select the one that should be used with the index value.

Modifier

Description

none

Test against all header fields

address

Test against all header fields that contain addresses, i.e. From, To, Cc, Bcc, Sender, Resent-From, Resent-To, ...

envelope

Test against email envelope (don't use this one unless you are familiar with email envelopes)

If a address or envelope modifier is in effect, you may choose, against which part of the email address the comparison should be done:

Type

Description

all

Compare against the whole email address user@domain.org

domain

Only compare against the domain part user@domain.org

local part

Only compare against the local part user@domain.org

user

Compare against the user part of the local part user+organisation@domain.org

detail

Compare against the detail part of the local part user+organisation@domain.org

When one of your rules matches against the body of an email message, you can choose, how the email body should be transformed prior to the comparison:

Modifier

Description

text

Compare against all text elemenst of an email body (i.e. plain text and html elements). Same as content type → text.

undecoded (raw)

Compare against the unmodified email body. Includes MIME part information. Don't use it unless you know what you are doing.

content type

Compare against all MIME parts of the message with the given content type. You would most often use content types like text, text/plain or text/html.

Manage email filtering rules with Mozilla Thunderbird

Setup

  1. First of all you need to install the Sieve plugin. Unfortunately, the version in the official Thunderbird plugin archive is too old (version 0.2.2). So you need to download the plugin directly from the Development Builds website. Choose the youngest release. At the moment of this writing this is sieve-0.2.3h.xpi. Once downloaded, you can install it in Thunderbird through Add-ons Manager → Install Add-on From File...

  2. Enable Sieve filtering by right-clicking on you D-ITET mailfolder and choosing Settings (see screenshot below).

  3. Under the Sieve Settings menu entry, enable Sieve with Yes, manage Sieve scripts for this account and then select Change Settings...
    Please make sure that you set at least the following options:

  4. In the Server tab:

    • Server Name → IMAP Server: imap.ee.ethz.ch

    • Server Port → 4190 (Default)

  5. In the Security tab:

    • Connection Security → Force TLS

    • Authentication → IMAP Username and Password

  6. Select OK to save the settings and to enable Sieve management from Thunderbird.

Edit Sieve Filters

  1. In the options menu, select Message Filters → Sieve Message Filters...
    There is another interface to edit Sieve filters, available in the Options menu through Message Filters → Message Filters → Server side filters. But we recommend the Sieve Message Filters mentioned before.

  2. The interface is self-explanatory. Once you start to create a new Sieve filter, the right hand side shows help for available Sieve commands. But you will have to write your filters all by yourself without the help of dropdown lists as in the Roundcube webmailer.


    Here is a Sieve tutorial to get you started: https://tty1.net/blog/2011/sieve-tutorial_en.html

Sieve filtering examples

Catchall filter

A catchall domain is an email domain where emails to e.g. team@somename.ethz.ch are translated on our mailserver to somename+team, which in turn can then be redirected in the somename account through .forward+team files (if you maintain such a catchall domain, you will probably know, what we are speaking about here).

In order to do the same in Sieve, you will have to create a redirection rule for every email address. For example:

if address :is ["to", "cc", "bcc"] "team@somename.ethz.ch" {
    redirect "user1@ee.ethz.ch";
    redirect "user2@ee.ethz.ch";
    redirect "user3@ee.ethz.ch";
}

Deduplicate filter

With procmail it was possible to delete duplicate emails (i.e. if you get exactly the same email through different ways) with the following rule in .procmailrc:

:0 Wh: $HOME/.procmail_msgid_lock
| formail -D 8192 $HOME/.procmail_msgid_cache

With Sieve you can get the same result with the following rule:

require ["duplicate"];

if duplicate {
    discard;
}

Header manipulations

Email headers can be edited through the editheader plugin. The following example replaces the subject for emails sent to team@somename.ethz.ch with "TEST".

require ["editheader"];

if address :is ["to", "bcc", "cc"] "team@somename.ethz.ch" {
    deleteheader :index 1 "Subject";
    addheader :last "Subject" "TEST";
}

Email/Filtering (last edited 2020-09-11 10:58:50 by maegger)