In short,
the "problem" arises from the fact that, IMAP4 encodes folder names using a modified UTF-7 coding. offlineimap does not convert folder names to something readable before creating local repositories (e.g. in UTF-8). This, in turn, derives unreadable folder names like the ones shown in the screenshot of this question. Hence, it's neither Mutt's nor offlineimap's mishandling or misconfiguration per se.
The issue is discussed, in detail, and solved in the following blog-posts and git repository:
The solution
Essentially, a python script (provided below) that helps in deriving readable folder names, is fed into offlineimap's configuration file (that is offlineimaprc
, as explained in OfflineIMAP Manual). In addition, an instructive line of code for the proper folder name translation (using the function(s) defined in the python script), i.e.,
# Name translation from UTF7 to UTF8
nametrans = lambda foldername: foldername.decode('imap4-utf-7').encode('utf-8')
is added at offlineimap's configuration file under the section with the options for the remote repository.
Update, (April 2015)
Another rule is required for the reverse operation, see Folder filtering and Name translation. This instructions may be something like
# Name translation, reverse!
nametrans = lambda foldername: foldername.decode('utf-8').encode('imap4-utf-7')
In the mailboxes
file, the one created by offlineimap, Greek names appear correctly. This resolves the issue inside mutt and folder names appear as intented (in this case, Greek names).
Remaining issue?
However, the local repository folder names (directory names) are still in an unreadable state. I.e., the above shown Greek folder name (Υποτροφία), is actually, the directory &A6UDwAO,A8QDwQO,A8YDrwOx-
. Does this mean that, folder name conversions take place after, and not during, the syncing of folder names and messages? Or, is it required to remove these directories (from the local repository) and force another syncing via offlineimap (taking care not to let the respective mailbox folders be removed from the remote repository)?
Python script to deal with international mailbox names (IMAP, UTF-7):
# vim:fileencoding=utf-8
r"""
Imap folder names are encoded using a special version of utf-7 as defined in RFC
2060 section 5.1.3.
From: http://piao-tech.blogspot.com/2010/03/get-offlineimap-working-with-non-ascii.html
5.1.3. Mailbox International Naming Convention
By convention, international mailbox names are specified using a
modified version of the UTF-7 encoding described in [UTF-7]. The
purpose of these modifications is to correct the following problems
with UTF-7:
1) UTF-7 uses the "+" character for shifting; this conflicts with
the common use of "+" in mailbox names, in particular USENET
newsgroup names.
2) UTF-7's encoding is BASE64 which uses the "/" character; this
conflicts with the use of "/" as a popular hierarchy delimiter.
3) UTF-7 prohibits the unencoded usage of "\"; this conflicts with
the use of "\" as a popular hierarchy delimiter.
4) UTF-7 prohibits the unencoded usage of "~"; this conflicts with
the use of "~" in some servers as a home directory indicator.
5) UTF-7 permits multiple alternate forms to represent the same
string; in particular, printable US-ASCII chararacters can be
represented in encoded form.
In modified UTF-7, printable US-ASCII characters except for "&"
represent themselves; that is, characters with octet values 0x20-0x25
and 0x27-0x7e. The character "&" (0x26) is represented by the two-
octet sequence "&-".
All other characters (octet values 0x00-0x1f, 0x7f-0xff, and all
Unicode 16-bit octets) are represented in modified BASE64, with a
further modification from [UTF-7] that "," is used instead of "/".
Modified BASE64 MUST NOT be used to represent any printing US-ASCII
character which can represent itself.
"&" is used to shift to modified BASE64 and "-" to shift back to US-
ASCII. All names start in US-ASCII, and MUST end in US-ASCII (that
is, a name that ends with a Unicode 16-bit octet MUST end with a "-
").
For example, here is a mailbox name which mixes English, Japanese,
and Chinese text: ~peter/mail/&ZeVnLIqe-/&U,BTFw-
"""
import binascii
import codecs
# encoding
def modified_base64(s):
s = s.encode('utf-16be')
return binascii.b2a_base64(s).rstrip(b'\n=').replace(b'/', b',').decode('ascii')
def doB64(_in, r):
if _in:
r.append('&%s-' % modified_base64(''.join(_in)))
del _in[:]
def encoder(s):
r = []
_in = []
for c in s:
ordC = ord(c)
if 0x20 <= ordC <= 0x25 or 0x27 <= ordC <= 0x7e:
doB64(_in, r)
r.append(c)
elif c == '&':
doB64(_in, r)
r.append('&-')
else:
_in.append(c)
doB64(_in, r)
return (''.join(r).encode('ascii'), len(s))
# decoding
def modified_unbase64(s):
b = binascii.a2b_base64(s.replace(b',', b'/') + b'===')
return str(b, 'utf-16be')
def decoder(s):
r = []
decode = []
for c in s:
if c == b'&' and not decode:
decode.append(b'&')
elif c == b'-' and decode:
if len(decode) == 1:
r.append('&')
else:
r.append(modified_unbase64(b''.join(decode[1:])))
decode = []
elif decode:
decode.append(c)
else:
r.append(c.decode('ascii'))
if decode:
r.append(modified_unbase64(b''.join(decode[1:])))
bin_str = ''.join(r)
return (bin_str, len(s))
class StreamReader(codecs.StreamReader):
def decode(self, s, errors='strict'):
return decoder(s)
class StreamWriter(codecs.StreamWriter):
def decode(self, s, errors='strict'):
return encoder(s)
def imap4_utf_7(name):
if name == 'imap4-utf-7':
return (encoder, decoder, StreamReader, StreamWriter)
codecs.register(imap4_utf_7)
Best Answer
I finally found the resolution, so if someone got in the same issue, the answer is to add to the account1 and account2 files mapping for the sortcuts, that is from
muttrc
, the linesalso have to be at the end of sourced accout1 and account2 files. It seems that
mutt
fills in the value of the variables when reading the configuration for the first time rather than keeping the variable and evaluate the expression for shortcut each time the shortcut is run with the actual variable value.