This issue has been fixed by Zimbra, the email application we use. Can CiviCRM be adjusted to deal with this issue or is it purely on the IMAP server side? Would using POP3 have a different effect?
Their solution is with a caveat that has a drastic effect at the enterprise level:
The issue stems from the bug reported at -
https://bugzilla.zimbra.com/show_bug.cgi?id=73266
Pasting an internal comment from the bug, which help explains the
issue --- This goes back to the fix for bug 63644. There are two
limits at play zimbraImapMaxRequestSize and zimbraMtaMaxMessageSize.
The latter is used for all literals, regardless of command. Due to the
way literals are processed there isn't a really clean way to use
zimbraImapMaxRequestSize for some literals and allow larger literals
commands such as APPEND where we really want to allow huge literals;
at least without a big rewrite that is too much for 8.0 at this stage.
Staff, for the purposes of the test can you also set
zimbraMtaMaxMessageSize to a really small value (say 5). I think the
main purpose of those two limits is to protect against buffer
overflow; so as long as we have some limitations for all commands that
should be safe enough for now.
The downside of disabling NIO is that you loose out on the performance
improvement that NIO offer - http://wiki.zimbra.com/wiki/IMAP_NIO :
> NIO implementation helps reduce the mailboxd memory footprint with a
small CPU overhead. NIO eliminates the need of 1 (IMAP
connection)-to-1 (server thread) mapping and helps reduce the memory
footprint of the ZCS server. It is useful when there are many
concurrent IMAP users. The exact benefit depends on the number of IMAP
users/server and number of live connections/users.
> The saving is linear to the number of active IMAP connections that are
in a single mailbox server.
> Example for 10,000 concurrent IMAP accounts:
> With NIO: If there are 10,000 IMAP accounts, each of the IMAP accounts
has an IMAP client which maintains 3 to 4 connections. There are at
most 100 to 200 Java threads required in the mailboxd process to
handle requests from those clients. If one thread uses 256k memory, it
uses 25 to 50MB Java memory.
> Regarding the CPU and depending on the load, the overhead usage can
range from 0 to 20%. Since ZCS is not CPU bounded, the CPU increase
should not be a significant factor to impact the client side response
time.
*>
Without NIO: If there are 10,000 IMAP accounts, each of the IMAP
accounts has an IMAP client which maintains 3 to 4 connections. There
are 10,000*3 or 10,000*4 Java threads required in the mailboxd process
to handle requests from those clients. If one thread uses 256k memory,
it uses 7,500MB to 10,000MB of Java memory.*