[Users] ZCS 8.8.5 locking and reverse proxy issues

[Malte S. Stretz]

Hi again,

short update below.


On 03.01.2018 21:46, Malte S. Stretz wrote:
> I haven't yet upgraded a single production system to 8.8.5 (waiting 
> for the 8.8.6 fix for the virtual hostname issue for one thing) but 
> have an eye on the forums to judge by the issues reported if upgrading 
> to 8.8 is safe yet. Recently there were a few posts where people have 
> problems with ZCS stopping to respond either (or both) due to reverse 
> proxy issues or mailbox locking problems. Here are the threads:
>
> * https://forums.zimbra.org/viewtopic.php?f=15&t=63343 "Upgrade and 
> New Install 8.8.5 Web Stops Responding"
> * https://forums.zimbra.org/viewtopic.php?f=13&t=63353 "8.8.5 NGINX 
> recv() failed (111: Connection refused)"

An update on the forums hints that this is the same core issue, the 
internal mailbox locking.

> * https://forums.zimbra.org/viewtopic.php?f=13&t=63339 "Since 
> Upgrading To Zimbra 8.8.5, Have Been Getting Numerous Lock Errors"

There also appeared a new thread with the same core issue:

* https://forums.zimbra.org/viewtopic.php?t=63354&p=280724 "IMAP 
locking, LDAP TLS/OpenDKIM issues after 8.7 to 8.8 upgrade"

> Has anybody else seen similar issues with 8.8.5?
>
> I've seen similar locking issues already on 8.7.x, especially if macOS 
> Mail and Calendar were involved. But up till now they only brought 
> down a single mailbox, not a whole server.

 From the reports it looks like the issue is indeed mostly caused by 
IMAP traffic from Apple devices which would match what I experienced 
already on 8.7. These locking issues can be caused by performance issues 
(especially slow IO) and many concurrent connections to a mailbox but 
since more and more people report this for 8.8 I think this is a 
regression. Or some other part slowed down and this is just a side effect.


Some related links:
* 
https://github.com/Zimbra/zm-mailbox/blob/develop/store/src/java/com/zimbra/cs/mailbox/MailboxLock.java
* https://bugzilla.zimbra.com/show_bug.cgi?id=84490

Cheers,
Malte