Changing pixelformat on the fly

[Peter Åstrand]

I believe I've discovered a problem with the VNC protocol, related to 
changing the pixel format on the fly. Take a look at this chart:

FUR=FramebufferUpdateRequest
FU=FramebufferUpdate
SPF=SetPixelFormat

Client			Server
------------------------------
FUR sent
			FUR recieved
			FU1 sent
FU1 recieved
SPF sent
			(deferred) FU2 sent
			SPF recieved
FU2 recieved

As you can see, FU2 is sent *before* the server has recieved the SPF, but 
on the client, FU2 is recieved *after* the SPF has been sent. Bang! The 
client inteprets the FU according to the new pixel format, but the FU is 
in fact using the old pixel format. Then everything gets out of sync, and 
the client crashes. 

I've verified this problem with Xvnc from TightVNC 1.2.9, and vncviewer 
from RealVNC 4.0 (slightly modified). It's easy to trigger the crash by 
using a SSH tunnel over a slow connection, and using AutoSelect. When 
AutoSelect determines that the pixelformat should change, the client 
crashes. 

As I understand it, the same problem exists with the TightVNC 1.2.9
Windows viewer, if you change the pixelformat on the fly. 

This problem would go away if deferred updates were not used. In that
case, a FU could be interpreted as a ACK response for a FUR. As it is now,
several FUs can be sent in response to a single FUR. Is RealVNC using
deferred updates? Then the same problem should exist in a RealVNC-only
configuration.

How can this be solved? The only solutions I can think if now is:

1) Disconnect TCP connection, and make a new one. Not very nice. 

2) Add some kind of synchronization to the protocol. For example, we could 
add a pseudo rectangle, pretty much like rfbEncodingNewFBSize, that 
indicates that the new pixmap formats should be used now. 

Comments?

-- 
Peter Estrand		Chief Developer
Cendio			www.thinlinc.com
Teknikringen 3		www.cendio.se
583 30 Linkvping        Phone: +46-13-21 46 00