Zlib status (was Re: Any zlib updates?)

Quentin Stafford-Fraser quentin "at" orl.co.uk
Thu, 10 Dec 1998 01:23:16 +0000

Scott Dudley wrote:

> Quentin,
> I'm certain you're growing tired of being asked but... are there any
> updates on the zlib integration?

No, it's not that - it's more that I don't want to disappoint people with
the answer... :-(  But this is Open Source, so I'd better be open!

OK, here's the basic story.  The easiest way to try out zlib and get some
statistics with it was to write it as a separate pixel encoding alongside
hextile, CoRRE etc.  It just compressed the raw pixels, because I felt
that encoding and then compressing was likely to be a waste of time.  We
did some experiments and found with that screens which were already
efficiently encoded by hextile, eg.a boring twm/motif desktop with some
xterms and emacs windows on it, there was little, if any, gain from
encoding them with zlib, and on a fast network zlib introduced latency
which was quite noticable.

On more interesting screens, however, such as those with large images or
dithered backgrounds, zlib gave quite considerable improvements in
compression, sometimes a factor of 3 or 4 over hextile. In extreme cases,
such as the rather artificial one of the default X crosshatched background
with nothing else on the screen, the compression ratios can be much
larger.  So we agree that zlib compression is certainly worth doing,
especially if you like using pretty screens over slow links!  (We don't
think it would generally improve things over local ethernet).

So I started adding this to all our clients and servers. However, I knew
this was a bit of a hack.  The compression, like authentication and
encryption, should really be a wrapper around the VNC stream, and not just
another encoding.  This was reinforced by a message from someone on the
list, (I forget at present from whom, sorry), saying that they had done
some experiments indicating that it *was* worth using one of the current
encodings followed by compression, rather than just compressing raw
pixels.  I imagine this depends a lot on the contents of the desktop, but
I'm willing to believe it.  And after all, if the zlib is a separate stage
from the encoding, you can play about with the encodings and experiment,
using 'raw' if it does turn out to be the best.

There were a lot of enquiries about the WinCE viewer, so I quickly stuck
that back together, released it, and then went away to the States for 3

When I came back I started thinking more about substantial changes to the
protocol, as have been discussed here before, where compression,
encryption, authentication, clipboard encoding etc are dealt with in a
more modular way, both in the protocol and in the code, allowing people to
modify one part without changing the whole protocol, and so forth.  It was
clear this would be a much more substantial change, and that if we were
going to do this, it was pointless spending much time creating protocol
3.4.  We had better move onto 4.0, which is the Right Way to Do It.

I may decide to do protocol 3.4 anyway, even though it will delay 4.0,
because 4.0 will probably not be backwards compatible with older versions
in any way, and there are hundreds of thousands of people out there using
software based on 3.3.  But, while the number of people working on VNC
here varies over time, just at the moment it's about 0.8 even when I am
home :-)

Apologies for the essay, but now you know the current status and why
progress has been slow. At present, if compression is important, I suggest
you use Dave DeBarr's version, or wrap the stream in SSH as mentioned on
the web.  But we haven't forgotten zlib!

Comments & suggestions welcomed.

Quentin Stafford-Fraser
Dr Quentin Stafford-Fraser     www.orl.co.uk/~qsf
ORL  -  The Olivetti & Oracle Research Laboratory

The VNC mailing list     -   see http://www.orl.co.uk/vnc/intouch.html