Compositing is provided by Compiz, which is a compositing window manager. What it means is explained below:
A compositing window manager is a
component of a computer's graphical
user interface that draws windows and
their borders. It also controls how
they display and interact with each
other, and with the rest of the
desktop environment. The main
difference between a compositing
window manager and other window
managers is that instead of outputting
to a common screen, programs each
output first to a separate and
independent buffer, or temporary
location inside the computer, where
they can be manipulated before they
are shown.[1][2]
The window manager then processes and
combines, or composites, output from
these separate buffers onto a common
desktop. The result is that the
programs now behave as independent 2D
or 3D objects.[1] Compositing allows
for advanced visual effects, such as
transparency, fading, scaling,
duplicating, bending and contorting,
shuffling, and redirecting
applications. The addition of a
virtual third dimension allows for
features such as realistic shadows
beneath windows, the appearance of
distance and depth, live thumbnail
versions of windows, and complex
animations, to name just a few.[3][4]
Because the programs draw to the
off-screen buffer, all graphics are
naturally double buffered and thus do
not flicker as they are updated.
The most commonly-used compositing
window managers include the Desktop
Window Manager in Microsoft Windows,
the Quartz Compositor in Mac OS X, and
Compiz, Metacity and KWin for Linux,
FreeBSD and OpenSolaris systems.
http://en.wikipedia.org/wiki/Compositing_window_manager
If you see a black border around docky, then compositing is not properly enabled. The warning you are getting is most likely a result of Docky starting before compositing kicks in.
Best Answer
enabling docky effects in xfce
By default, Docky should work with the built-in XFCE/Xubuntu compositor.
In 12.04 the compositor can be checked (if it is not already) via the window-manager tweaks
The image above shows Docky does not have the 3D effects, but by checking the options you'll have the 3D effects as per the next image:
If, for whatever reason the inbuilt compositor does not work for you, you'll need another compositor to help out Docky.
alternative method for both XFCE/LXDE
A good lightweight compositor is
xcompmgrxcompmgr
As the image above shows, the inbuilt compositor is not running.
xcompmgris running, which gives Docky the 3D effects you require.Use session and startup to run
xcompmgrwhen you login.