Yes you can!

Basic prerequisites

First, since the newest firmware upgrade you need version 1.6.3 or later of Argyll. If you're on 15.04/Vivid, you already have 1.6.3, but if you're on Trusty you should add this ppa: https://launchpad.net/~pmjdebruijn/+archive/argyll-testing (otherwise you might just get dispread: Error - icoms - usb_is_usb_portno: port number 1 out of range 1 - 0).

sudo add-apt-repository ppa:pmjdebruijn/argyll-testing
sudo apt-get update

Then, whether you're on Vivid or Trusty, do:

sudo apt-get install xcalib argyll colorhug-client gnome-color-manager

Register with colord

In order for the calibration program to notice your monitor, you need it registered with colord. This is actually something XFCE settings should do, but since it's not implemented yet we need a workaround. There are two possibilities, either installing gnome-settings-daemon, or xiccd. (Your icc profile can also be registered with colord through one of these daemons, which is handy since colour-managed programs can pick up on that and then "use the system profile" automatically.)

You can install gnome-settings-daemon with apt-get, while xiccd requires a PPA. I recommend xiccd since you can use that under XFCE with no conflicts, but gnome-settings-daemon is easier to install. Read on for how to set up each of these options.

If you choose gnome-settings-daemon, beware that you shouldn't have that running all the time since it can change your keyboard layout and who knows what else – if you use this option, you should probably log in as Guest before calibrating. When you're logged in as Guest, go into the power management settings and ensure the screen doesn't blank or the screensaver turn on. Then, open a terminal and do

/usr/lib/gnome-settings-daemon/gnome-settings-daemon

Leave that terminal running throughout the calibration.

If you choose xiccd, then you can keep that running in your XFCE profile to register your calibration profile with colord, and let colour-managed programs automatically know about it. But you'll have to install it from a PPA. If you're on 14.04 Trusty, just do

sudo add-apt-repository ppa:pmjdebruijn/xiccd-testing
sudo apt-get update

If you're on 15.04 Vivid, there's no PPA yet, but you can download xiccd's deb and libcolord1's deb and install them with

sudo dpkg -i libcolord1_1.0.9-0pmjdebruijn1~trusty_amd64.deb 
sudo dpkg -i xiccd_0.2.2-0pmjdebruijn2~trusty_amd64.deb

Then create the file /etc/xdg/autostart/xiccd.desktop with these contents:

[Desktop Entry]
Encoding=UTF-8
Name=xiccd
GenericName=X11 ICC Daemon
Comment=Applies color management profiles to your session
Exec=xiccd
Terminal=false
Type=Application
Categories=
OnlyShowIn=XFCE;

and log out and in again.

Profiling

Now, whether you chose xiccd or gnome-settings-daemon, you should be able to do

colormgr get-devices

and see a list of devices. Look for your monitor in there. It should have a line with "Device ID". If this is e.g. "xrandr-Lenovo Group Limited", you can start calibration with the command

gcm-calibrate --device "xrandr-Lenovo Group Limited"

After calibration is done, you will have a file in ~/.local/share/icc/ called GCM-something.icc. Do

cp ~/.local/share/icc/GCM* /tmp/

and switch to your regular user to grab that file from /tmp and put it somewhere more permanent.

Applying the profile

Profiles have two parts: a vcgt/LUT part that does white point correction, appplied by X11 itself, and a gamma/hue/saturation part that has to be applied by individual colour-managed programs.

To apply the vcgt-part, run xcalib /path/to/your/GCM-something.icc to test the profile (and xcalib -c to remove it again). You can put that in your list of startup programs to get it applied on login.

To apply the gamma+matrix part, you should enable colour management in GIMP (just give GIMP the path to the same file) and Firefox, and any other colour-managed programs. If you have xiccd running, most such programs should pick up on the path to the .icc file automatically.

More info

If you want a more thorough guide which shows command line colour profiling on Linux and talks about how to set up your display before you start, I highly recommend https://encrypted.pcode.nl/blog/2013/11/24/display-color-profiling-on-linux/ (this also shows how to load the profile for all users on startup).

If you got the too red display after calibration problem, you can either create a more minimal profile (without white-point correction) or you can try using one of the alternative CCMX files (start colorhug-ccmx to select alternative CCMX'es, e.g. Community Average, before running gcm-calibrate).

In 15.04 Vivid you may get an error new_disprd() failed with "Unknown calibration display type selection" – for me it worked again when I selected a different choice in the screen type selection (I tried White LED and RGB LED, this is a Lenovo A740 which I just know is LED at least). Also, if your Colorhug isn't recognized, try plugging it out and in again (for me it was picked up when I first started colorhug-flash and then plugged it in).