Linux – gcc running out of memory in Ubuntu 12.10

gcclinuxmemoryUbuntuvirtualbox

I'm trying to compile my C project in Ubuntu (32bit) running in VirtualBox. Compiling the code in question requires quite a bit of memory, at least 3 gig. So I gave the VM 2 gig. Along with 2 gig of swap space available, it should be enough. For some reason though gcc is failing with an out -of-memory error after it has allocated 900 meg or so. Bumping the amount of memory to 2.7 gig (the maximum allowed by VirtualBox) didn't help. There seems to be a limit on how much memory a process can use. But when I run ulimit, it shows "unlimited".

UPDATE – Here's the make log:

libtool: compile: cc -msse2 -I. -I/home/cleong/qb -DPHP_ATOM_INC
-I/home/cleong/qb/include -I/home/cleong/qb/main -I/home/cleong/qb -I/usr/include/php5 -I/usr/include/php5/main -I/usr/include/php5/TSRM -I/usr/include/php5/Zend -I/usr/include/php5/ext -I/usr/include/php5/ext/date/lib -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DHAVE_CONFIG_H -g -O2 -c /home/cleong/qb/qb_interpreter_gcc.c -fPIC -DPIC -o
.libs/qb_interpreter_gcc.o

cc1: out of memory allocating 408 bytes after a total of 924852224 bytes

make: *** [qb_interpreter_gcc.lo] Error 1

Output from /bin/time -v:

Command exited with non-zero status 2
    Command being timed: "make"
    User time (seconds): 62.09
    System time (seconds): 11.28
    Percent of CPU this job got: 64%
    Elapsed (wall clock) time (h:mm:ss or m:ss): 1:53.02
    Average shared text size (kbytes): 0
    Average unshared data size (kbytes): 0
    Average stack size (kbytes): 0
    Average total size (kbytes): 0
    Maximum resident set size (kbytes): 1848592
    Average resident set size (kbytes): 0
    Major (requiring I/O) page faults: 9433
    Minor (reclaiming a frame) page faults: 1391779
    Voluntary context switches: 5642
    Involuntary context switches: 6069
    Swaps: 0
    File system inputs: 630360
    File system outputs: 1376
    Socket messages sent: 0
    Socket messages received: 0
    Signals delivered: 0
    Page size (bytes): 4096
    Exit status: 2

The version of gcc is 4.7.2.

The source code is here:

https://github.com/chung-leong/qb

It's a PHP extension. The build process uses phpize.

UPDATE – I'm encountering the same problem in 32-bit Mageia 3, which also includes gcc 4.7.2. The failure happens at around 2.7 gig instead. In a 64-bit environment, it doesn't happen.

If I install gcc 4.6.3 and build using that, it works.

Best Answer

The fact that with optimization level of -O1 or lower, 64-bit Ubuntu manages to compile the project, means that the problem lies with the optimization phase of gcc.

I would guess that some source-file in your project happens upon a bug or inefficiency in the gcc optimizer introduced in version 4.7.2. which causes the use of an unreasonable amount of memory.

So what I would suggest is :

  1. A 32-bit version of gcc can only access 2-3 GB of memory (whatever size is the swap).
  2. Enlarge the swap space, but keep on reading if this doesn't help. In any case, for a 32-bit gcc there will be a limit to the amount of memory that it can use.
  3. By compiling the source files one-by-one, find the one that is responsible for the problem. If this is all of them, then the problem is with some include file(s).
  4. If the problem is not with an include file, further break up the problematic source file into several parts, until you either stop getting the error or you locate the very function that causes it.
  5. As a temporary workaround, modify the make file so as to compile that function with -O1. The problem is in this case with gcc itself, and you can submit that function in a bug report (together with all your include files).
  6. You can of course also opt to compile the entire project with -O1 which should be sufficient for a PHP extension, or stay with gcc 4.6.3 for the moment.
  7. Keep on trying out new updates to gcc, since the bug could be fixed (or the inefficiency removed) unrelatedly to your bug report.
Related Question