Adapt the memory allocation parameters in the Karaf start script at the following location: <RuntimeContainerPath>/bin/setenv.
you can set different values depending whether your JVM is 64-bit. 64-bit-JVMs need significantly more memory.
On Solaris, there is an additional flag JAVA_64BIT_SOLARIS, which is set in <RuntimeContainerPath>/bin/setenv. The default is JAVA_64BIT_SOLARIS=true. This flag is evaluated only for Solaris and determines whether the JVM operates in 64-bit or 32-bit mode. This is necessary, because in Solaris (only), a JVM capable of 64-bit starts normally in 32-bit mode. Set JAVA_64BIT_SOLARIS=false if you really want to choose 32-bit mode.
Note that in Redhat Linux Enterprise 5/64 Bit, the memory allocation is different depending on whether you use the container through the wrapper utility or not. If you have installed the service with the wrapper feature, you need to tweak the JVM parameters in the wrapper.conf file, see http://wrapper.tanukisoftware.com/doc/english/prop-java-maxmemory.html for details.