Container causes core dump when using mmap/memset

Discussion in 'General Discussion' started by PeterSteele, Feb 5, 2014.

  1. PeterSteele

    PeterSteele Bit Poster

    Messages:
    3
    I'm converting a project from a KVM based environment running CentOS 6.3 to one that uses VZ, primarily to see if there are performance benefits. I've hit a problem with a mmap/setmem operation that works fine under my KVM VM environment but causes a core dump under my VPS. I've come up with the following test program to reproduce this error:

    Code:
    #include <stdio.h>
    #include <memory.h>
    #include <sys/mman.h>
    #include <math.h>
    
    void main() {
        printf("Starting...\n");
        int i;
        unsigned long nbytes = 1 * pow(1024, 3) / 2;
        for (i = 0; i < 20; i++) {
            char *mp;
            mp = (char *)mmap(0, nbytes, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANON, -1, 0);
            printf("i=%d, mp=%p\n", i, mp);
            memset(mp, 0, nbytes);
       }
    }
    
    This loop allocates a one-half gigabyte block on each iteration and then sets the memory block to all zeroes. The mmap call succeeds on each call. However, after about 2.5G of memory has been allocated with mmap, the memset call fails, causing a core dump to occur. The same code on a similarly configured KVM based VM runs until all memory has been exhausted and the process is killed by the out-of-memory killer.

    If I change the mmap call to a malloc call, the core dump does not occur. Instead, the program runs until memory is exhausted and the process is killed by the out-of-memory killer. In this case, the behavior of the program is identical under the container and the VM.

    So, my question is why does the program crash with a core dump when a memset is used on memory allocated with mmap?
     
    Last edited: Feb 5, 2014
  2. PeterSteele

    PeterSteele Bit Poster

    Messages:
    3
    I should add that I am using a custom CentOS 6 template. I just tried running my test app in a container based on the stock centos-6-x86_64 template and this behaves worse--the code cores on the first loop of the test app.

    So, the question I have to ask is mmap supported inside a container?
     
  3. Vasily Averin

    Vasily Averin Plesk Developers

    Messages:
    16
    Probably you have used 32-bit container?
    IIRC 32-bit userspace was unable to map more than 3 Gb.
     
  4. PeterSteele

    PeterSteele Bit Poster

    Messages:
    3
    I was using the 64-bit container, but it turned out I had an problem with the container's bean counters. I ended up setting everything to unlimited, but I believe the one that was causing me problems was PRIVVMPAGES. With this set to unlimited, I'm able to allocate shared memory up to the size of the container's total memory.
     

Share This Page