Memory Allocation Parameters / user_beancounters in Virtuozzo 2.6.2

Discussion in 'General Discussion' started by icontact, Aug 29, 2005.

  1. icontact

    icontact Guest

    Memory Allocation - Confirmation?

    I was just wondering if Im reading this correctly.

    I ran cat /proc/user_beancounters from inside my VPS and I have the following results below...

    Code:
    root@srv1 [~]# cat /proc/user_beancounters
    Version: 2.5
           uid  resource           held    maxheld    barrier      limit    failcnt
         10039: kmemsize        5125208    5128321   40098656   44108521          0
                lockedpages           0          0        881        881          0
                privvmpages       77431      77666     750000     825000          0
                shmpages           9051       9051      33324      33324          0
                dummy                 0          0          0          0          0
                numproc              67         67        440        440          0
                physpages         44243      44371          0 2147483647          0
                vmguarpages           0          0     125000 2147483647          0
                oomguarpages      59239      59367     125000 2147483647          0
                numtcpsock           37         38        440        440          0
                numflock              3          3        704        704          0
                numpty                1          1         44         44          0
                numsiginfo            0          1       1024       1024          0
                tcpsndbuf         79920      88800    4212558    6014798          0
                tcprcvbuf          2220       4440    4212558    6014798          0
                othersockbuf      19552      91280    2106279    3908519          0
                dgramrcvbuf           0       2220    2106279    2106279          0
                numothersock         18         20        440        440          0
                dcachesize       406435     410022    8750726    9013248          0
                numfile            1080       1081       7040       7040          0
                dummy                 0          0          0          0          0
                dummy                 0          0          0          0          0
                dummy                 0          0          0          0          0
                numiptent            71         71        512        512          0

    What I want to know is, how much RAM I have been allocated here, because Im not entirely sure what Im suppose to be looking at. Any feedback from anyone would be great just to point what I should be looking for in determining my guaranteed amount of RAM.

    Thank you
     
  2. QuickLearner

    QuickLearner Kilo Poster

    Messages:
    18
    I hope someone from SWSoft answers this question as their answer would be most authentic in this case. I'm usually hearing different versions from people all around. Even I've been seriously looking for an answer to this same question for sometime now.
     
  3. fenster

    fenster Tera Poster

    Messages:
    429
    Sure we'll answer :)

    let's just look at UBC parameters.

    Code:
                kmemsize        5125208    5128321   40098656   44108521          0
    Kernel memory: used 5125208 bytes. Kernel memory keeps process information and cannot be swapped.

    Code:
                privvmpages       77431      77666     750000     825000          0
    Private virtual memory (mem + swap in fact): allocated (but probably not used) 77431 pages (each page is 4Kb)

    Code:
                physpages         44243      44371          0 2147483647          0
    Physical pages: really used 44243 pages of memory.

    The difference between 'allocated' and 'used' is quite simple. Here is a code which allocates memory but does not use it (privvmpages are increased, but not physpages):

    Code:
    #include <stdio.h>
    #include <stdlib.h>
    
    int main()
    {
            size_t len = 40L * 1024 * 1024;
            void *p = malloc(len);
            if (!p) {
                    perror("malloc");
                    exit(1);
            }
            free(p);
            printf("Success\n");
            return (0);
    }
    And here is a code which allocates and uses the memory:

    Code:
    #include <stdio.h>
    #include <stdlib.h>
    
    int main()
    {
            size_t len = 40L * 1024 * 1024;
            void *p = malloc(len);
            if (!p) {
                    perror("malloc");
                    exit(1);
            }
            memset(p, 0, len); /* note this line */
            free(p);
            printf("Success\n");
            return (0);
    }
    If anyone has any questions, feel free to ask here :)
     
  4. QuickLearner

    QuickLearner Kilo Poster

    Messages:
    18
    Thanks fenster for the prompt response.

    What parameters in the output posted by icontact, would tell us how much RAM is guaranteed for him and how much can he burst upto?
     
  5. fenster

    fenster Tera Poster

    Messages:
    429
    Code:
                vmguarpages           0          0     125000 2147483647          0
    The guarantee is vmguarpages barrier = 125000 pages. In any case, the VPS will be able to allocate that size of memory. Well... in almost any case, with one exception...

    Code:
                oomguarpages      59239      59367     125000 2147483647          0
    ...when the whole node is in out of memory situation. In that case, the guarantee is oomguarpages barrier (the same 125000 pages in this example). It means that no VPS process will be killed if memory usage by the VPS is less than 125000 * 4096 bytes. If it is more, some process can be killed, and failcounter of oomguarpages will be increased.

    Maximum size of memory which can be allocated is defined by privvmpages (see my previous post).
     
  6. QuickLearner

    QuickLearner Kilo Poster

    Messages:
    18
    With the help of your answer, we can say that the guaranteed RAM (vmguarpages) in the given example is 125000 * 4096 = 512000000 (488.28125 MB).

    Further, we may also calculate privvmpages (RAM+SWAP) as 750000 * 4096 = 3072000000 (2929.6875 MB).

    would I be wrong here to note that RAM and SWAP allocation ratio is about 1:5. It seems there is a wide gap betwen the two, am i right? I used to think the ratio of RAM and SWAP should usually be 1:2.

    What I also fail to understand is the fact that out of the two parameters (vmguarpages & oomguarpages), only oomguarpages seems to be used upto some extent. vmguarpages seems be completely unused at all. What does this situation indicate? Or am i missing something?

    Please correct me if I went wrong in any calculations above.
     
  7. fenster

    fenster Tera Poster

    Messages:
    429
    You are a bit wrong here. Let me explain it more carefully.

    vmguarpages barrier is a guarantee: in any case (except out-of-memory aka OOM) you will be able to allocate 125000 pages. Well, probably you will be able to allocate more, up to 750000 pages (and if you are cool high priority process, you will be able to allocate 825000 pages, but definitely not more). We don't say anything about RAM or SWAP here. These pages can be swapped by HWnode kernel if needed, and can live in RAM if possible. Frankly, you can't say "I want this VPS to have X MB RAM and Y MB SWAP". There is no such parameter.

    vmguarpages does not account anything, so its current usage always equals to zero.

    oomguarpages parameter's 'current usage' field accounts total amount of RAM+SWAP used.

    physpages parameter accounts total amount of RAM used by the VPS (the memory shared between VPSes is divided and accounted in physpages for all VPSes which use it in equal fractions). It cannot be limited, it just shows the current situation.

    Let's do some math here.

    1. Normal situation:

    guarantee: vmguarpages barrier = 125000 pages = 488 MB.

    current usage: physpages current value = 44243 physical (RAM) pages = 172 MB.

    current usage (RAM + SWAP): oomguarpages current value = 59239 pages = 231 MB.

    may allocate (but no garantees if allocation will be successful) up to privvmpages barrier = 750000 pages = 2929 MB, and system processes -- up to privvmpages limit = 825000 pages = 3222 MB.

    2. OOM situation.

    If your oomguarpages current value + sockets buffers current value + kmemsize current value is more than oomguarpages barrier, you are the bad guy here and we'll kill your process.

    If something is unclear, please ask. We need some good thread about UBC parameters on this forum, indeed :)
     
  8. QuickLearner

    QuickLearner Kilo Poster

    Messages:
    18
    Thanks indeed for the detailed answer.
    It has really cleared a lot of doubts. I'm surely going to come back here later and ask more. But I would love to see others contributing here too and/or by asking questions.

    Can I request you to illustrate this the maths way? it really gets in the quicker with some maths.
     
  9. fenster

    fenster Tera Poster

    Messages:
    429
    OK, here is the maths. OOM sutiation on the node means that there is no free memory (or sometimes that there is no free low memory).

    In that case, the system will count the sum of:

    oomguarpages current value (MEM+SWAP actual usage) = 59239 pages = 242642944 bytes

    socket buffers current value: 79920 + 2220 + 19552 + 0 bytes

    kmemsize current value: 5125208 bytes

    the sum is 247869844 bytes. If this sum is more than oomguarpages barrier = 125000 pages = 512000000 bytes (it is not), some of the processes on the VPS can be killed in case of OOM situation.
     
  10. QuickLearner

    QuickLearner Kilo Poster

    Messages:
    18
    Hi, Thank you very much for answering that question. I'm back to ask more and I hope others are watching and will jump in sooner or later with inputs and/or questions.
    1. Is there a way to know for a VPS account holder (me) that a server has reached an OOM situation?
    2. Are you aware of a script that would run as a cron job and do the OOM math on a regular basis and mail me if I've hit the red zone or maybe at 85-90%?
     
  11. fenster

    fenster Tera Poster

    Messages:
    429
    1. 'free' inside a VPS will show memory statistics for the hardware node (total memory, used memory, total swap, used swap). It's OK for Linux system to use as much RAM as possible, so free memory size is going to be not more than a few megabytes.

    2. We don't have such script but you can just take a look at the last column of /proc/user_beancounters. Nonzero values there mean that resource allocations were rejected.
     
  12. QuickLearner

    QuickLearner Kilo Poster

    Messages:
    18
    Ok, I shall try to write a script for that myself in a few days time. Just if you could advise me what should an ideal script do. I thought of the following ideas:
    1. This script should be run on a regular basis (maybe on an hourly basis?)
    2. It should mail the root the moment the last column value changes to non-zero (i think positive values would be a problem right? or is there a meaning attached to the negative values too?)
    3. It should do the math as follows:
    add up the following curr vals:
    oomguarpages + tcpsndbuf + tcprcvbuf + othersockbuf + kmemsize
    If this sum of curr vals is 85% of the oomguarpages barrier value, it should send a mail to the root....prolly thats the time for an upgrade of services or taking a closer look at the VPS and checking what is the reason for such a rise of those values.

    Could you suggest something on this topic? Or do you think this exercise is really not necessary.

    Further, I have some more questions:
    1. The hosting company decides the barrier values for each of their plans, also they might be setting different values for different clients. Are there really any standards for setting these values to the optimum level?

    For instance, we have some known performance degradation of a Windows system when the VM is set tooo low or even tooo high. Ofcourse it also depends on what the system is being used for....but in general afaik twice the amount of RAM is the standard on a Windows system

    2. I just wanted to know if there are some variables that i could query instead of manipulating the user_beancounters file.

    Thanks a lot for your time
     
  13. QuickLearner

    QuickLearner Kilo Poster

    Messages:
    18
    Bump....sorry about that...just wanted to know the answers if possible
     
  14. dcabbar

    dcabbar Guest

    Hi,

    This has been a great thread, and thanks for the explanations.

    Can you please elaborate the difference between oomguarpages and privvmpages, because both seem to be showing mem + swap. Is the difference between two about allocated but not used pages? I mean, privvmpages show allocated (some of which might not be used) for mem + swap, whereas oomguarpages show allocated, all of which are used, mem + swap?

    Once again, thanks a lot for the explanations.
     
  15. fenster

    fenster Tera Poster

    Messages:
    429
    Yes, oomguarpages current value shows actual usage of MEM+SWAP.
     
  16. fenster

    fenster Tera Poster

    Messages:
    429
    QuickLearner: sure you can write such script which will monitor UBC parameter. In fact, VZMC / VZPP do that (it has all these red zones, black zones, etc.)

    Failcounters cannot be negative. Positive value N does not mean that the VPS is broken, it just means that resource allocation has failed N times.

    Regarding standard configuration... please take a look at *sample files in /etc/sysconfig/vz-scripts/ which contain examples of configuration for certain amounts of memory for the VPS. See also: http://faq.sw-soft.com/article_130_692_en.html (yes, we created some kind of Virtuozzo FAQ :))
     
  17. GhostBuster

    GhostBuster Kilo Poster

    Messages:
    24
    What a superb thread!

    I have learnt so many things from this. Thanks to QuickLearner and PLESK Support.
     
  18. Hi All,

    Could you please explain what the shmpages parameter is for?
    We have 2 pretty much identical VPS (256 MB guaranteed and 1GB burstable), the only difference is:
    VPS1 shmpages: 131,072
    VPS2 shmpages: 10,000

    Thank you.

    --
    Sergiy Korobkov
    http://koredesign.com
     
  19. fenster

    fenster Tera Poster

    Messages:
    429
    Here is a quote from Virtuozzo Management or System Resources guide, pp. 39-40. This guide is shipped with Virtuozzo distribution.

    Just to make it more clear: this parameters limits resources shown by 'ipcs' command inside a VPS. It is accounted in 4k pages (for i386). These pages are also accounted in privvmpages parameter.
     
  20. Thanks, Alexander!
    This helps a lot.
     

Share This Page