Length limit of IP addresses per interface/container?

Discussion in 'Networking Questions' started by SteveITS, Apr 22, 2015.

  1. SteveITS

    SteveITS Mega Poster

    Messages:
    218
    Having found out that I have to preassign IPs to containers in order to have them work when Plesk Automation sets up the IP, I came up with a way to generate prlctl commands to do that for IPv6 blocks. It was humming along my block of 256 from :0 to :CC and suddenly started erroring out with "Failed to configure the virtual machine." The container's ifcfg-eth0 file ends like so:

    ...
    aaaa:bbbb:0:4C::2:C7/64 aaaa:bbbb:0:4C::2:C8/64 aaaa:bbbb:0:4C::2:C9/64 aaaa:bbbb:0:4C::2:CA/64 aaaa:bbbb:0:4C::2:CB/64 aaaa:bbbb:0:4C::2:CC/64 "

    However the list in PVA ends truncated:
    ...
    aaaa:bbbb:0:4C::2:CA
    aaaa:bbbb:0:4C::2:CB
    aaaa:bbbb:0:4C::

    So presumably there's a string length limit at work here, but for some reason PVA truncates the last line in the web interface. Makes me nervous to leave it and nervous to remove the truncated one. (If I get that far I'll probably remove :CB first)

    Presumably the workaround is to add more network interfaces?

    Edit to remind myself: the limit seems to be about 2000 characters so this effectively limits containers to 64 IPv6 IPs (a /122 subnet) per interface. One can, however, add multiple interfaces with a "placeholder" IPv4 before adding the container as a service node into PPA. (for some reason PPA won't detect interfaces added on later, nor detect interfaces with only IPv6...)
     
    Last edited: Oct 23, 2015
  2. SteveITS

    SteveITS Mega Poster

    Messages:
    218
    Removing a valid IP didn't work since prlctl still tried to validate the truncated one. I had to remove that one as listed in the error message output:

    prlctl set containername --ipdel aaaa:bbbb:0:4C:0:0:0: --ifname eth0
     
  3. KonstantinB

    KonstantinB Odin Team

    Messages:
    68
    Hi,

    If you are using bridged network interfaces, then it looks like already known issue coded as PSBM-32709.

    Best regards,
     
  4. SteveITS

    SteveITS Mega Poster

    Messages:
    218
    I am using bridged. Is there a place the known issues are documented online?
     
  5. IP^__^

    IP^__^ Odin Team

    Messages:
    80
    Hello Steve,

    Our bugtracker is internal only.
    Some issues are described in our KB: http://kb.odin.com/
     
  6. SteveITS

    SteveITS Mega Poster

    Messages:
    218
    I'm pretty sure when I first encountered this I got an error when I hit the limit. I don't know if it's because IPV6ADDR_SECONDARIES ended with a space this time, instead of a partial address, but prlctl now just hangs when trying to add the IP that would go over the limit, and also hangs when trying to remove one.

    Editing ifcfg-eth1 inside the service node and restarting networking seems to work, but of course PVA doesn't know that the IP was removed.

    When at the limit, PVA can't remove addresses because "Request-URI Too Large: The requested URL's length exceeds the capacity limit for this server." So the network configuration page in PVA uses GET not POST to submit its form (in addition to nesting the <form> tag inside a <tr> tag but outside a <td> tag, which seems a tad odd).
    I did find that in the PVA container, if I temporarily add a LimitRequestLine 16380 line to /etc/httpd/conf.d/anyfile.conf and run "apachectl graceful" to restart Apache, I can remove IPs using PVA (I didn't try prlctl).

    Note to devs: if prlctl would put one space between IPv6 addresses instead of two, it would probably fit a few more addresses in there...obviously the length of the string is dependent on the length of the collapsed IPv6 addresses so every bit helps. Seems like it would be a better long term solution to use POST on the form though.
     
    Last edited: Oct 23, 2015
  7. SteveITS

    SteveITS Mega Poster

    Messages:
    218
    I ran into this yet again, I think, even with my configuration change, when adding 64 IPv6 addresses to eth1 (x:x:x:x::2:140-17f). I could add up to :172 and prlctl hangs. PVA can add the rest (with my configuration change above) but I cannot ping anything over :172, even if I remove and re-add several at the end. In fact if I remove :172 and add :173 instead it still can't ping6 (Destination unreachable: Address unreachable).

    Is it possible that Virtuozzo is in a state where it thinks :173 - :17f route somewhere else?

    Or is it safest to say the Virtuozzo is limited to only 32 IPv6 per interface, and multiple network interfaces are necessary to get to 64, 96, 128, etc.? I added a third interface and moved :173 to it, and after a couple seconds I can ping it there, so I guess 32 is a safer limit than 64.

    Update: prlctl hangs adding the first additional IPv6 to eth2 (the second IPv6 on the interface). So apparently one needs to use PVA to add IPv6s to additional interfaces reliably.

    Update 2: with 4 interfaces of 32 IPv6 each, the LimitRequestLine directive on the PVA server needs to be higher than 16380. I just set it to 65520.
     
    Last edited: Nov 23, 2015
  8. SteveITS

    SteveITS Mega Poster

    Messages:
    218
    And if any lurkers are wondering why we can't just use shared IPv6 addresses, we can't...Plesk Automation apparently only supports dedicated IPv6 addresses for a webspace. It makes sense, if only we could get PVA to let our 4 quintillion IPv6 addresses be usable...
     
  9. SteveITS

    SteveITS Mega Poster

    Messages:
    218
    I ran into this again, adding a 6th IPv6 address to a server that had ::2:260 assigned at creation. Experimenting a bit, I found that I can remove and add ::2:261 - ::2:264 at will, using prlctl, but anything ::2:265 and over hangs. If I add through PVA, it completes the task and the IPv6 shows in "ip a l" inside the container, but it is not pingable from any hardware node ("Destination unreachable: Address unreachable").

    I noticed that when it works, a bunch of firewall related entries are logged to parallels.log on the hardware node, but when it doesn't work, nothing is logged. Removing or adding ::2:261 - ::2:264 does not log anything to parallels.log. Should it?

    Is the problem simply that the iptables rules are not being updated correctly and the firewall is blocking everything?

    Are we the only ones using IPv6?
     
  10. SteveITS

    SteveITS Mega Poster

    Messages:
    218
    The strange thing is the last container I set up, I added 31(+1) addresses via a script calling prlctl 31 times, no problem.
     
  11. Pavel

    Pavel A.I. Auto-Responder Odin Team

    Messages:
    432
    Interesting issue. There is no guarantee that this is the same as initially reported - as it seems that you have not exceeded the limit.
    It could actually be related to the ebtables configuration. I'd be able to answer if you provided me with output of container's config file, "ip a l" and "ebtables -nL" outputs.
    Output should be collected when you've added IP addresses which are not reachable.
     
  12. SteveITS

    SteveITS Mega Poster

    Messages:
    218
    I agree the behavior seems to have changed since my first report. I didn't realize that at first so sorry for mixing issues in one thread.

    I created a ticket in our system and scheduled time to talk to support, however, since my post on Jan. 8 I have created a few more service node containers with a "sleep 5" command after each, with no issues adding addresses:

    prlctl set containername --ipadd 2607:ff50:0:4c::aaa:bb1/64 --ifname eth1 && sleep 5
    prlctl set containername --ipadd 2607:ff50:0:4c::aaa:bb2/64 --ifname eth1 && sleep 5
    etc.

    ...so either something changed with a PCS update and it started working Jan.8 on, or the "sleep" command helps it sort something out. I haven't tried lately without the "sleep"...don't break it if it's working!

    I opened one of the affected nodes, verified it was missing an IP6 address, and tried to add it from the hardware node as above. It hung. Output is:

    [edit: IP address and command output details removed]

    Note I can ping6 all the IP addresses listed above (pings from you will be blocked by our firewall). So on this node the issue is that adding a 12th IPv6 address causes prlctl to hang.
     
    Last edited: Feb 2, 2016
  13. Pavel

    Pavel A.I. Auto-Responder Odin Team

    Messages:
    432
    It definitely has nothing to do with the string length since I was able to assign more than 70 ip addresses without any issues. I mean, amount of addresses is not at fault here.
    Reason why prlctl hangs is unclear and I don't even have a guess.
    Normally what prlctl is doing - it connects to dispatcher via port 64000 locally and sends a command. Dispatcher starts to execute it, once it's done - returns the command result.
    If prlctl is hanging here it might mean either dispatcher itself has got some issues, or some sub-operation (essential for adding IP address) is hanging.

    I'd recommend you to contact technical support for deeper investigation - it might require debugging dispatcher which would be quite difficult to do over the forum thread.

    P.S. "ebtables -L" is the correct command, it doesn't take "-n", however I no longer need the output since clarification I got proves ebtables are unrelated here
     
  14. SteveITS

    SteveITS Mega Poster

    Messages:
    218
    Thanks for the follow up Pavel. I looked up our notes, and we installed Virtuozzo 6.0 Update 10 Hotfix 9 on Jan. 8 but that was before prlctl hung on at least one server. I'll contact support and in the meantime continue using "sleep" since that seems to work, or at least hasn't failed yet.

    I am thinking there are three issues:
    1) the original string length issue for lots of IPv6 addresses (maybe networking needs to allow networks like /122 on NICs)
    2) prlctl hanging when adding addresses
    3) PVA supposedly successfully adding addresses but they are not usable afterwards

    On to tech support then!
     
  15. Carl Harris

    Carl Harris Bit Poster

    Messages:
    1
    Thanks for the info!
     
  16. SteveITS

    SteveITS Mega Poster

    Messages:
    218

Share This Page