There have been several situations where we found the Nimble clusters
routing commodity iSCSI traffic through their management (1Gb) network
interfaces, possibly as a ramification of our network environment: storage
traffic is routed on a private IP space that is not NAT-ed to the outside
world, so it's necessary to have IP addresses on both the private and public
Spent some time last night flailing with the "route" command as implemented
in Nimble's more-or-less bash shell. Found that "route --edit" commands do not
reliably actually make the changes they should. In one case the changes seemed
to have been applied (a "route --list" afterward showed the new settings). In
another case the edit command seemed to apply without issues but route --list
showed no change in the settings.
In one case we were able to make the changes stick by removing and replacing
the route. For the 0.0.0.0 route, however, we were averse to removing it as that
would break existing connections to a lot of running services. Instead we
applied the changes needed so that "route --list" showed the correct settings
regardless of whether they were being applied. Then we failed over the cluster
from node A to node B, and then back.
The short version. Nimble's "route --edit" command does not appear to reliably enforce
changes. Try removing the route and replacing it, or applying it and then
failing over the cluster to reload settings.
Life is good.