Globalnet controller now uses internal services with external-ips
to support exported services. On OCP Clusters, we require an explicit
RBAC to create services with external-ips, this PR includes the
necessary RBAC for Globalnet pods.
Related to: https://github.com/submariner-io/submariner/issues/1166
Signed-Off-by: Sridhar Gaddam <sgaddam@redhat.com>
As part of Globalnet enhancement where kubeproxy dependency
is removed, the Globalnet Pod will now create internal
services for every exported service in the respective
namespace where the original service resides. This PR
adds the necessary clusterRole to allow Globalnet pod
to create/delete such internal services.
Related to: https://github.com/submariner-io/submariner/issues/1166
Signed-off-by: Sridhar Gaddam <sgaddam@redhat.com>
In the new event framework we use node listeners to support certain use-cases.
Currently in helm, routeagent does not have that role, so e2e tests are failing.
This PR enables it.
Related to: https://github.com/submariner-io/submariner/issues/858
Signed-Off-by: Sridhar Gaddam <sgaddam@redhat.com>
In an OCP Cluster, openshift-dns namespace has dns-default
service and this is controlled by its operator.
When Globalnet is deployed on OCP, it was seen that globalip
annotation added to it is periodically getting deleted by the
operator, so Globalnet tries to re-add the annotation and this
goes on forever. This will cause Globalnet to consume CPU
unnecessarily and could affect user-experience with Submariner
Globalnet. We have plans to enhance Globalnet to improve its
scalability, but until then we can exclude annotating services
in openshift-dns namespace.
Signed-Off-by: Sridhar Gaddam <sgaddam@redhat.com>
In an OCP Cluster, openshift-monitoring namespace has couple of
services and these are controlled by their respective operators.
When Globalnet is deployed on OCP, it was seen that globalip
annotation added to such services are periodically getting
deleted by the operators, so Globalnet tries to re-add the
annotation and this goes on forever. This will cause Globalnet
to consume CPU unnecessarily and could affect user-experience
with Submariner Globalnet. We have plans to enhance Globalnet
to improve its scalability, but until then we can exclude
annotating services in openshift-monitoring namespace.
Signed-Off-by: Sridhar Gaddam <sgaddam@redhat.com>
Currently, the updateStrategy for Route-agent pods is set to RollingUpdate
with maxUnavailable as 1. Because of this, when the route-agent DaemonSet
is updated, the Pods are updated one after the other. While the rolling
update strategy is good for user-facing services, for route-agent DaemonSet
its not the most suitable one as it takes time for all the route-agent pods
to be updated in a large cluster.
Ideally, it would have been great if "Recreate" updateStrategy was supported
for DaemonSets, but unfortunately K8s does not support it. But an alternate
way to achieve something similar is to configure rollingUpdate.maxUnavailable
to 100% so that the DaemonSet controller can update all the route-agent Pods
in a single go.
https://kubernetes.io/docs/tasks/manage-daemon/update-daemon-set/#daemonset-update-strategy
Fixes issue: https://github.com/submariner-io/submariner/issues/734
Signed-Off-by: Sridhar Gaddam <sgaddam@redhat.com>
In one of the earlier PRs, we modified the terminationGracePeriodSeconds
to 10 secs. However, its seen that this is causing more CI failures
during e2e redundancy tests. Ideally, once the Pods are terminated,
it should cleanup itself ASAP but it is seen that SM Pods are sometimes
taking time to exit and during this Period since there is no active SM
Pod running, this is triggering some failures.
Until we figure out the exact reason why the Pods are taking time for
cleanup, this PR reduces the terminationGracePeriodSeconds.
This issue would be properly addressed via
https://github.com/submariner-io/submariner/issues/694
Signed-Off-by: Sridhar Gaddam <sgaddam@redhat.com>
In K8s the default termination grace period for a Pod is 30
secs. However, for Submariner Pods, we were configuring this as
0 secs, because of which, cleanup was not happening in a
consistent manner and we are seeing failures in CI jobs.
This PR modifies the terminationGracePeriodSeconds to 10 secs.
Along with this change, it also uses the proper namespace of
submariner in the excluded namespaces.
Fixes issue: https://github.com/submariner-io/submariner-charts/issues/39
Signed-off-by: Sridhar Gaddam <sgaddam@redhat.com>
As part of supporting connectivity from HostNetwork to remoteClusters, globalnet
controller requires the CNIInterfaceIP on each node. This PR adds the necessary
clusterRoles for the route-agent daemonSet which will annotate the node with the
CNIInterfaceIP on that respective node.
Signed-off-by: Sridhar Gaddam <sgaddam@redhat.com>
As part of supporting connectivity from HostNetwork to remoteClusters, globalnet
controller annotates a node with globalIP. This PR adds the necessary roles for
globalnetController.
Signed-off-by: Sridhar Gaddam <sgaddam@redhat.com>
This PR provides a mechanism to configure the following
Submariner Gateway leader election values
1. leaseDuration
2. renewDeadline
3. retryPeriod
Signed-off-by: Sridhar Gaddam <sgaddam@redhat.com>
Currently, the Submariner Engine is a deployment with nodeSelector
(for submariner.io/gateway=true). It is seen that when the label
is removed from the node after SM engine is deployed, SM engine
continues to run on that node. This behavior is not in Sync with
the DaemonSet (used by GlobalnetController) behavior where the POD
is immediately terminated when the label is removed from the node.
This PR updates SM engine as DaemonSet so that we have consistent
behavior between the SM Engine POD and GlobalnetController POD.
Signed-off-by: Sridhar Gaddam <sgaddam@redhat.com>
Submariner-route-agent daemonset pod needs to query the list of other sm-route-agent
pods in the cluster for their hostIP addresses. The ipaddress will be used as remote
VxLAN Vtep IPs. Currently, sm-route-agent pod does not have the necessary privileges
to list the pods in the submariner namespace of local cluster. This patch addresses
this issue by adding the necessary role.
Error seen:
Failed to list *v1.Pod: pods is forbidden: User
"system:serviceaccount:submariner:submariner-routeagent" cannot list resource "pods"
in API group "" in the namespace "submariner"
Both iptables and nftables use netfilter framework in the kernel for
packet filtering. Many distributions are moving in the direction of
using nftables over iptables. Although, nftables uses a new command
line utility (named nft), starting from iptables >=1.8, it uses
nftables under the hood while continuing to support the same iptables
syntax from the user.
Quoting from Dan's comment [#]
"In iptables 1.8, the maintainers have "deprecated" the classic ip_tables:
the iptables tool now does userspace translation from the legacy UI/UX,
and uses nf_tables under the hood. So, the commands look and feel the
same, but they're now programming a different kernel subsystem.
The problem arises when you mix and match invocations of iptables 1.6
(the previous stable) and 1.8 on the same machine, because although they
look identical, they're programming different kernel subsystems.
Empirically, this causes weird and wonderful things to happen - things
like if you trace a packet coming from a pod, you see it flowing through
both ip_tables and nf_tables, but even if both accept the packet, it then
vanishes entirely and never gets forwarded"
So, as long as we are programming either nf_tables or iptables, we would
not have any issues. Currently, there is no easy way to identify what type
of rules are programmed on the host. This patch follows the same approach
(as described here [*]) that is taken in OpenShift where the host file
system is mounted inside the docker container and iptables utility on the
host is exec'ed for programming any firewall rules.
[#] https://github.com/kubernetes/kubernetes/issues/71305#issuecomment-448052889
[*] https://github.com/kubernetes/kubernetes/issues/71305#issuecomment-521978797
Normally in a vanilla kubernetes deployment, there is a single endpoint for kubernetes.
However, in some deployments (like OpenShift), there could be multiple endpoints.
This patch updates the SUBMARINER_BROKER_URL to use the appropriate "https" endpoint.
