Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

k8s 1.11.3 sysctl can't work for deployment #69213

Closed
slpcat opened this issue Sep 28, 2018 · 11 comments
Closed

k8s 1.11.3 sysctl can't work for deployment #69213

slpcat opened this issue Sep 28, 2018 · 11 comments
Assignees
Labels
kind/bug Categorizes issue or PR as related to a bug. sig/node Categorizes an issue or PR as relevant to SIG Node.

Comments

@slpcat
Copy link

slpcat commented Sep 28, 2018

Is this a BUG REPORT or FEATURE REQUEST?:

/kind bug
/sig node

apiVersion: apps/v1
kind: Deployment
metadata:
name: example-web
namespace: default
labels:
app: example-web
..........
securityContext:
capabilities: {}
privileged: false
sysctls:
- name: kernel.msgmax
value: "65536"
- name: net.ipv4.tcp_syncookies
value: "0"
- name: net.ipv4.ip_local_port_range
value: "1024 65535"
- name: net.core.somaxconn
value: "65535"

kubectl create --dry-run -f example-deploy2.yml
error: error validating "example-deploy2.yml": error validating data: ValidationError(Deployment.spec.template.spec.containers[0].securityContext): unknown field "sysctls" in io.k8s.api.core.v1.SecurityContext; if you choose to ignore these errors, turn validation off with --validate=false

@k8s-ci-robot k8s-ci-robot added needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. kind/bug Categorizes issue or PR as related to a bug. sig/node Categorizes an issue or PR as relevant to SIG Node. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Sep 28, 2018
@dims
Copy link
Member

dims commented Sep 28, 2018

@slpcat specifying it with Pod(s) work fine?

@dims
Copy link
Member

dims commented Sep 28, 2018

Also, please share the full yaml file

@slpcat
Copy link
Author

slpcat commented Sep 28, 2018

yes, it works with POD
cat pod.yml
apiVersion: v1
kind: Pod
metadata:
name: sysctl-example
spec:
containers:
- image: tomcat
name: tomcat
securityContext:
sysctls:
- name: net.ipv4.tcp_syncookies
value: "0"
- name: net.ipv4.ip_local_port_range
value: "1024 65535"
- name: net.core.somaxconn
value: "65535"

and in pod:
root@sysctl-example:/usr/local/tomcat# sysctl -a|grep maxconn
net.core.somaxconn = 65535

but deployment can't work

apiVersion: apps/v1
kind: Deployment
metadata:
name: example-web
namespace: default
labels:
app: example-web
spec:
replicas: 1
selector:
matchLabels:
app: example-web
template:
metadata:
labels:
app: example-web
spec:
containers:
- name: example-web
image: tomcat
imagePullPolicy: Always
resources:
requests:
cpu: 50m
memory: 64Mi
limits:
cpu: 500m
memory: 2048Mi
securityContext:
capabilities: {}
privileged: false
sysctls:
- name: net.ipv4.tcp_syncookies
value: "0"
- name: net.ipv4.ip_local_port_range
value: "1024 65535"
- name: net.core.somaxconn
value: "65535"
env:
- name: PROFILE
value: "test"
ports:
- name: http
containerPort: 8080

@WanLinghao
Copy link
Contributor

/assign

@WanLinghao
Copy link
Contributor

@slpcat I have test your example in v1.13.0-alpha and no error occurs.

@slpcat
Copy link
Author

slpcat commented Oct 18, 2018

@slpcat I have test your example in v1.13.0-alpha and no error occurs.

in my test 1.11.3 and 1.12.0 can‘t work, I don’t have v1.13.0-alpha

@slpcat
Copy link
Author

slpcat commented Oct 23, 2018

it can work now , this is the yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: example-web
namespace: default
labels:
app: example-web
spec:
replicas: 1
selector:
matchLabels:
app: example-web
template:
metadata:
labels:
app: example-web
spec:
securityContext:
sysctls:
- name: net.ipv4.tcp_syncookies
value: "0"
- name: net.ipv4.ip_local_port_range
value: "1024 65535"
- name: net.core.somaxconn
value: "65535"
containers:
- name: example-web
image: tomcat
imagePullPolicy: Always
resources:
requests:
cpu: 50m
memory: 64Mi
limits:
cpu: 500m
memory: 2048Mi
env:
- name: PROFILE
value: "test"
ports:
- name: http
containerPort: 8080

@slpcat slpcat closed this as completed Oct 23, 2018
@zzvara
Copy link

zzvara commented Apr 1, 2019

It does not work for me.

spec:
      securityContext:
        sysctls:
        - name: net.ipv4.ip_forward
          value: "1"
      containers:
      - name: {{ .Chart.Name }}
        image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
        imagePullPolicy: {{ .Values.image.pullPolicy }}
        command: ["/etc/openvpn/setup/configure.sh"]
        ports:
        - containerPort: {{ .Values.service.internalPort }}
          name: openvpn
        securityContext:
          capabilities:
            add:
              - NET_ADMIN
          privileged: true
        resources:
          requests:

I have to use privileged and go in manually to change it. kubelet.env contains the allowance for this unsafe sysctl namaspace. What else has to be done?

@grealish
Copy link

So I've hit this issue recently and it's not solved, please reopen the issue as it's blocking allot of people can costing allot of money on a client.
Does anyone form the CNCF have any advise on esulcating this issue?

@jeffdesc
Copy link

jeffdesc commented Jan 14, 2020

Any update here? If I set the sysctls value it's still not working:

apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: test
  spec:
    replicas: 1
    template:
      spec:
        securityContext:
          sysctls:
            - name: net.ipv4.ip_forward
              value: "1"

Of course my kubelet allows net.ipv4.ip_forward and the PSP is configured for this.

However yet ip_forward is still 0:

user@test-d90fdsfd:/$ cat /proc/sys/net/ipv4/ip_forward
0

Running on v1.16.3:

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.3", GitCommit:"b3cbbae08ec52a7fc73d334838e18d17e8512749", GitTreeState:"clean", BuildDate:"2019-11-14T04:24:34Z", GoVersion:"go1.12.13", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.3", GitCommit:"b3cbbae08ec52a7fc73d334838e18d17e8512749", GitTreeState:"clean", BuildDate:"2019-11-13T11:13:49Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}

What's going wrong here?

@jeffdesc
Copy link

jeffdesc commented Jan 14, 2020

Created #87198 for my behaviour

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Categorizes issue or PR as related to a bug. sig/node Categorizes an issue or PR as relevant to SIG Node.
Projects
None yet
Development

No branches or pull requests

7 participants