Mongodb statefulset in kubernetes with arbiter
Objective is to deploy Mongodb ReplicaSet on kubernetes with an arbiter instance, where we have 1 primary, 1 secondary and 1 artiber.
Why we need arbiter?
Arbiters are mongod instances that are part of a replica set but do not hold data (i.e. do not provide data redundancy). They can, however, participate in elections. Arbiters have minimal resource requirements and do not require dedicated hardware.
Show me the yaml
Let’s get started with the yamls, my assumption here is that you are aware of statefulset, headless service and managing persistence volume in kubernetes.
Headless service for data nodes and arbiter.
---
apiVersion: v1
kind: Service
namespace: test #Change this to the required namespace
metadata:
name: mongodb-data-service
labels:
name: mongodb-data
spec:
ports:
- port: 27017
targetPort: 27017
clusterIP: None
selector:
role: mongodb-data# Replace test with the namespace
# mongodb-data-0.mongodb-data-service.test.svc.cluster.local
# mongodb-data-1.mongodb-data-service.test.svc.cluster.local
---
apiVersion: v1
kind: Service
namespace: test #Change this to the required namespace
metadata:
name: mongodb-arbiter-service
labels:
name: mongodb-arbitar
spec:
ports:
- port: 27017
targetPort: 27017
clusterIP: None
selector:
role: mongodb-arbiter# Replace test with the namespace
# mongodb-arbitar-0.mongodb-arbiter-service.test.svc.cluster.local
Statefulset for data nodes and arbiter
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: mongodb-data
spec:
serviceName: mongodb-data-service
replicas: 2 # 1 for primary and 1 for secondary. Increase this if more replicas are required
selector:
matchLabels:
role: mongodb-data
template:
metadata:
labels:
role: mongodb-data
replicaset: MainRepSet
spec:
volumes:
- name: config # You can skip this if you dont want Keyfile Access Control
configMap:
name: mongo-security
mode: 0400
defaultMode: 0400
terminationGracePeriodSeconds: 10
containers:
- name: mongod-container
image: mongo
command:
- "mongod"
- "--enableMajorityReadConcern"
- "false"
- "--bind_ip"
- "0.0.0.0"
- "--replSet"
- "MainRepSet"
- "--dbpath"
- "/data/db"
resources:
requests:
cpu: 0.2
memory: 200Mi
ports:
- containerPort: 27017
volumeMounts:
- name: mongodb-persistent-storage-claim
mountPath: /data/db
- name: config # You can skip this if you dont want Keyfile Access Control
mountPath: /etc/mongo.key
subPath: mongo.key
affinity:
podAntiAffinity: # To force scheduling on different nodes
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: role
operator: In
values:
- mongodb-data
topologyKey: kubernetes.io/hostname
- labelSelector:
matchExpressions:
- key: role
operator: In
values:
- mongodb-arbiter
topologyKey: kubernetes.io/hostname
updateStrategy:
type: OnDelete
volumeClaimTemplates:
- metadata:
name: mongodb-persistent-storage-claim
annotations:
volume.beta.kubernetes.io/storage-class: "gp2" # based on storage class you already have configured
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 5Gi # Change this according to your requirement
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: mongodb-arbiter
spec:
serviceName: mongodb-arbiter-service
replicas: 1 # Best practice is to use only one
selector:
matchLabels:
role: mongodb-arbiter
template:
metadata:
labels:
role: mongodb-arbiter
replicaset: MainRepSet
spec:
volumes:# You can skip this if you dont want Keyfile Access Control
- name: config
configMap:
name: mongo-security
mode: 0400
defaultMode: 0400
terminationGracePeriodSeconds: 10
containers:
- name: mongod-container
image: mongo
command:
- "mongod"
- "--enableMajorityReadConcern"
- "false"
- "--bind_ip"
- "0.0.0.0"
- "--replSet"
- "MainRepSet"
- "--dbpath"
- "/data/db"
resources:
requests:
cpu: 0.2
memory: 200Mi
ports:
- containerPort: 27017
volumeMounts:
- name: mongodb-arbiter-storage-claim
mountPath: /data/db
- name: config # You can skip this if you dont want Keyfile Access Control
mountPath: /etc/mongo.key
subPath: mongo.key
affinity: # To force scheduling on different nodes
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: role
operator: In
values:
- mongodb-data
topologyKey: kubernetes.io/hostname
updateStrategy:
type: OnDelete
volumeClaimTemplates:
- metadata:
name: mongodb-arbiter-storage-claim
annotations:
volume.beta.kubernetes.io/storage-class: "gp2"
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 1Gi # As we are not storing any data we can keep this to minimumBefore you apply this, you can also make use of taints, tolerations and node selectors for dedicated usage of nodes.
To create Keyfile, go to terminal and execute. (You can skip this if you dont want Keyfile Access Control)
openssl rand -base64 756 > ./mongo.keyCreate config map
kubectl create configmap mongo-security --from-file=./mongo.keyApply all yaml’s and wait for the pod to get created, once pods are up follow the below mentioned steps
- Exec into the pod
- Connect mongodb
- Add members and artbiter
- Check status
- Add admin user
$ kubectl -n test exec mongodb-data-0 -- bash$ mongo$ rs.initiate({ _id: "MainRepSet", version: 1,
members: [
{ _id: 0, host: "mongodb-data-0.mongodb-data-service.test.svc.cluster.local:27017" },
{ _id: 1, host: "mongodb-data-1.mongodb-data-service.test.svc.cluster.local:27017" }
]});$ rs.addArb("mongodb-arbitar-0.mongodb-arbitar-service.test.svc.cluster.local:27017")$ rs.status()$ db.getSiblingDB("admin").createUser({
user : "admin",
pwd : "password",
roles: [ { role: "root", db: "admin" } ]
});
Once user is created you try logging using.
$ use admin$ db.auth("admin","password")
Exec into other data node and check if you can login and get stats
$ rs.status()If you need Keyfile Access Control
Edit sts and change arguments for mongod-container. It should look like this.
command:
- "mongod"
- "--enableMajorityReadConcern"
- "false"
- "--keyFile"
- "/etc/mongo.key"# path to key file
- "--bind_ip"
- "0.0.0.0"
- "--replSet"
- "MainRepSet"
- "--dbpath"
- "/data/db"Start deleting pods one by one, i prefer to delete secondary and arbiter first.
This is it. Now, you should have a working replicaset.
