r/devops • u/MountainTruth6073 • 11d ago
Discussion Kubernetes interview gone really bad
I went to a kubernetes tech interview, expecting that they are going to ask me about my experience using kubernetes and some basic question or some system design about how I could possibly build a cluster some scratch but nop they end up asking me questions that I found it very difficult to answer from the top of my head:
- First warm up question was, Can you explain to me what it is Quorum?
- Next question, I guess it was a follow up question, Do you know what ETCD? What is the difference between ETCD and Redis?
- Next question, Given this CNI Flannel, Cilum and Calico, which one you will use and why?
- And the killer question that literally frozen me was explain to me under the hood what is happening when a user clicks a button to download a file.
As a Senior Kubernetes Engineer, it is realistic to know all this stuff from the top of my head? Does it makes sense?
I have the feeling that the interview was setting me to fail, I never have to have to memorize things in my career, I always have to understand and get into conclusion by reasoning. But this is the third interview where the interviewer expect me to know things by heart... I was in another interview where the interviewer asked me to name all of the type of kubernetes services and explain them, I forgot to mention the headless service which I never use in my life.
Maybe it is realistic, that is why I need to vent and have another point of view
43
u/thomsterm 11d ago
well, look, even if you flopped you learn what people will ask you in an interview, so you have experience and know what to learn more.
Now the other thing is if you'll need to know that to actually do your job properly, cause this is a totally diff thing.
But from what I'm looking at those questions aren't that hard and unrelated to what we do. Those are the basic components and I'd kind of expect from someone that has a senior level to know something about them.
12
u/BlakkMajik3000 Platform Engineer 11d ago
Platform engineer here, mainly devex.
You…probably should know those answers, even at the junior level.
Is this a bot trolling?
3
u/WholeBet2788 11d ago
I hope this is a troll. Even junior k8s with 1 year could answer these :-D
4
u/Negative-Thinking 11d ago
Not really. People who only use managed EKS and AKS mostly never touch those things. On-premise is a different thing
0
u/BlakkMajik3000 Platform Engineer 11d ago
I'd disagree. Before doing DevEx myself, I had to know a bit about "what lies beneath" even though we were using managed AKS.
It also doesn't help that OP themselves said they were a senior k8s engineer. So...🤷🏾♂️
1
u/Low-Opening25 11d ago
yeah but you can still totally manage GKE and to large extent EKS without knowing any of it, so there is probably lots of “engineers” that fall into this category.
1
u/Negative-Thinking 11d ago
Exactly, just because you were exposed to it does not mean everyone who manages EKS clusters has to know each and every concept by heart, I surely don't and it never once was a problem in many years of managing multiple clusters with tens of services. Can I still call myself a senior engineer? If I will have a practical need for anything I don't know I will learn it on the job, otherwise it's just noise.
1
u/WholeBet2788 11d ago
Are you really senior if you dont know difference between etcd and redis?
0
u/Negative-Thinking 10d ago
Seniority is not defined by knowledge of every little thing. Sure, you might know what redis is. Etcd? EKS manages it for you, you don't even have to know about it. Noise. If you ever have to work with on premise, you will have to know about it, but learning this concept takes literally minutes. So it's a BS metric.
5
u/WholeBet2788 10d ago
Yes, its not defined by knowledge of every little thing.
But if you claim to be senior in specific technology, which OP does, i expect you to know the technology in depth. An etcd question is not depth. If you go into official k8s website, its probably on first picture you will find...
Surely you dont believe that senior k8s engineers means you know how to upgrade cluster inside console and deploy application, right?
3
u/Negative-Thinking 10d ago
You know what, you are right, if the title specifically says senior K8s engineer, then I agree.
→ More replies (0)1
u/Cute_Activity7527 10d ago
Must be AI troll, or those offshore visa engineers applying for job that have zero experience.
43
u/smarzzz 11d ago
> As a Senior Kubernetes Engineer
I have some doubts about that
I might be too autistic to not understand this as a parody. But these questions are very basic Kubernetes platform engineering questions.
How do hell do you not know what quorum is?! Of the difference between ETCD and redis. Kubernetes 101 mate. Were you only a user, and never a hoster/supplier of this platform?
-5
11d ago edited 11d ago
[deleted]
7
u/BrocoLeeOnReddit 11d ago edited 11d ago
Etcd and Redis are both key-value stores, with etcd being the default storage in K8s. It has more limited data types but is much faster.
Quorum basically means a voting majority when it comes to state, e.g. let's say you only have two servers and one goes down, then the two servers can't decide which is the correct state because both think it's their own. If you have 3 servers and one has a different state, the other two "outvote" it, that's what's called quorum.
1
u/PixelOrange 11d ago
Yes all of those things pertain to k8s.
I don't know that I could answer why I would use calico over cilium or vice versa. I think that's a preference thing or special use cases. I've never used flannel but the interwebs says it's more for simple setups.
Etcd vs redis is about how you store your data. That one is important.
11
u/OGicecoled 11d ago edited 11d ago
You’re a senior engineer you should be able to explain quorum. Literally a fundamental CS topic.
The other two you should be able to answer by just being a “senior Kubernetes engineer”.
2
u/Obvious-Jacket-3770 11d ago
Senior KUBERNETES Engineer...
1
u/OGicecoled 11d ago
Nuts lol senior engineer freezing up on what happens when you download a file.
1
u/Obvious-Jacket-3770 11d ago
I mean shit I don't use K8S and I barely know anything about it but I can answer those questions lol.
1
6
u/KageRaken 11d ago
I'm not even a kube admin and know all by heart. That's just knowing well known Linux services and a bit of web tech. Mind telling me what you've been doing all this time as a senior kube engineer if you've never heard of these concepts?
Not trying to get under your skin. I'm genuinely interested to know.
If you're expected to manage and debug kube clusters, this is basic stuff.
If you're only going to use an aws managed service... No you don't need to know.
5
u/No_Lifeguard7725 11d ago
If you can't understand those questions, you don't understand Kubernetes.
26
u/highball11 11d ago
These are basic bare minimum questions buddy
4
6
12
u/EgorWasHere 11d ago
A common format of DevOps interview is this kind of quiz and in my experience they pick really esoteric questions because if you know then then you definitely know the basics, I have also failed 3 interviews of a similar style, all I can say is learn the answers to all the questions asked and be ready for that one next time.
11
u/_N0K0 11d ago
This is not esoteric questions though. The CNI question is a bit funky, but I do have opinions there too.
5
2
u/Low-Opening25 11d ago
well it isn’t funky when you consider that majority of what kubernetes does is manipulating network stack, choice of CNI is critical.
6
3
u/dxlsm 11d ago
I hate the overuse of kubernetes with an everliving passion and will die on the hill that it isn’t the answer for every workload. I begrudgingly use it in a few non-ideal places where other container orchestration would be a better choice. I would consider myself senior devops and not a kubernetes expert, but answers to those questions are things that a senior should know or at least be able to talk about intelligently. I’m not super familiar with the differences between the CNI stacks they mentioned, but if I was actually a k8s engineer who had set up and/or maintained native clusters on a regular basis, I wager I’d be able to talk about at least one or two of them and know some of the advantages and disadvantages.
These were the questions to weed out the senior k8s engineers from the “senior k8s engineers.”
2
u/jumpsCracks 11d ago
Yeah that's pretty typical.
With that said, I'm a working senior k8s/automation eng also, and I couldn't answer all of those questions very well. Don't feel too bad about it. I'm terrible with that kind of detail oriented information, but I pick it up quickly enough that it doesn't matter. For example, I've done the exact CNI comparison and presented the comparison to leadership, and I genuinely couldn't tell you the difference now.
HOWEVER if you're job searching you should definitely know all of that information. It's the kind of thing that always gets asked in interviews, and isn't terribly hard to study.
2
u/hello2u3 11d ago
most shops use some flavor of managed K8s so it's hardly typical. Also I hold a CKA and CKS cert and those tests are open reference. I learned those things and they promptly left my my memory because I work on actual problems like running workloads.
2
2
u/strongjz 11d ago
I'm a principal SA who works with kubernetes, I would expect a senior kubernetes engineer to know these answers.
2
u/lavahot 11d ago
As a senior kubernetes engineer you should know at least something about each of these questions. Like,... you have to know what a quorum is and why they're important, right? You might not remember the exact mechanisms that implement quorums, but as someone who has deployed a kubernetes cluster before you for sure have planned quorums.
2
u/KarneeKarnay 11d ago
As someone who got into DevOps with K8s 6 years ago, I couldn't tell you most if these. The etcd Vs redis is the one I feel like I could answer effectively, the others not so much.
The download one as well is one I could answer. That's not got a single answer. It's used more to see how deep a grasp people have on networking, protocols, even UI stuff. Lot of depth you can go into. The version I've used to ask people was "what happens when you enter a URL into a browser?"
Listen, you have these bad interviews and they suck. You use the experience from this one to work out what you need to brush up on.
Lots of people seem to giving you shit, I've no idea why. The majority of K8s doesn't happen on bare installation, it happens with Open shift, eks or some other tooling because it's a pain in the ass to do.
What I would suggest is brush up on your K8s, maybe go through the effort of setting up a bare metal install and see how it goes.
I think the Calico one is DNS?
Tbc things change and you as a DevOps engineer just have to try and be on top of them. Most companies I think will understand that you can't retain knowledge about everything or stay up to date on everything. Be honest and tell them what you'd do to find out the answer.
3
u/PartemConsilio 11d ago
There is an in-between type between “cloud managed” and “bare metal”. I a work in the government and being cloud agnostic is huge in my area, so they use Rancher RKE2 kubernetes with OCI VMs for the backing. We manage everything, even though it all lives in the cloud.
We’re not trying to give him shit but show the reality is that moving up in Kubernetes knowledge to the point of having “senior” in your title means that there should be more depth in your understanding of the infra whether you use it in your day to day or not.
1
3
u/The_Userz 10d ago
agreed. I dont know shit about any of those answers. i hear the word redis, and i can tll you its a db. I know you use a yml file to spin up a k8 cluster, then set apps to each of them and additional features based on whatever requirements are needed. Honestly you shoulda just spoke how you used k8 vs answer those questions. Sometimes its not what you can recite for verbatim but what you can prove through idea concept.
2
u/hitesh_iat1 Senior Devops and Azure SRE 11d ago
Quorum is heart beat if the node is alive or not That's based on coordination systems like zookeper and based on CAP theorem. Good questions though
2
u/Fun_Protection5273 10d ago
You are venting about two different problems and its worth separating them.
Problem1: Some interviews do set unrealistic bars. Asking a senior engineer to name every service type including headless something you've never used in prod is a trivia test not competency assessment. That's a bad interview and you're right to be frustrated
Problem2: some of those questions are actually fair for a senior level. Quorum. etcd CNI tradeoff are things a senior engineer should be able to reason through confidently even if not from pure memory. And the background workings of the question what happens when user clicks a download button is all about checking whether you can think in systems.
The pattern you're describing suggests the gap is not technical know-how its more about how you perform when you are put on the spot. This is a very different problem to solve and the good news is it is completely solvable. Mock interviews under realistic pressure conditions close this gap faster than any amount of cramming tech docs.
2
u/ExternalComment1738 10d ago
honestly for a senior k8s/platform role those questions are pretty normal 😭 not because they expect wikipedia-perfect definitions, but because they’re testing whether you understand distributed systems underneath kubernetes instead of only operating yaml
the “download a file” question especially is usually less about memorization and more about seeing how deep your systems reasoning goes 💀 DNS/network/TLS/load balancer/ingress/backend/storage etc
BUT i also think modern infra interviews became kinda ridiculous lately because companies increasingly expect platform engineers to know networking + distributed systems + cloud + security + observability + devops + architecture all at once
so i wouldn’t take it as “you’re bad at kubernetes.” sounds more like you hit an infra-heavy/systems-heavy interview loop rather than a pure operational k8s one
2
u/Mobile_Particular895 8d ago
depends on what they listed as the role level. for "senior kubernetes engineer" those questions actually are reasonable; quorum (and etcd's raft consensus generally) is a real production concern when you have a 3-node control plane and lose one node. for anything below senior they were trying to trick you out of the role.
honest read: either the interviewer set the bar way too high for the level (which says more about their hiring process than your skills) or your resume is overselling depth you don't have. either way the recovery is the same: go learn the etcd / raft / control-plane internals at the level the questions implied (kubernetes the hard way is the gold-standard walkthrough), then re-interview. one bad k8s interview teaches you what to study for the next ten. you didn't lose anything.
2
u/Mobile_Particular895 8d ago
depends on what they listed as the role level. for "senior kubernetes engineer" those questions actually are reasonable; quorum (and etcd's raft consensus generally) is a real production concern when you have a 3-node control plane and lose one node. for anything below senior they were trying to trick you out of the role.
honest read: either the interviewer set the bar way too high for the level (which says more about their hiring process than your skills) or your resume is overselling depth you don't have. either way the recovery is the same: go learn the etcd / raft / control-plane internals at the level the questions implied (kubernetes the hard way is the gold-standard walkthrough), then re-interview. one bad k8s interview teaches you what to study for the next ten. you didn't lose anything.
4
u/tadamhicks 11d ago
I think in asking they’re trying to gauge how much you understand about the system. Quorum is something I’d expect a senior DevOps engineer to understand, yes. For k8s you should definitely know etcd and the role it plays. Maybe you don’t know all the fine points of raft but you should know what it is. And I’m going to guess they don’t care if you know Redis specifically but I’d expect you to know why etcd might be a better database than something else for kubernetes.
The CNI question I dunno…if I were interviewing you I’d hope you’d asking me some questions about ones you don’t have experience with that would show me that you understand what a CNI is and does and are capable of thinking about what critical capabilities differentiate each for different use cases.
The file download question is an old one. Another one is “what happens when you go to google in your browser?” That’s just gauging that you know how computing systems work in general. For those there’s often no 100% right answer…you could cover just the networking bits, the mem management, kernel modules, go to whatever depth…just display that you understand systems.
All this seems pretty brass tacks for me for a senior role.
1
4
u/RumRogerz 11d ago
If you’re senior you should have had no problems talking about any of those points
2
u/GiraffeWaste 11d ago
I mean I don't even use kubernetes in my projects(big financial org) and even I know what etcd is. It's beginner stuff in k8s really.
2
u/vantasmer 11d ago
Yeah now a days this is like basic for even junior admins, let alone Kubernetes engineers
3
2
u/redvelvet92 11d ago
I’m not even a Kubernetes engineer and I know all the answers to these questions lol. What a joke.
2
u/Kazcandra 11d ago
Tbf, these seem pretty basic. I don't work with k8s (adjacent, I'm with db ops), and i can answer all of those except flannel/cilium/calico.
2
1
1
u/Fun_Floor_9742 11d ago
what did you expect mister senior, did you work long time for an MSP? you sound like people i work with at my msp jobs
1
u/zmttoxics2 11d ago
I’ll be honest if you can’t explain etcd I wouldn’t hire you as a jr k8s admin.
1
1
u/Low-Opening25 11d ago
all valid questions and should not be difficult to answer for a SENIOR engineer. maybe you need to recalibrate your expectations and adjust your title accordingly.
2
u/Cute_Activity7527 10d ago
First three questions were pretty simple and any senior should have deepd understanding of those paradigms / tools.
Last question was asked to make fun of you. Turns out you know very little.
0
u/White0ut 11d ago
I have no idea the answer to any of the kubernetes questions.
The only good question their is the last one IMO. Similar to what happens when you enter google.com and pres enter.
3
u/justaguyonthebus 11d ago
I really like that last one too. Kinda a classic at this point. But it's really revealing because of where people choose to go into more or less details.
0
u/Maleficent-Self-4003 11d ago
The CNI question is a bit odd because almost no one uses Flannel but knowing Cilium and the possibilities that it brings is something a senior MUST know
Strange that you became frozen with the last question, you don't know how a http request/response works? You never debugged the traces of an application? Are you sure you a really a senior?
32
u/PartemConsilio 11d ago
TBF, I could understand being a Senior DevOps Engineer and not quite knowing those answers, but if your specific title has Senior KUBERNETES engineer than I would expect you to know that stuff.
The problem is the questions they were asking you are primarily deeper infrastructure questions and if you’re someone who is used to dealing with EKS, AKS or some other form of a managed cluster, you probably never have to worry about knowing those components. But when you are dealing with self-managed infrastructure? That stuff is vital, man.