In 2026, tech interviews have shifted dramatically. Hiring managers no longer ask basic definitions like “What is Docker?” or “What is MLOps?” Instead, they present real-world infrastructural bottlenecks, system crashes, model drift scenarios, and pipeline failures to test your hands-on troubleshooting abilities.
Whether you are aiming for a Senior DevOps Engineer, MLOps Specialist, or Infrastructure Architect role, mastering scenario-based problems is key to clearing technical rounds.
Below is a curated list of 20 real-world DevOps and MLOps scenario-based interview questions with detailed, production-grade solutions.
๐ ๏ธ Section 1: Core DevOps & Cloud-Native Scenarios
Q1: Your Kubernetes Pod crashes repeatedly with status OOMKilled (Exit Code 137). How do you debug and resolve this without causing downtime?
Scenario Solution:
- Identify the Container: Run
kubectl describe pod <pod-name>to check theLast Stateand confirmExit Code: 137. - Fetch Heap Dumps/Metrics: Check Grafana metrics or Prometheus
container_memory_working_set_bytesto see if memory leaked gradually or spiked instantly. - Analyze Resource Limits: Check
resources.limits.memoryversusresources.requests.memory. - Resolution Strategy:
- If it’s a memory leak (e.g., Python object retention or Java heap issue), profile the application using tools like Py-Spy or JProfiler.
- If it’s a sudden burst load, adjust the limit dynamically or implement Horizontal Pod Autoscaling (HPA) based on custom memory metrics.
- Configure Vertical Pod Autoscaler (VPA) in recommendation mode to right-size container requests.
Q2: A production Deployment pipeline in GitHub Actions worked fine, but during a rollout, 30% of traffic started receiving HTTP 500 errors. How do you automate instant recovery?
Scenario Solution: Use a Canary Deployment strategy with automated rollback capabilities via Argo Rollouts or Flagger.
# Argo Rollout Snippet with Automated Prometheus Metric Analysis
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
name: core-api-rollout
spec:
replicas: 10
strategy:
canary:
steps:
- setWeight: 20
- pause: { duration: 5m }
metrics:
- name: success-rate
interval: 30s
successCondition: result[0] >= 0.99
provider:
prometheus:
address: http://prometheus-k8s:9090
query: |
sum(rate(http_requests_total{status!~"5.*"}[2m]))
/ sum(rate(http_requests_total[2m]))
- Explanation: If the error rate exceeds 1% during the 5-minute pause at 20% canary traffic, Argo Rollouts automatically aborts the deployment and rolls back to the previous stable ReplicaSet within seconds without human intervention.
Q3: You need to migrate 50 microservices from Jenkins on-prem to GitHub Actions with minimal developer friction. What architecture would you propose?
Scenario Solution:
- Reusable Workflows: Create a centralized
.githubrepository containing reusable workflow templates for build, test, Docker image construction, and Helm deployments. - Custom Composite Actions: Abstract complex steps (e.g., SonarQube scanning, Trivy vulnerability checks, ECR authentication) into reusable internal composite actions.
- Self-Hosted Runner Groups: Provision ephemeral self-hosted runners on AWS EKS using KEDA (Kubernetes Event-driven Autoscaling) or Actions Runner Controller (ARC) to auto-scale runners based on pending job queues.
Q4: Terraform state file gets locked by a failed CI/CD run, and a critical hotfix needs to be deployed immediately. How do you resolve this safely?
Scenario Solution:
- Verify no active CI/CD execution is writing to the state backend (e.g., AWS S3 + DynamoDB state locking).
- Retrieve the
Lock IDfrom the Terraform error trace. - Run
terraform force-unlock <LOCK-ID>with authorization from the Lead Architect. - Run
terraform planimmediately to inspect drift before applying the hotfix. - Update CI/CD steps to use explicit pipeline timeouts (
timeout-minutes: 15) to prevent hanging jobs from keeping state locks indefinitely.
Q5: How do you secure database passwords and API tokens when deploying microservices across multiple Kubernetes clusters using GitOps (ArgoCD)?
Scenario Solution:
- Never commit raw secrets to Git.
- Use External Secrets Operator (ESO) or Sealed Secrets.
- Store sensitive data centrally in AWS Secrets Manager, HashiCorp Vault, or GCP Secret Manager.
- Install ESO in the cluster to sync secrets directly from Vault/AWS Secrets Manager into Kubernetes native
Secretobjects dynamically at runtime.
Q6: Your ingress controller is dropping connections during peak traffic spikes (100k RPM). What components do you check?
Scenario Solution:
- Linux Kernel Limits: Increase worker connection limits and
sysctlnet sockets (net.core.somaxconn = 65535,net.ipv4.tcp_max_syn_backlog = 65535). - Nginx/Envoy Ingress Adjustments: Tune
worker-connections,upstream-keepalive-connections, and HTTP keep-alive timeouts. - Pod Disruption & HPA: Set proper Pod Disruption Budgets (PDB) and configure HPA to scale Ingress replicas based on CPU and Request-Per-Second (RPS) metrics.
Q7: How do you achieve Zero-Downtime database migrations when schema changes (e.g., column rename) break running application pods?
Scenario Solution: Use the Expand and Contract (Parallel Run) Pattern:
- Phase 1 (Expand): Add the new column (
new_email) without deleting the old one (old_email). - Phase 2 (Dual Write): Update application code to write to both columns and read from
new_emailwith fallback toold_email. Deploy. - Phase 3 (Backfill): Run a background script to migrate legacy data from
old_emailtonew_email. - Phase 4 (Contract): Remove dual-write logic, point application exclusively to
new_email, and dropold_emailin a subsequent release.
Q8: A Jenkins master instance is running out of disk space due to build logs and artifact storage. How do you prevent this architecturally?
Scenario Solution:
- Offload Artifacts: Configure Jenkins plugins to upload artifacts directly to Amazon S3 or Nexus/JFrog Artifactory during build completion.
- Log Rotation Policies: Enforce
Discard Old Buildsin Jenkinsfiles (logRotator(numToKeepStr: '15', artifactNumToKeepStr: '5')). - Ephemeral Agents: Run Jenkins execution nodes exclusively inside Kubernetes pods so agents are destroyed completely post-job execution.
Q9: How do you trace a latency bottleneck across 10+ microservices communicating via gRPC and REST APIs?
Scenario Solution:
- Implement OpenTelemetry (OTel) distributed tracing instrumentation across all microservices.
- Propagate context using W3C
traceparentheaders across incoming REST and outgoing gRPC metadata. - Ship trace payloads to Jaeger, Tempo, or Datadog.
- Filter by root trace span to identify which database query, network call, or downstream service is causing the high P99 latency tail.
Q10: Docker build times in CI/CD take 15+ minutes because of heavy dependency downloads. How do you optimize them under 2 minutes?
Scenario Solution:
- Multi-Stage Builds: Separate build dependencies from runtime binaries.
- Layer Caching: Reorder Dockerfile instructionsโcopy dependency manifests (
package.json,requirements.txt,go.mod) and install dependencies before copying application source code. - BuildKit Remote Cache: Use
docker buildxwith--cache-from type=registryand--cache-to type=inlineto share layer caches across CI runners.
๐ค Section 2: Advanced MLOps & AI Infrastructure Scenarios
Q11: An image classification model deployed on Kubernetes shows 95% accuracy in staging, but performance degrades heavily in production over 3 weeks. How do you detect and handle this?
Scenario Solution: This is caused by Data Drift (input distribution change) or Concept Drift (relationship between inputs and labels change).
Python
# Detecting Data Drift using Evidently AI in Python
from evidently.report import Report
from evidently.metric_preset import DataDriftPreset
def check_input_drift(reference_data, current_production_data):
drift_report = Report(metrics=[DataDriftPreset()])
drift_report.run(reference_data=reference_data, current_data=current_production_data)
report_json = drift_report.json()
if report_json['metrics'][0]['result']['dataset_drift']:
trigger_ml_pipeline_retraining()
def trigger_ml_pipeline_retraining():
print("Drift detected above threshold! Triggering Airflow/Kubeflow DAG...")
- Resolution:
- Setup continuous data quality monitoring via Evidently AI or WhyLogs.
- Emit custom metrics to Prometheus/Grafana.
- Automatically trigger retraining pipelines in Kubeflow or Airflow when Statistical Kolmogorov-Smirnov (KS) test p-values drop below threshold.
Q12: Your team needs to fine-tune an LLM (e.g., Llama-3) using QLoRA across multiple GPUs. How do you design the MLOps training and deployment pipeline?
Scenario Solution:
- Distributed Training: Use DeepSpeed + PyTorch FSDP (Fully Sharded Data Parallel) with QLoRA 4-bit quantization to fit model weights across multiple AWS
p4d.24xlarge(A100) org5nodes. - Experiment Tracking: Use MLflow or Weights & Biases to log hyperparameter sweeps (rank $r$, lora_alpha, learning rate), memory usage, and loss metrics.
- Model Registry: Save adapter weights (LoRA weights) into MLflow Model Registry.
- Serving Infrastructure: Deploy base model + LoRA adapters using vLLM or TGI (Text Generation Inference) on Kubernetes with GPU operator acceleration to allow fast dynamic adapter swapping.
Q13: A production ML API experiences high latency spikes during peak usage because GPU inference is slow. How do you optimize throughput without doubling hardware costs?
Scenario Solution:
- Dynamic Batching: Configure Triton Inference Server or vLLM to batch incoming concurrent requests dynamically within a small time window (e.g., 5ms window).
- Model Optimization: Convert model weights to TensorRT-LLM, ONNX Runtime, or OpenVINO to run fused kernel operations.
- Quantization: Quantize FP32/FP16 models down to INT8 or INT4 precision using AWQ or GPTQ techniques.
- Auto-scaling: Scale GPU pods using KEDA based on pending request queue length (
http-request-queue-depth) rather than raw GPU memory/CPU utilisation.
Q14: How do you build an automated CI/CD pipeline for ML models (CT – Continuous Training)?
Scenario Solution:
[ New Data Ingestion ] โโ> [ Data Validation (Great Expectations) ]
โ
โผ
[ Trigger Retraining ] โโ> [ Model Evaluation vs Champion Model ]
โ
(If Evaluation Score > Champion)
โ
โผ
[ Register Model in MLflow ] โโ> [ Deploy Canary Ingress to K8s ]
- Automated Workflow:
- Trigger: Webhook or scheduled Airflow DAG when new data threshold is reached.
- Validation: Run Great Expectations checks on feature store data.
- Evaluation Gate: Retrain candidate model (Challenger) and compare its F1/RMSE score against the existing production model (Champion).
- Deployment: If Challenger beats Champion by > 2%, push model to MLflow Model Registry with
Stagingtag, build container, and deploy to Kubernetes staging environment automatically.
Q15: How do you handle Vector Database schema migrations and re-indexing (e.g., Qdrant, Milvus, Pinecone) in a live RAG system with zero downtime?
Scenario Solution:
- Blue-Green Collection Pattern: Create a new Vector Index / Collection (
collection_v2) alongside the active one (collection_v1). - Background Embedding Generation: Run batch ingestion workers using Ray or Spark to populate
collection_v2with updated embedding models or chunking strategies. - Dual Ingestion: Route newly arriving real-time data to both
v1andv2. - Router Swap: Update your API gateway or application configuration layer to point semantic search queries from
v1tov2. - Decommission: Verify search performance metrics and delete
collection_v1.
Q16: How do you prevent data leakage between Training and Evaluation sets in automated feature store pipelines?
Scenario Solution:
- Use an Enterprise Feature Store (e.g., Feast or Hopsworks) with Time-Travel / Point-in-Time Joins.
- Ensure feature retrieval for model training uses accurate historical timestamps (
event_timestamp). - This prevents future feature states from bleeding into past training records (Lookahead Bias).
Q17: Large Language Model (LLM) responses in production are hallucinating and giving sensitive details. How do you implement real-time Guardrails in MLOps?
Scenario Solution: Implement a Proxy Guardrail Layer (e.g., NeMo Guardrails, Guardrails AI, or Llama Guard) in front of the LLM pipeline:
[ User Prompt ] โโ> [ Input Guardrails (PII Detection / Prompt Injection) ]
โ
โผ
[ LLM Inference ]
โ
โผ
[ Output Response ] <โโ [ Output Guardrails (Hallucination Check / Toxicity) ]
- Input Checks: Reject prompt injection attacks, system prompt leaks, and redact PII (Personally Identifiable Information).
- Output Checks: Compare generated answer against context sources in RAG systems; fallback to generic safe answer if context grounding score is low.
Q18: How do you monitor cost and token usage for multi-tenant LLM applications across different client organizations?
Scenario Solution:
- Observability Tooling: Integrate LLM observability solutions such as LangSmith, Langfuse, or OpenLiteLLM / OpenTelemetry.
- Metadata Tracking: Attach
tenant_id,user_id, anduse_casetags to every API request header. - Aggregated Billing: Export token consumption metrics (prompt tokens + completion tokens) to ClickHouse or Prometheus, visualizing tenant-wise costs on Grafana dashboards.
- Rate-Limiting: Enforce dynamic token quota rate limits per tenant using Redis token bucket counters.
Q19: An MLOps engineer leaves, and no one knows how a production model running in Kubernetes was trained. How do you enforce ML Reproducibility moving forward?
Scenario Solution: Enforce strict compliance using the MLOps Lineage Rule:
- Code Versioning: Enforce Git commit hash tagging for every model run.
- Data Versioning: Use DVC (Data Version Control) or LakeFS to track precise dataset snapshots used during training.
- Environment Lock: Build containerized training environments using Docker with locked base images and explicit
requirements.txt/ Poetry lockfiles. - Metadata Logging: Automate MLflow artifact logging to capture Python runtime, environment flags, seed parameters, dataset hashes, and Git commits automatically:
Python
import mlflow
mlflow.autolog(log_input_examples=True, log_model_signatures=True)
with mlflow.start_run():
mlflow.set_tag("git_commit", os.getenv("GIT_COMMIT_HASH"))
mlflow.set_tag("dvc_data_version", os.getenv("DVC_DATA_VERSION"))
# Training logic executed here
Q20: Your company needs to run AI Workloads (PyTorch model training) on AWS EKS using Spot Instances without job failures. How do you build this fault-tolerant pipeline?
Scenario Solution:
- Node Termination Handling: Deploy
aws-node-termination-handleron the cluster to catch 2-minute Spot Interruption Notifications from AWS. - Checkpointing: Instrument PyTorch code to save training state checkpoints to shared storage (AWS EFS or S3) every $N$ steps/epochs.
- K8s Tolerations & Taints: Use Node Affinity and Interruption Tolerations so training pods schedule gracefully across multiple Spot Instance pools.
- Automatic Resumption: Configure training jobs as Kubernetes
JobswithbackoffLimitand auto-resumption scripts that load the latest checkpoint from S3 upon pod restart on a new Spot node.
๐ Summary Checklist for Tech Interviews
| Concept | Key Tool / Solution |
| Kubernetes OOMKilled | Metric Analysis, Adjust Requests/Limits, VPA/HPA |
| Zero Downtime Canary | Argo Rollouts + Prometheus Metrics |
| Secrets Management | External Secrets Operator + AWS Secrets Manager / Vault |
| Data/Model Drift | Evidently AI, WhyLogs, Airflow Retraining DAGs |
| LLM Serving & Fine-Tuning | QLoRA, DeepSpeed, vLLM, Triton Inference Server |
| LLM Observability | OpenTelemetry, Langfuse, LangSmith |
| ML Reproducibility | DVC + MLflow + Locked Docker Envs |
๐ก Boost Your Technical Career
Are you preparing for upcoming DevOps, MLOps, or Cloud Architecture interviews? Explore more role-specific roadmaps, interview guides, and hands-on system design challenges on CareerGraph!












Leave a Reply