Top 20 DevOps & MLOps Scenario-Based Interview Questions

top-20-devops-mlops-scenario-based-interview-questions

In 2026, tech interviews have shifted dramatically. Hiring managers no longer ask basic definitions like “What is Docker?” or “What is MLOps?” Instead, they present real-world infrastructural bottlenecks, system crashes, model drift scenarios, and pipeline failures to test your hands-on troubleshooting abilities.

Whether you are aiming for a Senior DevOps Engineer, MLOps Specialist, or Infrastructure Architect role, mastering scenario-based problems is key to clearing technical rounds.

Below is a curated list of 20 real-world DevOps and MLOps scenario-based interview questions with detailed, production-grade solutions.

๐Ÿ› ๏ธ Section 1: Core DevOps & Cloud-Native Scenarios

Q1: Your Kubernetes Pod crashes repeatedly with status OOMKilled (Exit Code 137). How do you debug and resolve this without causing downtime?

Scenario Solution:

  1. Identify the Container: Run kubectl describe pod <pod-name> to check the Last State and confirm Exit Code: 137.
  2. Fetch Heap Dumps/Metrics: Check Grafana metrics or Prometheus container_memory_working_set_bytes to see if memory leaked gradually or spiked instantly.
  3. Analyze Resource Limits: Check resources.limits.memory versus resources.requests.memory.
  4. Resolution Strategy:
    • If it’s a memory leak (e.g., Python object retention or Java heap issue), profile the application using tools like Py-Spy or JProfiler.
    • If it’s a sudden burst load, adjust the limit dynamically or implement Horizontal Pod Autoscaling (HPA) based on custom memory metrics.
    • Configure Vertical Pod Autoscaler (VPA) in recommendation mode to right-size container requests.

Q2: A production Deployment pipeline in GitHub Actions worked fine, but during a rollout, 30% of traffic started receiving HTTP 500 errors. How do you automate instant recovery?

Scenario Solution: Use a Canary Deployment strategy with automated rollback capabilities via Argo Rollouts or Flagger.

# Argo Rollout Snippet with Automated Prometheus Metric Analysis
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: core-api-rollout
spec:
  replicas: 10
  strategy:
    canary:
      steps:
      - setWeight: 20
      - pause: { duration: 5m }
      metrics:
      - name: success-rate
        interval: 30s
        successCondition: result[0] >= 0.99
        provider:
          prometheus:
            address: http://prometheus-k8s:9090
            query: |
              sum(rate(http_requests_total{status!~"5.*"}[2m])) 
              / sum(rate(http_requests_total[2m]))
  • Explanation: If the error rate exceeds 1% during the 5-minute pause at 20% canary traffic, Argo Rollouts automatically aborts the deployment and rolls back to the previous stable ReplicaSet within seconds without human intervention.

Q3: You need to migrate 50 microservices from Jenkins on-prem to GitHub Actions with minimal developer friction. What architecture would you propose?

Scenario Solution:

  • Reusable Workflows: Create a centralized .github repository containing reusable workflow templates for build, test, Docker image construction, and Helm deployments.
  • Custom Composite Actions: Abstract complex steps (e.g., SonarQube scanning, Trivy vulnerability checks, ECR authentication) into reusable internal composite actions.
  • Self-Hosted Runner Groups: Provision ephemeral self-hosted runners on AWS EKS using KEDA (Kubernetes Event-driven Autoscaling) or Actions Runner Controller (ARC) to auto-scale runners based on pending job queues.

Q4: Terraform state file gets locked by a failed CI/CD run, and a critical hotfix needs to be deployed immediately. How do you resolve this safely?

Scenario Solution:

  1. Verify no active CI/CD execution is writing to the state backend (e.g., AWS S3 + DynamoDB state locking).
  2. Retrieve the Lock ID from the Terraform error trace.
  3. Run terraform force-unlock <LOCK-ID> with authorization from the Lead Architect.
  4. Run terraform plan immediately to inspect drift before applying the hotfix.
  5. Update CI/CD steps to use explicit pipeline timeouts (timeout-minutes: 15) to prevent hanging jobs from keeping state locks indefinitely.

Q5: How do you secure database passwords and API tokens when deploying microservices across multiple Kubernetes clusters using GitOps (ArgoCD)?

Scenario Solution:

  • Never commit raw secrets to Git.
  • Use External Secrets Operator (ESO) or Sealed Secrets.
  • Store sensitive data centrally in AWS Secrets Manager, HashiCorp Vault, or GCP Secret Manager.
  • Install ESO in the cluster to sync secrets directly from Vault/AWS Secrets Manager into Kubernetes native Secret objects dynamically at runtime.

Q6: Your ingress controller is dropping connections during peak traffic spikes (100k RPM). What components do you check?

Scenario Solution:

  1. Linux Kernel Limits: Increase worker connection limits and sysctl net sockets (net.core.somaxconn = 65535, net.ipv4.tcp_max_syn_backlog = 65535).
  2. Nginx/Envoy Ingress Adjustments: Tune worker-connections, upstream-keepalive-connections, and HTTP keep-alive timeouts.
  3. Pod Disruption & HPA: Set proper Pod Disruption Budgets (PDB) and configure HPA to scale Ingress replicas based on CPU and Request-Per-Second (RPS) metrics.

Q7: How do you achieve Zero-Downtime database migrations when schema changes (e.g., column rename) break running application pods?

Scenario Solution: Use the Expand and Contract (Parallel Run) Pattern:

  1. Phase 1 (Expand): Add the new column (new_email) without deleting the old one (old_email).
  2. Phase 2 (Dual Write): Update application code to write to both columns and read from new_email with fallback to old_email. Deploy.
  3. Phase 3 (Backfill): Run a background script to migrate legacy data from old_email to new_email.
  4. Phase 4 (Contract): Remove dual-write logic, point application exclusively to new_email, and drop old_email in a subsequent release.

Q8: A Jenkins master instance is running out of disk space due to build logs and artifact storage. How do you prevent this architecturally?

Scenario Solution:

  • Offload Artifacts: Configure Jenkins plugins to upload artifacts directly to Amazon S3 or Nexus/JFrog Artifactory during build completion.
  • Log Rotation Policies: Enforce Discard Old Builds in Jenkinsfiles (logRotator(numToKeepStr: '15', artifactNumToKeepStr: '5')).
  • Ephemeral Agents: Run Jenkins execution nodes exclusively inside Kubernetes pods so agents are destroyed completely post-job execution.

Q9: How do you trace a latency bottleneck across 10+ microservices communicating via gRPC and REST APIs?

Scenario Solution:

  • Implement OpenTelemetry (OTel) distributed tracing instrumentation across all microservices.
  • Propagate context using W3C traceparent headers across incoming REST and outgoing gRPC metadata.
  • Ship trace payloads to Jaeger, Tempo, or Datadog.
  • Filter by root trace span to identify which database query, network call, or downstream service is causing the high P99 latency tail.

Q10: Docker build times in CI/CD take 15+ minutes because of heavy dependency downloads. How do you optimize them under 2 minutes?

Scenario Solution:

  1. Multi-Stage Builds: Separate build dependencies from runtime binaries.
  2. Layer Caching: Reorder Dockerfile instructionsโ€”copy dependency manifests (package.json, requirements.txt, go.mod) and install dependencies before copying application source code.
  3. BuildKit Remote Cache: Use docker buildx with --cache-from type=registry and --cache-to type=inline to share layer caches across CI runners.

๐Ÿค– Section 2: Advanced MLOps & AI Infrastructure Scenarios

Q11: An image classification model deployed on Kubernetes shows 95% accuracy in staging, but performance degrades heavily in production over 3 weeks. How do you detect and handle this?

Scenario Solution: This is caused by Data Drift (input distribution change) or Concept Drift (relationship between inputs and labels change).

Python

# Detecting Data Drift using Evidently AI in Python
from evidently.report import Report
from evidently.metric_preset import DataDriftPreset

def check_input_drift(reference_data, current_production_data):
    drift_report = Report(metrics=[DataDriftPreset()])
    drift_report.run(reference_data=reference_data, current_data=current_production_data)
    
    report_json = drift_report.json()
    if report_json['metrics'][0]['result']['dataset_drift']:
        trigger_ml_pipeline_retraining()

def trigger_ml_pipeline_retraining():
    print("Drift detected above threshold! Triggering Airflow/Kubeflow DAG...")
  • Resolution:
    1. Setup continuous data quality monitoring via Evidently AI or WhyLogs.
    2. Emit custom metrics to Prometheus/Grafana.
    3. Automatically trigger retraining pipelines in Kubeflow or Airflow when Statistical Kolmogorov-Smirnov (KS) test p-values drop below threshold.

Q12: Your team needs to fine-tune an LLM (e.g., Llama-3) using QLoRA across multiple GPUs. How do you design the MLOps training and deployment pipeline?

Scenario Solution:

  1. Distributed Training: Use DeepSpeed + PyTorch FSDP (Fully Sharded Data Parallel) with QLoRA 4-bit quantization to fit model weights across multiple AWS p4d.24xlarge (A100) or g5 nodes.
  2. Experiment Tracking: Use MLflow or Weights & Biases to log hyperparameter sweeps (rank $r$, lora_alpha, learning rate), memory usage, and loss metrics.
  3. Model Registry: Save adapter weights (LoRA weights) into MLflow Model Registry.
  4. Serving Infrastructure: Deploy base model + LoRA adapters using vLLM or TGI (Text Generation Inference) on Kubernetes with GPU operator acceleration to allow fast dynamic adapter swapping.

Q13: A production ML API experiences high latency spikes during peak usage because GPU inference is slow. How do you optimize throughput without doubling hardware costs?

Scenario Solution:

  1. Dynamic Batching: Configure Triton Inference Server or vLLM to batch incoming concurrent requests dynamically within a small time window (e.g., 5ms window).
  2. Model Optimization: Convert model weights to TensorRT-LLM, ONNX Runtime, or OpenVINO to run fused kernel operations.
  3. Quantization: Quantize FP32/FP16 models down to INT8 or INT4 precision using AWQ or GPTQ techniques.
  4. Auto-scaling: Scale GPU pods using KEDA based on pending request queue length (http-request-queue-depth) rather than raw GPU memory/CPU utilisation.

Q14: How do you build an automated CI/CD pipeline for ML models (CT – Continuous Training)?

Scenario Solution:

[ New Data Ingestion ] โ”€โ”€> [ Data Validation (Great Expectations) ]
                                    โ”‚
                                    โ–ผ
[ Trigger Retraining ] โ”€โ”€> [ Model Evaluation vs Champion Model ]
                                    โ”‚
                       (If Evaluation Score > Champion)
                                    โ”‚
                                    โ–ผ
[ Register Model in MLflow ] โ”€โ”€> [ Deploy Canary Ingress to K8s ]
  • Automated Workflow:
    1. Trigger: Webhook or scheduled Airflow DAG when new data threshold is reached.
    2. Validation: Run Great Expectations checks on feature store data.
    3. Evaluation Gate: Retrain candidate model (Challenger) and compare its F1/RMSE score against the existing production model (Champion).
    4. Deployment: If Challenger beats Champion by > 2%, push model to MLflow Model Registry with Staging tag, build container, and deploy to Kubernetes staging environment automatically.

Q15: How do you handle Vector Database schema migrations and re-indexing (e.g., Qdrant, Milvus, Pinecone) in a live RAG system with zero downtime?

Scenario Solution:

  1. Blue-Green Collection Pattern: Create a new Vector Index / Collection (collection_v2) alongside the active one (collection_v1).
  2. Background Embedding Generation: Run batch ingestion workers using Ray or Spark to populate collection_v2 with updated embedding models or chunking strategies.
  3. Dual Ingestion: Route newly arriving real-time data to both v1 and v2.
  4. Router Swap: Update your API gateway or application configuration layer to point semantic search queries from v1 to v2.
  5. Decommission: Verify search performance metrics and delete collection_v1.

Q16: How do you prevent data leakage between Training and Evaluation sets in automated feature store pipelines?

Scenario Solution:

  • Use an Enterprise Feature Store (e.g., Feast or Hopsworks) with Time-Travel / Point-in-Time Joins.
  • Ensure feature retrieval for model training uses accurate historical timestamps (event_timestamp).
  • This prevents future feature states from bleeding into past training records (Lookahead Bias).

Q17: Large Language Model (LLM) responses in production are hallucinating and giving sensitive details. How do you implement real-time Guardrails in MLOps?

Scenario Solution: Implement a Proxy Guardrail Layer (e.g., NeMo Guardrails, Guardrails AI, or Llama Guard) in front of the LLM pipeline:

[ User Prompt ] โ”€โ”€> [ Input Guardrails (PII Detection / Prompt Injection) ]
                                  โ”‚
                                  โ–ผ
                          [ LLM Inference ]
                                  โ”‚
                                  โ–ผ
[ Output Response ] <โ”€โ”€ [ Output Guardrails (Hallucination Check / Toxicity) ]
  • Input Checks: Reject prompt injection attacks, system prompt leaks, and redact PII (Personally Identifiable Information).
  • Output Checks: Compare generated answer against context sources in RAG systems; fallback to generic safe answer if context grounding score is low.

Q18: How do you monitor cost and token usage for multi-tenant LLM applications across different client organizations?

Scenario Solution:

  • Observability Tooling: Integrate LLM observability solutions such as LangSmith, Langfuse, or OpenLiteLLM / OpenTelemetry.
  • Metadata Tracking: Attach tenant_id, user_id, and use_case tags to every API request header.
  • Aggregated Billing: Export token consumption metrics (prompt tokens + completion tokens) to ClickHouse or Prometheus, visualizing tenant-wise costs on Grafana dashboards.
  • Rate-Limiting: Enforce dynamic token quota rate limits per tenant using Redis token bucket counters.

Q19: An MLOps engineer leaves, and no one knows how a production model running in Kubernetes was trained. How do you enforce ML Reproducibility moving forward?

Scenario Solution: Enforce strict compliance using the MLOps Lineage Rule:

  1. Code Versioning: Enforce Git commit hash tagging for every model run.
  2. Data Versioning: Use DVC (Data Version Control) or LakeFS to track precise dataset snapshots used during training.
  3. Environment Lock: Build containerized training environments using Docker with locked base images and explicit requirements.txt / Poetry lockfiles.
  4. Metadata Logging: Automate MLflow artifact logging to capture Python runtime, environment flags, seed parameters, dataset hashes, and Git commits automatically:

Python

import mlflow

mlflow.autolog(log_input_examples=True, log_model_signatures=True)
with mlflow.start_run():
    mlflow.set_tag("git_commit", os.getenv("GIT_COMMIT_HASH"))
    mlflow.set_tag("dvc_data_version", os.getenv("DVC_DATA_VERSION"))
    # Training logic executed here

Q20: Your company needs to run AI Workloads (PyTorch model training) on AWS EKS using Spot Instances without job failures. How do you build this fault-tolerant pipeline?

Scenario Solution:

  1. Node Termination Handling: Deploy aws-node-termination-handler on the cluster to catch 2-minute Spot Interruption Notifications from AWS.
  2. Checkpointing: Instrument PyTorch code to save training state checkpoints to shared storage (AWS EFS or S3) every $N$ steps/epochs.
  3. K8s Tolerations & Taints: Use Node Affinity and Interruption Tolerations so training pods schedule gracefully across multiple Spot Instance pools.
  4. Automatic Resumption: Configure training jobs as Kubernetes Jobs with backoffLimit and auto-resumption scripts that load the latest checkpoint from S3 upon pod restart on a new Spot node.

๐Ÿ“Œ Summary Checklist for Tech Interviews

ConceptKey Tool / Solution
Kubernetes OOMKilledMetric Analysis, Adjust Requests/Limits, VPA/HPA
Zero Downtime CanaryArgo Rollouts + Prometheus Metrics
Secrets ManagementExternal Secrets Operator + AWS Secrets Manager / Vault
Data/Model DriftEvidently AI, WhyLogs, Airflow Retraining DAGs
LLM Serving & Fine-TuningQLoRA, DeepSpeed, vLLM, Triton Inference Server
LLM ObservabilityOpenTelemetry, Langfuse, LangSmith
ML ReproducibilityDVC + MLflow + Locked Docker Envs

๐Ÿ’ก Boost Your Technical Career

Are you preparing for upcoming DevOps, MLOps, or Cloud Architecture interviews? Explore more role-specific roadmaps, interview guides, and hands-on system design challenges on CareerGraph!

Leave a Reply

Your email address will not be published. Required fields are marked *