II.
Specialization overview
Reference · livespecialization:ml-inference-serving
ML Inference Serving overview
Serving ML / LLM inference at scale — vLLM, TGI, TensorRT-LLM, Triton, batching, KV-cache management, and GPU autoscaling.
Attributes
displayName
ML Inference Serving
parentDomainId
description
Serving ML / LLM inference at scale — vLLM, TGI, TensorRT-LLM,
Triton, batching, KV-cache management, and GPU autoscaling.
Outgoing edges
specializes1
- domain:ml-ops·DomainMLOps
uses_tool4
- tool:vllm·ToolvLLM
- tool:triton-inference·ToolTriton Inference Server
- tool:onnx-runtime·ToolONNX Runtime
- tool:bentoml·ToolBentoML
Incoming edges
applies_to7
- skill-area:model-serving-deployment·SkillAreaModel Serving and Deployment
- skill-area:model-serving-operations·SkillAreaModel Serving
- skill-area:model-robustness-testing·SkillAreaModel Robustness Testing
- skill-area:inference-performance-testing·SkillAreaInference Performance Testing
- skill-area:model-monitoring-drift-detection·SkillAreaModel Monitoring and Drift Detection
- skill-area:inference-optimization·SkillAreaInference Optimization
- skill-area:model-compression·SkillAreaModel Compression
contains3
- domain:machine-learning·DomainMachine Learning
- domain:ml-ops·DomainMLOps
- domain:ml-ai·DomainML/AI
lib_belongs_to_specialization1
- lib-skill:data-science-ml--inference-performance-testing·LibrarySkillinference-performance-testing