- Home
- Model Hosting
- KServe
On the recordRead 5 Oct 202620 facts
KServe
On the record
- Deployment API
- KServe provides a Kubernetes custom resource definition that handles autoscaling, networking, health checks, and server configuration for model deployments.kserve.github.io · read 5 Oct 2026
- Frameworks
- Documented serving runtimes include TensorFlow Serving, Triton, Hugging Face, LightGBM, XGBoost, AutoGluon, PMML, scikit-learn, and PaddlePaddle.kserve.github.io · read 5 Oct 2026
- Installation
- KServe documents installation with Kustomize components or Helm charts and supports multiple deployment modes.kserve.github.io · read 5 Oct 2026
- Install options
- KServe can be installed with Kustomize components or Helm charts, and can serve predictive models alone or alongside separate LLM and model-caching components.kserve.github.io · read 5 Oct 2026
- Integrations
- Documented integrations include Istio, Knative Serving, Envoy Gateway, Envoy AI Gateway, and the vLLM runtime.kserve.github.io · read 5 Oct 2026
- Known limitation
- The optional LocalModel component currently supports InferenceService workloads; support for LLMInferenceService is planned for future releases.kserve.github.io · read 5 Oct 2026
- LLM API
- The Hugging Face runtime uses vLLM and provides OpenAI-compatible API endpoints for generative tasks including text generation, embeddings, and reranking.kserve.github.io · read 5 Oct 2026
- LLM features
- Its generative AI features include vLLM and llm-d backends, OpenAI protocol compatibility, GPU acceleration, model caching, KV cache offloading, and request-based autoscaling.kserve.github.io · read 5 Oct 2026
- Predictive AI features
- It supports TensorFlow, PyTorch, scikit-learn, XGBoost, and ONNX, with routing, InferenceGraph deployments, explainability, monitoring, and scale-to-zero.kserve.github.io · read 5 Oct 2026
- Predictive model features
- Predictive serving supports canary rollouts, inference pipelines and ensembles, scale-to-zero, explainability, and monitoring features such as payload logging and drift detection.kserve.github.io · read 5 Oct 2026
- Project affiliation
- The site says KServe has joined CNCF and identifies it as a Series of LF Projects, LLC project.kserve.github.io · read 5 Oct 2026
- Protocols
- KServe documents gRPC support for its V1 and V2 inference protocols, including binary tensor data and streaming responses.kserve.github.io · read 5 Oct 2026
- Purpose
- KServe is a Kubernetes platform for serving generative and predictive AI models across multiple frameworks.kserve.github.io · read 5 Oct 2026
- Requirements and limits
- KServe runs on Kubernetes; its optional LocalModel component currently supports InferenceService workloads, while support for LLMInferenceService is described as planned for a future release.kserve.github.io · read 5 Oct 2026
- Rollouts
- KServe supports canary rollouts and A/B testing.kserve.github.io · read 5 Oct 2026
- Security
- The Istio integration documents mutual TLS, automatic certificate management, and fine-grained access-control and authorization policies.kserve.github.io · read 5 Oct 2026
- Standard API
- It provides a Kubernetes Custom Resource Definition that abstracts autoscaling, networking, health checks, and server configuration.kserve.github.io · read 5 Oct 2026
- Support
- Users can get help through the documentation, the #kserve Slack channel, GitHub Discussions, and community meetings.kserve.github.io · read 5 Oct 2026
