Please confirm you are human
This browser or connection looks automated. Press and continuously hold the control for 3 seconds to enable Google-hosted web results and, when separately allowed, AI-assisted answers.
A successful check enables 100 search requests. Interactive access does not authorize scraping, systematic collection, or reuse of search output.
News
fault_tolerance
6+ hour, 11+ min ago (60+ words) Manages fault tolerance state for a single engine core. Push current health to the client so it can refresh its cache. Reinit DP process group if in DP mode. Returns worker params. Dispatch an FT command by instruction name. Called…...
engine_core_sentinel
8+ hour, 22+ min ago (60+ words) Manages fault tolerance state for a single engine core. Push current health to the client so it can refresh its cache. Reinit DP process group if in DP mode. Returns worker params. Dispatch an FT command by instruction name. Called…...
startup
8+ hour, 28+ min ago (1278+ words) When passing JSON CLI arguments, the following sets of arguments are equivalent: Additionally, list elements can be passed individually using +: Configuration for the model. To swap the Rust BPE backend that powers HF fast tokenizers for the fastokens implementation, set…...
mxfp8
3+ mon, 5+ day ago (54+ words) vLLM Online MXFP8 (microscaling FP8, block-32) quantization methods. Online MXFP8 linear method. MoE method for online MXFP8 (block) quantization. Online MXFP8 linear method. Loads bf16/fp16 checkpoints and quantizes weights to MXFP8 (microscaling FP8 with block-32 scales) during weight loading. Batch quantization: bf16/fp16 weights -> MXFP8 (fp8 + uint8 scales)....
OpenAI Chat Completion With Reasoning Streaming
2+ mon, 4+ week ago (13+ words) vLLM OpenAI Chat Completion With Reasoning Streaming¶...
glm4_moe
8+ mon, 2+ week ago (15+ words) vLLM Inference-only GLM-4.5, GLM-4.6, GLM-4.7 model compatible with HuggingFace weights....
mori
3+ mon, 2+ day ago (71+ words) vLLM Prepare/Finalize using MoRI kernels. Returns a tuple of: prepare(a1, topk_weights, topk_ids, num_experts, expert_map, apply_router_weight_on_input, quant_config, defer_input_quant=False) ¶ Returns a tuple of: - quantized + dispatched a. - Optional quantized + dispatched a1_scales. - Optional ExpertTokensMetadata containing gpu/cpu tensors as big as the number of local experts with the information about…...
qwen3_asr_realtime
5+ mon, 5+ day ago (29+ words) vLLM Inference-only Qwen3-ASR realtime model. Audio buffer for Qwen3-ASR realtime streaming. Accumulates audio samples and yields segments when enough audio has been buffered for processing....
utils
8+ mon, 2+ week ago (23+ words) vLLM Utility methods for model layers. Applies penalties in place to the logits tensor apply_penalties(logits, prompt_tokens_tensor, output_tokens_tensor, presence_penalties, frequency_penalties, repetition_penalties) ¶...
selective_state_update_replayssm_output_only
20+ hour, 51+ min ago (26+ words) vLLM selective_state_update_replayssm_output_only(state, x, dt, A, B, C, D=None, dt_bias=None, z=None, dt_softplus=False, x_cache=None, dt_cache=None, B_cache=None, bc_pre=None, write_pos=None, is_flush=None, max_cache_len=16, state_batch_indices=None, null_block_id=NULL_BLOCK_ID, out=None, enable_stochastic_rounding=False, cache_philox_rounds=0) ¶...