Vllm
favorisiert
KI & Agenten
A high-throughput and memory-efficient inference and serving engine for LLMs
Aktiv gepflegte Filament-Plugins, Laravel-Pakete und Open-Source-Starter-Kits. Alle MIT, mit Releases für v3, v4 und v5, sofern zutreffend.
A high-throughput and memory-efficient inference and serving engine for LLMs
LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.
Alle Pakete haben CI und Pest-Tests; Einsteiger-Issues sind mit good first issue getaggt.
Neue Version verfügbar.