Vllm
favorisiert
Anwendung
A high-throughput and memory-efficient inference and serving engine for LLMs
Aktiv gepflegte Filament-Plugins, Laravel-Pakete und Open-Source-Starter-Kits. Alle MIT, mit Releases für v3, v4 und v5, sofern zutreffend.
A high-throughput and memory-efficient inference and serving engine for LLMs
LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.
Alle Pakete haben CI, Pest-Tests und offene Issues mit dem Tag good first issue.
Neue Version verfügbar.