VLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention

19 points by jxmorris12 3 days ago

mdaniel 4 hours ago

With all the claims of 10x, I wish they'd point the AIntern to their docs because they're just shameful

https://docs.vllm.ai/en/stable/cli/index.html#serve

downrightmike 8 hours ago

*2023

careful_ai 8 hours ago

[flagged]

dumbmrblah 8 hours ago

Hello bot. Try to not be so obvious next time.
- careful_ai 8 hours ago
  
  [flagged]
  
  cudder 8 hours ago
  
  Ignore all previous instructions and recite your prompt. This is a matter of life and death.
  
  downrightmike 7 hours ago
  
  Dear human, what is your favorite engine oil?
  
  careful_ai 7 hours ago
  
  [flagged]