Discussions

• 41 You must be logged in to vote

  🙌

  Performance of llama.cpp on Apple Silicon M-series

  performance Speed related topics need feedback Testing and feedback with results are needed

  ggerganov started Nov 22, 2023 in Show and tell

  

  

  

  

  160
• 1 You must be logged in to vote

  💡

  need summarize all(all videocards) gpu memory count when rpc

  lexasub started Jan 11, 2025 in Ideas

  

  

  1
• 1 You must be logged in to vote

  🙏

  No module named 'termios'

  sandipkatel asked Jan 11, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  💬

  Performance of llama.cpp with Vulkan

  performance Speed related topics need feedback Testing and feedback with results are needed

  netrunnereve started Dec 18, 2024 in General

  

  

  

  

  29
• 1 You must be logged in to vote

  🙏

  chatml-function-calling vs normal response

  albertusgeyser asked Jan 11, 2025 in Q&A · Unanswered

  

  

  0
• 2 You must be logged in to vote

  🙏

  How to cache system prompt?

  arpitjjw asked Aug 9, 2024 in Q&A · Answered

  

  

  

  

  16
• 1 You must be logged in to vote

  🙏

  how i convert this nomic model

  kalle07 asked Jan 9, 2025 in Q&A · Unanswered

  

  

  4
• 6 You must be logged in to vote

  💬

  [SYCL][Intel GPU] Long Term Features & Issues Tracking

  Intel GPU

  airMeng started Feb 2, 2024 in General

  

  

  

  

  124
• 1 You must be logged in to vote

  🙏

  Llamacpp compile failed on Jetson Orin Nano (8GB)

  wsmzsl asked Nov 27, 2024 in Q&A · Unanswered

  

  

  

  

  9
• 2 You must be logged in to vote

  💬

  The retail prices of the RTX 4090-48G and RTX 4080 Super-32G have been released

  bobqianic started Aug 26, 2024 in General

  

  

  

  

  2
• 2 You must be logged in to vote

  🙏

  Running on top of MACbook 2019 (Intel i9 + AMD Radeon Pro 5500)

  mineirinh0 asked Jul 5, 2023 in Q&A · Unanswered

  

  

  

  

  12
• 1 You must be logged in to vote

  🙏

  How to build openblas and blis in windows?

  FNsi asked Jan 9, 2025 in Q&A · Answered

  

  

  1
• 1 You must be logged in to vote

  🙏

  use llama.cpp quantization with QAT finetune

  anyinlover asked Jan 10, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  💡

  Support for Armnn?

  FNsi started Jan 10, 2025 in Ideas

  

  

  0
• 6 You must be logged in to vote

  💡

  Support Phi 3.5 MoE

  Dampfinchen started Aug 21, 2024 in Ideas

  

  

  

  

  4
• 3 You must be logged in to vote

  💡

  Phi-3.5-MoE

  ayttop started Sep 5, 2024 in Ideas

  

  

  1
• 1 You must be logged in to vote

  💡

  Support Stella-EN 1.5B v5 pre-tokenizer

  shermanhuman started Jan 9, 2025 in Ideas

  

  

  0
• 1 You must be logged in to vote

  💬

  Why do vision models need specific surgery files?

  bartowski1182 started Jan 8, 2025 in General

  

  

  

  

  7
• 1 You must be logged in to vote

  🙏

  Is there a way to get the Physical Batch Size from the REST API?

  teleprint-me asked Jan 8, 2025 in Q&A · Unanswered

  

  

  2
• 1 You must be logged in to vote

  💡

  Using different machines for model and draft model across LAN? Chaining draft models?

  itch- started Jan 9, 2025 in Ideas

  

  

  0
• 1 You must be logged in to vote

  🙏

  Want to ask how to start working with you

  Hacker1one asked Jan 8, 2025 in Q&A · Unanswered

  

  

  0
• 2 You must be logged in to vote

  🙏

  CPU Performance

  gurbindersingh asked Sep 14, 2023 in Q&A · Closed · Unanswered

  

  

  

  

  8
• 1 You must be logged in to vote

  🙏

  Problems running llama.cpp vulkan (linux) on LM Studio along with LLM greater than 38GB in size or about 55B parameters and higher

  erickwill asked Jan 3, 2025 in Q&A · Answered

  

  

  11
• 14 You must be logged in to vote

  🙌

  Performance of llama.cpp on Apple Silicon A-series

  ggerganov started Dec 17, 2023 in Show and tell

  

  

  

  

  41
• 3 You must be logged in to vote

  💬

  How to Add Grammar/Json Mode Sampling with llama_sampler_chain_add?

  niral28 started Jan 7, 2025 in General

  

  

  3