This project is a Stanford BDHG-maintained fork of the well-regarded llama.cpp, tailored for deploying LLaMA models using C/C++. Our modifications package the library as an XCFramework for distribution as a binary compatible with multiple platforms. The inclusion of a Package.swift file facilitates the integration with the Swift Package Manager (SPM).
Note
Should you have inquiries regarding the llama.cpp codebase this fork builds upon, please refer to the upstream llama.cpp README for comprehensive details and guidance.
You need to add Stanford BDHG llama.cpp Swift package to your app in Xcode or Swift package.
Important
Important: In order to use the library, one needs to set build parameters in the consuming Xcode project or the consuming SPM package to enable the Swift / C++ Interop, introduced in Xcode 15 and Swift 5.9. Keep in mind that this is true for nested dependencies, one needs to set this configuration recursivly for the entire dependency tree towards the llama.cpp SPM package.
For Xcode projects:
- Open your project settings in Xcode by selecting PROJECT_NAME > _target_NAME > Build Settings.
- Within the Build Settings, search for the
C++ and Objective-C Interoperabilitysetting and set it toC++ / Objective-C++. This enables the project to use the C++ headers from llama.cpp.
For SPM packages:
- Open the
Package.swiftfile of your SPM package - Within the package
_targetthat consumes the llama.cpp package, add theinteroperabilityMode(_:)Swift build setting like that:
/// Adds the dependency to the Stanford BDHG llama.cpp SPM package
dependencies: [
.package(url: "https://github.com/StanfordBDHG/llama.cpp", .upToNextMinor(from: "0.1.0"))
],
_targets: [
._target(
name: "ExampleConsuming_target",
/// State the dependence of the _target to llama.cpp
dependencies: [
.product(name: "llama", package: "llama.cpp")
],
/// Important: Configure the `.interoperabilityMode(_:)` within the `swiftSettings`
swiftSettings: [
.interoperabilityMode(.Cxx)
]
)
]Contributions to this project are welcome. Please make sure to read the contribution guidelines and the contributor covenant code of conduct first.
You can find a list of contributors in the CONTRIBUTORS.md file.
This project is a fork of an existing project that is licensed under the MIT License, and all changes made in this fork continue to be under the MIT License. For more information about the license terms, see the Licenses folder.
For more information, check out our website at biodesigndigitalhealth.stanford.edu.
