Connect a Spark device
Use the DGX and RTX-class Spark devices in your office as your team's AI inference server.
Spark is a GPU device kept in the office. Use the Keural desktop app to connect a Spark and install models, and use it as a shared inference server for your team. NVIDIA DGX Spark and other devices that run Linux ARM64 connect as servers on the same network. RTX-class Spark PCs are used with local models in that PC's desktop app.
Find the device
Open Settings → Servers. The app automatically finds Spark devices on the same network. If the connection drops, the app reconnects automatically.
Check the device and install models
When connected, the app checks the GPU, memory, and disk and recommends suitable models. Choose vLLM or llama.cpp as the inference engine. The app installs the models and the engine.
Start using it
Models running on the Spark are shown in the Remote group of the level menu in the input box. Check the model status in the Remote group under Settings → Models.
Spark Drive
When you select Remote (the connected Spark) as the storage location for Drive, files are saved to the connected Spark instead of the cloud. You can ask questions about them in the same way. If you use a cloud model in a conversation, your questions and the referenced content are sent to that model. To keep materials from being sent outside your office, select a Spark model.
Requirements
- Keural desktop app (macOS or Windows), signed in
- A Spark device on the same network as the app
- macOS: allow the Local Network permission the first time you connect.