ARCANATogether AI
Together AI provides fast inference for open-source models with competitive pricing. They host a wide range of models including Llama, Mixtral, and support fine-tuned model deployment.
Setup
# 1. Get your API key from https://api.together.xyz/settings/api-keys
export TOGETHER_API_KEY=...
# 2. Verify detection
arcana doctor
# 3. Run
arcana run "hello"Available Models
| Model | Parameters | Best For |
|---|---|---|
meta-llama/Llama-3.3-70B-Instruct-Turbo | 70B | Best quality, fast inference |
meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo | 8B | Fast, general purpose |
meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo | 405B | Highest quality open model |
mistralai/Mixtral-8x22B-Instruct-v0.1 | 8x22B | Mixture of experts, versatile |
mistralai/Mistral-7B-Instruct-v0.3 | 7B | Fast, instruction following |
deepseek-ai/DeepSeek-R1 | 671B MoE | Reasoning, math, code |
Qwen/Qwen2.5-72B-Instruct-Turbo | 72B | Multilingual, coding |
See the full list at api.together.xyz/models.
Config File
{
"provider": "together",
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo"
}Environment Variables
| Variable | Description |
|---|---|
TOGETHER_API_KEY | Your Together AI API key (required) |
OpenAI-Compatible API
Together AI is OpenAI-compatible. You can also use it via OpenAI's client by setting the base URL:
export OPENAI_API_KEY=$TOGETHER_API_KEY
export OPENAI_BASE_URL=https://api.together.xyz/v1
export ARCANA_MODEL=meta-llama/Llama-3.3-70B-Instruct-TurboPricing Notes
Together AI charges per-token. Approximate costs per 1M tokens:
| Model | Input | Output |
|---|---|---|
| Llama 3.3 70B | $0.88 | $0.88 |
| Llama 3.1 405B | $3.50 | $3.50 |
| Llama 3.1 8B | $0.10 | $0.10 |
| Mixtral 8x22B | $1.20 | $1.20 |
| DeepSeek R1 | $3.00 | $7.00 |
Free tier: $1 credit for new accounts. Check together.ai/pricing for current rates.
Fine-Tuned Models
Together AI supports deploying your own fine-tuned models. After fine-tuning, you can reference your model directly:
# Use your fine-tuned model
export ARCANA_MODEL=your-org/your-fine-tuned-model
# Or in config
# { "provider": "together", "model": "your-org/your-fine-tuned-model" }Tips
- The
Turbovariants of models are optimized for faster inference - Together AI is OpenAI-compatible — most OpenAI tools work out of the box
- Great choice for open-source model inference at scale
- Their pricing is very competitive, especially for smaller models
- Supports both streaming and non-streaming responses
Limitations
- Only hosts open-source models (no GPT, Claude, or Gemini)
- Fine-tuning has additional costs beyond inference
- Some models may have longer cold-start times
- Rate limits vary by plan (free tier: 200 RPM)