Tev1-0.8B-experimental

A 873M dense checkpoint from togethercomputer only earns a slot on disk if Q4 at a usable context fits the card you already own. Tev1-0.8B-experimental local is the listing for that decision, not a cloud demo.

I care about what the official card actually promises, whether Tev1-0.8B-experimental holds a usable quant and context on your GPU, and where it tends to disappoint once the weights are on disk.

What Tev1-0.8B-experimental is

The weights live at togethercomputer/Tev1-0.8B-experimental on Hugging Face. That repo is the source of truth for license (unknown), config, tokenizer, and file names. This directory does not host the blobs.

Architecture is dense. Stored parameters decide disk and load size. If this is a mixture-of-experts row, active parameters decide tokens per second; do not mix those two numbers.

From the official card, trimmed: --- base model: Qwen/Qwen3.5-0.8B pipeline tag: text-generation library name: transformers tags: - together-ai - qwen3.5 - decision-model - experimental --- Tev1-0.8B-experimental Tev1-0.8B-experimental is an experimental 0.8B decision model from Together AI. It is a supervised fine-tune of Qwen3.5-0.8B trained to choose one option from a structured state, question, and list of choices. This is a Jev-inspired experiment, not a non-autoregressive Jev runtime.

It retains Qwen’s standard next-token language-model head. Resources - Learn how to train your own classifier for $17: https://www.together.ai/blog/how-to-train-your-own-jev - Full data recipe & code that we used to train Tev1: https://github.com/togethercomputer/tev1 Intended interface Provide a system instruction followed by a structured decision containing state , question , and 2–24 labeled options . The model should return exact

Context, license, and what the card actually promises

Context is advertised at 4,096 tokens. That is a model limit, not a VRAM promise. A long window on a large Q4 load will not fit a 12 GB card even if the architecture allows the window.

License unknown decides whether you can ship a product around the weights. Read the card. A research-only clause is not the same as Apache-2.0.

Release cadence on Hugging Face is noisy. Quant dumps, FP8 replicas, and unofficial merges are not this listing. This row tracks the weights repo, not every community file.

If you came here from a “best local model” list, remember those lists mix 8 GB toys with 70B workstation models. Filter by the GPU you have, then read this page.

Will it run on your GPU

VRAM fit here comes from parameter count, quant, context, and KV cache — not from a leaderboard score. If the VRAM calculator says it does not fit, believe the calculator before a social-media screenshot.

Check the repo siblings for GGUF. If none exist, convert from safetensors or wait for a community quant.

Use Q4_K_M as the default local quant. Go Q5 or Q6 only after you have headroom. Go Q3 or IQ2 only if you accept more quality loss to stay on a small card.

When you first load Tev1-0.8B-experimental, watch nvidia-smi or Activity Monitor. Shared GPU memory climbing is the same as a failed fit, even if the process stays alive.

Tokens per second track memory bandwidth. A 4090 and a 3060 12 GB can both load a 7B–8B Q4; they will not feel the same at 8k context.

Tev1-0.8B-experimental on a local stack is a hardware decision first. If the card cannot hold the quant you want, pick a smaller sibling or a higher compression — do not keep retrying the same file.

How people actually use it

For chat, keep a short system prompt and a fixed sampler. Changing temperature every test makes local A/B comparisons useless.

For coding, prefer a GGUF that matches the instruct template the authors shipped. A base model with a chat wrapper bolted on will look worse than the same weights with the right template.

For RAG, context window is not free. Retrieval dumps that fill 16k tokens will evict the system prompt on a card that only holds 8k comfortably.

Do not treat missing composite scores as missing capability. Scores appear after two sourced benchmarks with URLs. Until then the listing still has VRAM math and the official card.

If you later add sourced MMLU, LiveCodeBench, or similar numbers, they show in the benchmark table with links. Until a URL exists, this page will not invent a score.

Keep the how-to page for install steps. This deep dive is for whether Tev1-0.8B-experimental belongs on your machine and what you should expect once it does.

If you still need a shortlist by GPU size, start from What can I run and filter by the card you already own before opening another listing.

What to watch for

Sampler defaults differ across UIs. A “creative” preset in one frontend can look like a broken model next to llama.cpp defaults. Lock temperature, top-p, and the chat template before you compare.

Thinking-mode checkpoints (when the card documents them) spend tokens on a hidden chain of thought. That is extra VRAM and latency, not free quality. Turn the mode off if you only wanted a fast chat reply.

Unofficial quants can ship the wrong tokenizer or a stripped chat template. If replies are garbage after a clean load, check the template before you blame the GPU.

Long-context claims on the card still have to fit KV cache. YaRN or similar extensions do not magically add VRAM. Drop context first when the process dies at 16k.

Driver and runtime updates change tokens per second more often than weight revisions do. Re-run one fixed prompt after a CUDA or llama.cpp bump.

If you serve Tev1-0.8B-experimental on a LAN, treat it like any other unauthenticated app: private bind, no port-forward, auth if more than one person can reach it.

This listing will not rank Tev1-0.8B-experimental against closed APIs. Local means the weights are on disk and the meter is your wall power, not a vendor invoice.