Thinking Machines released Inkling-Small just fourteen days after launching its original 975-billion-parameter flagship, Inkling. Led by former OpenAI CTO Mira Murati, the startup didn't shrink its model to run on smartphones or local consumer laptops. Instead, it targeted the pragmatic reality facing enterprise engineering teams: running nearly trillion-parameter models in production drains infrastructure budgets and hogs rack space faster than most companies can scale their GPU clusters.
For enterprise teams, raw capability score isn't the only metric that matters. Hardware footprint, memory bandwidth, inference latency, and operational control often outweigh minor gains on artificial benchmarks. Inkling-Small sits right in that sweet spot. It delivers nearly identical overall reasoning scores while cutting active parameters per token by more than 70%. That trade-off gives companies running private cloud infrastructure or dedicated GPU instances a way to serve high-level coding and multimodal tasks without maintaining colossal server footprints.
MoE Architecture: 276 Billion Parameters, 12 Billion Active
Under the hood, Inkling-Small achieves its footprint reduction through a sparse Mixture-of-Experts (MoE) design. While the model holds 276 billion total parameters across its learned weights, its 42-layer decoder routes each incoming token to just six out of 256 specialized experts, alongside two shared experts that remain active across every calculation.
As a result, only 12 billion parameters execute per token inference step, compared to the 41 billion active parameters required by the 975-billion parameter Inkling. That architecture preserves broad multimodal capabilities across text, image, and audio inputs while supporting a context window up to one million tokens. Thinking Machines also built in variable reasoning effort settings, allowing developers to adjust test-time compute dynamically. If an application handles simple text classification, engineering teams can dial down execution compute; for complex algorithmic code generation, they can boost test-time compute to maximize accuracy.
Benchmark Surprises and Factual Deficits
On third-party evaluations, Inkling-Small challenges the standard assumption that bigger models always win. On the Artificial Analysis Intelligence Index, Inkling-Small scores 40 out of 100—just a single point behind the 975-billion flagship's score of 41. Artificial Analysis reported that no open-weight model at or below Inkling-Small's size scores higher on their index.
More strikingly, the smaller model actually outperforms its predecessor across several specialized benchmarks:
- On SWE-bench Verified, Inkling-Small reaches 80.2%, beating Inkling's 77.6%.
- On Terminal Bench 2.1, it reaches 64.7%, outscoring Inkling's 63.8%.
- It edges out the flagship on SciCode, GPQA Diamond, CritPt, and Humanity's Last Exam.
According to Thinking Machines researcher Horace He, these gains stemmed from refining the pre-training data mix, tuning the training recipe, using on-policy distillation with Inkling as a teacher, and running two weeks of targeted agentic coding reinforcement learning.
However, the downsized footprint carries a clear operational deficit: factual memory. On τ³-Banking, Inkling-Small scores 15.5%, falling well short of Inkling's 23.7%. Its Artificial Analysis Omniscience score is negative, reflecting compressed internal factual storage despite maintaining a slightly lower hallucination rate overall. For enterprise architects, this distinction dictates deployment strategy: Inkling-Small excels at logic, code generation, document parsing, and tool usage, but any high-stakes factual task requires pairing the model with Retrieval-Augmented Generation (RAG) and human verification pipelines.
Real-World Hardware Footprints and VRAM Limits
Despite the "Small" label, enterprise deployment teams shouldn't expect to run Inkling-Small on a local MacBook or workstation. The standard 16-bit brain floating-point (BF16) weight checkpoint requires at least 600 GB of aggregate VRAM. Production deployments demand either four NVIDIA B300 GPUs or eight NVIDIA H200 GPUs.
For teams constrained by GPU availability, quantized checkpoints offer a practical alternative. The NVFP4 4-bit quantized version cuts aggregate VRAM requirements down to roughly 180 GB. In W4A4 mode, Inkling-Small runs on a single NVIDIA B300 GPU; in W4A16 mode, it spans two NVIDIA H200 GPUs.
While that rules out desktop gaming rigs and consumer hardware, it brings hosting requirements within reach for medium-sized enterprise private clouds and specialized hosting providers. Thinking Machines provides official deployment tooling support for frameworks including SGLang, vLLM, TokenSpeed, Unsloth, and Hugging Face.
API Costs and Fine-Tuning via Tinker
For organizations that prefer managed infrastructure over self-hosting, Thinking Machines launched API access alongside full weights on Hugging Face. To incentivize early adoption, the company introduced a launch pricing model at a 50% discount:
- Input tokens (64K context prefill): $0.58 per million tokens
- Cached prefill tokens: $0.116 per million tokens
- Output tokens (sampled): $1.44 per million tokens
- Fine-tuning training tokens: $1.73 per million tokens
A 256K context variant is available at higher tier pricing. Crucially, enterprise customization relies on the company's Tinker API, which exposes direct fine-tuning controls. Enterprise teams can adapt Inkling-Small to specialized internal codebases, domain-specific terminology, and custom tool-calling formats without exposing sensitive operational data to external hosted foundation model platforms.
Licensing and the Open Source Enterprise Advantage
One of the most consequential decisions in Inkling-Small's release is its licensing. Thinking Machines released the model under the permissive Apache 2.0 license. This permits commercial modification, private distribution, internal hosting, and integration into commercial products with standard attribution requirements.
That contrasts sharply with recent releases like Moonshot AI's Kimi K3, which arrived under custom licensing terms containing commercial usage restrictions and revenue thresholds. For enterprise procurement and legal compliance teams, permissive licenses remove licensing friction and legal ambiguity around proprietary fine-tunes.
For external reporting details, see the original announcement coverage on VentureBeat. Ultimately, Inkling-Small signals a maturity shift in frontier AI development: enterprise value isn't just about building the largest possible model, but engineering efficient models that organizations can audit, fine-tune, and run on their own terms.
