Open Models vs. Closed Models in 2026

Our point of view as of November 6, 2026, based on our research benchmarking AI models on data analytics.

Key takeaways

• Closed models are still the most accurate, and the cheapest and fastest at a given accuracy. On our data analytics benchmark, the best closed models score 43 of 43, and the best open-weight model, Moonshot's Kimi K3.1 Pro, scores 42, for $13.47 a run at a median of about 2½ minutes per question. Grok 4.7, a closed model, matched that 42 for $7.85 in 38 seconds. Further down, Qwen 3.8 27B, GLM 5.3 Flash and Meta's Muse Spark Open score 38, and GPT-6 Luna, a closed model, matched that 38 for $0.13 a run, less than any open model, and answers in 18 seconds, against 48 seconds or more for the open models at 38. OpenAI's new GPT-6.1 Terra scored 40 for $1.31, less than Qwen 3.8 27B's $1.71 for 38, in 15 seconds, and Particle-1, a fine-tune of DeepSeek V4.1 Flash whose weights haven't been released, scored 40 for $0.35. Tencent's Hy4 scored 42 for $2.11, and Tencent says it will release the weights in the coming days. Once it does, Hy4 will be the cheapest model at that score, open or closed.
• Open models give you control. You can run them on your own hardware and keep data and IP inside your network, and the wider community can fine-tune them and inspect the weights.
• Running locally works, but slowly. On a desktop, the best open models take minutes per question, not seconds. Hardware ranges from a Mac mini to a GPU server that costs several hundred thousand dollars.
• Fine-tuning open models is becoming more common. A fine-tune of the open-weight Kimi K3 scored 39 of 43, four more than Kimi K3 itself, and Particle-1, a fine-tune of DeepSeek V4.1 Flash, scored 40, four more than its base model.
• The Plotly and Dash Python libraries are well supported by both open and closed models, and Plotly Studio supports open and custom models on the Enterprise tier.

How we're comparing models

We built DataAnalyticsBench to measure how well Plotly Studio handles real-world data analytics for Plotly customers and Dash Enterprise customers. It's a 43-question exam that an AI agent answers against a 20M-row data warehouse, and we've run it on 43 models.

Data analytics is different from software engineering. The tasks are shorter, use less context and are more question-and-answer: one question, several queries, then an answer. So these results say how open and closed models compare for data analytics, not for long coding tasks.

What open models offer

• Run on your own hardware instead of renting it. You can run an open-weight model on a desktop, a server or your own cloud account, with no per-token bill and no dependence on a provider's uptime or pricing.
• Keep private data and IP private. Hosted providers offer agreements not to train on your data, but many organizations still worry about where their data ends up.
• Fine-tune it. You can train an open model further on your own data, terminology and questions.
• Inspect it. Researchers can study the weights directly to learn how a model works and what it prefers, which isn't possible with a model behind an API.

How do open models perform?

Accuracy. On our exam, the best open-weight model is Moonshot's Kimi K3.1 Pro, at 42 of 43. Below it, Qwen 3.8 27B, GLM 5.3 Flash and Meta's Muse Spark Open score 38. The best closed models, GPT-6.1 Sol and GPT-6 Astra, score 43. Open and closed models alike handle straightforward SQL well. The gap is in the harder questions: spotting data quality problems, avoiding statistics traps and finding hidden patterns. MiniMax M3.1 Flash, released October 19 with its weights following on October 23, scored 41 and found all five hidden patterns. Moonshot released Kimi K3.1 Pro on October 23 and its weights on November 2, the closest an open-weight model has come to the top of our board. Tencent's Hy4, released November 4, also scored 42, and Tencent says it will release the weights in the coming days. The same day, Ricursive released Ripple-65B Preview, a 65B open-weight model from a US lab, which scored 35 for $2.21, less than half the cost of Muse Spark Open. We've also tested OpenAI's gpt-oss-120b, Google's Gemma 4 31B, Meta's Muse Glimmer 30B, and Thinking Machines' Inkling-Small and Inkling. Inkling is the largest U.S. open-weight model, at 975 billion parameters, and scored 29 for $9.54. We compare every U.S. open-weight model in our U.S. open-weight models post.

Cost. Open models first caught on because they were cheap. On OpenRouter, they cost 10 to 100 times less per token than the frontier closed models: DeepSeek V4.1 Flash is $0.10 per million input tokens and $0.50 per million output tokens, against $4 and $20 for Claude Opus 5.5. That gap has closed for smaller models. GPT-6 Luna and Claude Haiku 5.5 now cost the same per token as DeepSeek V4.1 Flash, and on our exam they matched or beat it. The cost of a question also depends on how many tokens a model reads and writes, which is why GPT-6 Luna ran the whole exam for less than any open model:

Speed. Through hosted providers, the best open models are also slower. GPT-6 Luna and GPT-6.1 Sol answer a typical question in about 18 seconds and GPT-6.1 Terra in 15, while the open models that score 38 take 48 to 87 seconds, and Kimi K3.1 Pro, at 42, about 2½ minutes. MiniMax M3.1 Flash is the slowest model on our leaderboard, at about 3 minutes per question, because it writes about 32,000 tokens for each one. The two fastest models on our board, Inception's closed Mercury 3 and OpenAI's open gpt-oss-120b, both answer in about 9 seconds, with 35 and 27 of 43. On your own hardware, open models are slower still, as we cover below.

A short history of open models

2025: DeepSeek makes open models mainstream

• January 20. DeepSeek releases R1, a reasoning model that comes close to the best closed models at a fraction of their price.
• April. Meta releases Llama 4 and Alibaba releases Qwen 3.
• July. Moonshot releases Kimi K2 and Z.ai releases GLM 4.5.
• August 5. OpenAI releases gpt-oss, its first open-weight models since GPT-2, two days before GPT-5.

January to June 2026: Chinese labs keep the pace

• February. Z.ai releases GLM 5 and Alibaba releases Qwen 3.5.
• March. NVIDIA releases Nemotron 3 Super.
• April. Google releases Gemma 4, Moonshot releases Kimi K2.6 and DeepSeek releases V4. Meta launches Muse Spark as a closed model, a break from its open Llama models.
• June. NVIDIA releases Nemotron 3 Ultra.

July 2026 to now: open models close in on our exam

• July. Moonshot releases Kimi K3, and Thinking Machines releases its first open-weight model, Inkling.
• August. Meta returns to open weights with Muse Glimmer 30B. Alibaba releases Qwen 3.8 27B and Z.ai releases GLM 5.3 Flash, both scoring 38 of 43 on our exam.
• September. DeepSeek releases V4.1 Flash. Fireworks releases Ember-1, a fine-tune of Kimi K3 that scores 39 of 43.
• October. Mistral releases Mistral Large 4, and on October 15 NVIDIA releases Nemotron 3.5 Super VL and Meta releases Muse Spark Open, the first open-weight model from outside China to score 38 of 43 on our exam. On October 16 Reflection releases the weights for Beam 1.0, and on October 19 MiniMax releases M3.1 Flash, which scores 41 of 43, and releases the weights on October 23. On October 23 Moonshot releases Kimi K3.1 Pro, which scores 42 of 43, with the weights promised by November 4. On October 28 Particle AI releases Particle-1, a fine-tune of DeepSeek V4.1 Flash that scores 40 of 43, without releasing its weights.
• November. On November 2 Moonshot releases the weights for Kimi K3.1 Pro, the most accurate open-weight model we've tested, at 42 of 43. On November 4 Tencent releases Hy4, the full version of the Hy4 preview it open-sourced in August, which also scores 42, and says the weights will follow in the coming days (more in our Hy4 post). The same day Ricursive releases Ripple-65B Preview, a 65B open-weight model trained from scratch, which scores 35 of 43 (more in our Ripple-65B Preview post).

Were open model prices subsidized?

A common assumption was that open models' low prices were subsidized, but the evidence suggests they mostly weren't. Open models are served by many independent hosts competing on price. DeepSeek V4.1 Flash is available from about 30 providers on OpenRouter, including Together, Fireworks, DeepInfra and CoreWeave, and several charge less than DeepSeek's own API. These hosts pay for their own GPUs. DeepSeek itself said in March 2025 that, at list prices, serving its models would earn a theoretical 545% return on its GPU costs. What the price doesn't cover is training: the lab pays for that and gives the weights away.

Most labs that release open weights still keep their best model closed. OpenAI released gpt-oss on August 5, 2025, two days before GPT-5, and described its larger version as close to o4-mini, a smaller model from earlier that year. Google's open Gemma models are smaller than its closed Gemini models. Meta follows the same pattern: on our exam, Muse Spark Open scored 38 of 43, while Meta's closed Muse Spark 1.3 scored 40 and Muse Wonder 41. The main exceptions are Chinese labs like DeepSeek, Moonshot and Z.ai, which publish the weights of their flagship models.

The rise of sovereign AI

The newest trend is sovereign AI: citizens and companies in many countries want models built and trained by AI companies in their own country, and some national funding for this is starting to appear. The Sovereign AI Index from the Center for a New American Security counted a single government-backed sovereign AI project in January 2023, and 184 projects across 67 countries by mid-2026. Many of the models are open-weight:

• South Korea. The Ministry of Science and ICT's Independent AI Foundation Model project gives competing teams NVIDIA B200 GPUs, about 768 so far with plans for about 1,000. LG AI Research, SK Telecom and Upstage are still in the running. Motif Technologies was eliminated in August 2026, and the final two teams are due to be picked in early 2027.
• Japan. METI's GENIAC program pays for part of the computing Japanese companies need to train models. Rakuten AI 3.0, a model of about 700 billion parameters released under the Apache 2.0 license in March 2026, was trained partly with GENIAC support.
• India. The IndiaAI Mission chose Sarvam in April 2025 to build a foundation model and gave it government-supported GPUs. Sarvam unveiled Sarvam 30B and Sarvam 105B, trained from scratch in India, at the AI Impact Summit in Delhi in February 2026, and has released the weights.
• Switzerland. ETH Zurich, EPFL and the Swiss National Supercomputing Centre trained Apertus, a fully open model with published training data and code, on the public Alps supercomputer. Apertus 1.5 came out on July 24, 2026.
• United Arab Emirates. The Institute of Foundation Models at MBZUAI, Abu Dhabi's AI university, released K2 Horizon, six open models from 0.9 billion to 375 billion parameters, on September 3, 2026.
• Germany. Aleph Alpha released Kolibri, a 78-billion-parameter open-weight model for government and industry, on October 3, 2026. Aleph Alpha is privately funded, with Lidl owner Schwarz Group among its backers, and is merging with Canada's Cohere.
• Canada. The federal government committed up to C$240 million to Cohere, the country's main developer of large language models, toward a C$725 million data centre project in Canada under its Sovereign AI Compute Strategy, and finalized the investment in March 2025. After the Aleph Alpha merger, the combined company will be headquartered in both Toronto and Berlin. On October 29, 2026, Cohere announced a Series E round targeting up to C$7 billion. Its open-weight Command A+, released in May, scored 18 of 43 on our exam (more in our Command A+ post).
• France. Mistral raised €3 billion in September 2026 in a round led by Samsung. President Macron said the round reflected France and South Korea's goal of "building a third way in AI."
• United Kingdom. The government launched a Sovereign AI Unit in April 2026 with up to £500 million to invest in British AI companies.

Most of these models aren't on OpenRouter yet, so we haven't been able to benchmark them.

Open vs. closed models today

Running on your own hardware is compelling, but slow for interactive use. On a desktop, Qwen 3.8 27B runs on a 64 GB Mac mini and Muse Spark Open on two 512 GB Mac Studios. From measured hardware speeds and the tokens each model used on our exam, we estimate both would take about 3½ minutes for a typical question, against under a minute through a hosted provider. If answering a question takes more than 20 to 30 seconds, people tend to switch to something else. Local models fit analysis that runs in the background better than questions someone is waiting on.

Privacy and independence still favor open models. Running a model yourself means your data stays in your network and you aren't tied to one provider's subscription, pricing or terms. For many Plotly customers in regulated industries, that matters more than a few points of accuracy.

Fine-tuning is becoming feasible for more companies. Fireworks' Ember-1 shows what it can do: it's a fine-tune of the open-weight Kimi K3, trained to reason with fewer tokens, and on our exam it scored 39 of 43, against 35 for Kimi K3, catching all three planted data quality problems where Kimi K3 caught one. Particle-1, from the startup Particle AI, is a second example: a fine-tune of the open-weight DeepSeek V4.1 Flash, trained with reinforcement learning to use fewer tokens on coding tasks. It scored 40 of 43 against 36 for DeepSeek V4.1 Flash, with the gains on statistics traps and hidden patterns, which it wasn't specifically trained for. Fine-tuning is also getting much easier, because AI agents can now do much of the work themselves: preparing the training data, running jobs on services like Thinking Machines' Tinker, and evaluating the results. We expect to see many more companies tuning open models for their own kinds of analysis.

Hardware options for running open models

Try open models through a hosted provider first. You don't need your own hardware to try an open model. OpenRouter lists most popular open models, each served by several independent hosts, so you can test one on your own questions for a few dollars before buying anything. Not every open model is there yet: Aleph Alpha's Kolibri and models from Motif, SK Telecom and K2 Horizon still aren't available on OpenRouter, so we haven't been able to benchmark them. Reflection's Beam, which wasn't available when we published DataAnalyticsBench, was added on October 16, 2026.

If you want to run an open model yourself, the hardware you need depends on the size of the model and how many people will use it at once. The hardware can be tricky to price out, especially as usage patterns are changing so much. Here are some estimates from a few sources.

• A Mac on your desk. Apple's unified memory lets a Mac load models that would need several GPUs elsewhere. A Mac mini with 64 GB runs Qwen 3.8 27B, which scored 38 of 43 on our exam. At the high end, Apple's machine learning research team published results on October 15, 2026 for Muse Spark Open running on M5 Ultra Mac Studios with MLX, Apple's open source machine learning framework. At 4-bit precision the model needs about 493 GB, more than a single 512 GB Mac Studio can hold once you leave room for inference, so Apple splits it across two or four machines connected over Thunderbolt 5, using a new RDMA driver in macOS 26.2. With an 8,000-token prompt, two Mac Studios generated about 25 tokens per second and four generated about 37. Long prompts take a while to read: with 65,000 tokens of context, four machines took about 40 seconds before they started answering. The catch is supply: AI demand and a global memory shortage have pushed Mac mini and Mac Studio delivery times out by months this year, and the 512 GB Mac Studio configuration isn't shipping yet.
• NVIDIA's desktop AI computers. The DGX Spark is a small desktop box with 128 GB of memory, now about $6,950 after a recent price increase, with a 64 GB version at $4,999. It fits models like Nemotron 3.5 Super VL. The much larger DGX Station has up to 748 GB of memory, enough to hold Muse Spark Open in a single machine, and sells through hardware vendors for roughly $90,000 to $160,000.
• An on-prem server. To serve a team instead of one person, you need a GPU server. An 8-GPU server with NVIDIA H200 or B200 GPUs costs roughly $350,000 to $500,000 and can run the largest open models for many people at once. Servers built on NVIDIA's RTX PRO 6000 Blackwell cards, with 96 GB each, are a cheaper middle option. You can also rent the same GPUs by the hour from GPU clouds or in your own AWS, Azure or Google Cloud account.

How many servers you need depends on how people use them. A server doesn't handle one person per GPU: inference software batches many people's requests together. One 8-GPU server ran DeepSeek R1 for up to 128 requests at once, and DeepSeek's own service averages about 700 users per server by spreading the model across many servers.

Day-to-day questions. If 1,000 people each ask 10 questions a day, only about 30 are being answered at any moment, so two or three servers would do, at $700,000 to $1.5 million before power and staff. Renting costs less. A year of those questions through OpenRouter costs:

• GPT-6 Luna (38 of 43): about $7,500
• Claude Haiku 5.5 (37 of 43): about $22,000
• GPT-6.1 Sol (43 of 43): about $150,000
• Muse Spark Open (38 of 43): about $300,000

Background agents. An agent works all day, so 1,000 people with one agent each keep about 1,000 requests going at once. At the speed a hosted provider gives each request, that takes dozens of 8-GPU servers, roughly $10 million to $25 million up front. Renting the same work with Muse Spark Open would cost around $10 million a year, so owning only catches up after one to three years of constant use. And a small closed model does the same work for much less: on our exam, GPT-6 Luna cost about a fortieth as much per question as Muse Spark Open.

So today, owning hardware makes sense for control and privacy, not to save money.

Getting started with open models in Plotly Studio

Plotly Studio works with any model, and supports open-weight and custom models, including ones you host yourself, on the Enterprise tier. If you're weighing open and closed models for analytics in your organization, get in touch and we'll help you test them on your own data.

For the full benchmark results, including cost per person per year and how each model handles ambiguity, data quality and statistics, see Introducing DataAnalyticsBench.

Frequently asked questions

Are open models as accurate as closed models for data analytics?

Not yet. On our data analytics benchmark, as of November 6, 2026, the best closed models (GPT-6.1 Sol and GPT-6 Astra) score 43 of 43, and the best open-weight model (Kimi K3.1 Pro) scores 42. Most models, open and closed, handle straightforward SQL well. The gap shows up on data quality problems, statistics and hidden patterns in the data. Tencent's Hy4 also scored 42, and Tencent says it will release the weights in the coming days.

What does your benchmark measure?

We ran every model through DataAnalyticsBench, our benchmark for data analytics. Each model answers the same 43 questions against the same 20M-row data warehouse, a synthetic wind-energy fleet, with the same agent and tools: listing tables, running SQL queries and making charts and tables. We grade every answer against a key. The questions cover straightforward analytics, ambiguous and impossible requests, data quality problems, statistics traps and hidden patterns in the data.

These results cover short-horizon data analytics tasks: one question, several queries, then an answer. That's different from world knowledge tasks, where a model answers from what it learned in training, and from large software engineering projects, where an agent works for hours across a big codebase. A model that does well on our exam may rank differently on those.

Are open models cheaper than closed models?

Not necessarily. Small closed models now cost the same per token as small open ones: GPT-6 Luna, Claude Haiku 5.5 and DeepSeek V4.1 Flash all cost $0.10 per million input tokens and $0.50 per million output tokens on OpenRouter. What a question costs also depends on how many tokens a model reads and writes. GPT-6 Luna ran our whole exam for $0.13 with 38 of 43. The cheapest open model, OpenAI's gpt-oss-120b, cost $0.22 but scored 27.

What is the best open-weight model for data analytics?

As of November 6, 2026, Moonshot's Kimi K3.1 Pro, whose weights came out on November 2, is the most accurate open-weight model we've tested, at 42 of 43, but at about 1.6 TB it needs a GPU server. Among smaller models, Qwen 3.8 27B, GLM 5.3 Flash and Meta's Muse Spark Open are tied at 38 of 43. Qwen 3.8 27B is the smallest and runs on a 64 GB Mac mini. Muse Spark Open is the best open-weight model we've tested from a US lab (see our ranking of U.S. open-weight models). MiniMax M3.1 Flash, whose weights came out on October 23, scored 41.

Can I run an open model on my own computer for data analysis?

Yes. Qwen 3.8 27B runs on a Mac with 64 GB of memory, Nemotron 3.5 Super VL fits on a 128 GB Mac or an NVIDIA DGX Spark, and Muse Spark Open needs two 512 GB Mac Studios. Expect minutes per question rather than seconds, which suits background analysis better than interactive questions.

Do open models keep my data private?

Only if you run them yourself. An open model on your own hardware or in your own cloud account keeps your data inside your network. If you call an open model through a hosted provider, your data goes to that provider, just as it would with a closed model.

What's the difference between open-weight and open-source models?

An open-weight model's trained weights can be downloaded and run, but the training data and code usually aren't released. Licenses vary: some allow any commercial use, while Meta's Muse Model License for Muse Spark Open has limited commercial terms. Check the license before you deploy.

Does Plotly support open models?

Yes. The Plotly and Dash Python libraries are well supported by both open and closed models, and Plotly Studio supports open and custom models on the Enterprise tier.