Open Models vs. Closed Models in 2026

Our point of view as of October 16, 2026, based on our research benchmarking AI models on data analytics.

Key takeaways

• Closed models are still the most accurate, and the cheapest and fastest at a given accuracy. On our data analytics benchmark, the best closed models score 43 of 43, and the best open-weight models (Qwen 3.8 27B, GLM 5.3 Flash and Meta's Muse Spark Open) score 38. GPT-6 Luna, a closed model, matched that 38 for $0.13 a run, less than any open model, and answers in 18 seconds, against 48 seconds or more for the open models at 38.
• Open models give you control. You can run them on your own hardware and keep data and IP inside your network, and the wider community can fine-tune them and inspect the weights.
• Running locally works, but slowly. On a desktop, the best open models take minutes per question, not seconds. Hardware ranges from a Mac mini to a GPU server that costs several hundred thousand dollars.
• Fine-tuning open models is becoming more common. A fine-tune of the open-weight Kimi K3 scored 39 of 43, four more than Kimi K3 itself.
• The Plotly and Dash Python libraries are well supported by both open and closed models, and Plotly Studio supports open and custom models on the Enterprise tier.

How we're comparing models

We built DataAnalyticsBench to measure how well Plotly Studio handles real-world data analytics for Plotly customers and Dash Enterprise customers. It's a 43-question exam that an AI agent answers against a 20M-row data warehouse, and we've run it on 28 models.

Data analytics is different from software engineering. The tasks are shorter, use less context and are more question-and-answer: one question, several queries, then an answer. So these results say how open and closed models compare for data analytics, not for long coding tasks.

What open models offer

• Run on your own hardware instead of renting it. You can run an open-weight model on a desktop, a server or your own cloud account, with no per-token bill and no dependence on a provider's uptime or pricing.
• Keep private data and IP private. Hosted providers offer agreements not to train on your data, but many organizations still worry about where their data ends up.
• Fine-tune it. You can train an open model further on your own data, terminology and questions.
• Inspect it. Researchers can study the weights directly to learn how a model works and what it prefers, which isn't possible with a model behind an API.

A short history of open models

2025: DeepSeek makes open models mainstream

• January 20. DeepSeek releases R1, a reasoning model that comes close to the best closed models at a fraction of their price.
• April. Meta releases Llama 4 and Alibaba releases Qwen 3.
• July. Moonshot releases Kimi K2 and Z.ai releases GLM 4.5.
• August 5. OpenAI releases gpt-oss, its first open-weight models since GPT-2, two days before GPT-5.

January to June 2026: Chinese labs keep the pace

• February. Z.ai releases GLM 5 and Alibaba releases Qwen 3.5.
• March. NVIDIA releases Nemotron 3 Super.
• April. Google releases Gemma 4, Moonshot releases Kimi K2.6 and DeepSeek releases V4. Meta launches Muse Spark as a closed model, a break from its open Llama models.
• June. NVIDIA releases Nemotron 3 Ultra.

July 2026 to now: open models close in on our exam

• July. Moonshot releases Kimi K3, and Thinking Machines releases its first open-weight model, Inkling.
• August. Meta returns to open weights with Muse Glimmer 30B. Alibaba releases Qwen 3.8 27B and Z.ai releases GLM 5.3 Flash, both scoring 38 of 43 on our exam.
• September. DeepSeek releases V4.1 Flash. Fireworks releases Ember-1, a fine-tune of Kimi K3 that scores 39 of 43.
• October. Mistral releases Mistral Large 4, and on October 15 NVIDIA releases Nemotron 3.5 Super VL and Meta releases Muse Spark Open, the first open-weight model from outside China to score 38 of 43 on our exam.

Were open model prices subsidized?

For most of this time, the gap that mattered was price. On OpenRouter, open models cost 10 to 100 times less per token than the frontier closed models: DeepSeek V4.1 Flash is $0.10 per million input tokens and $0.50 per million output tokens, against $4 and $20 for Claude Opus 5.5.

A common assumption was that those prices were subsidized, but the evidence suggests they mostly weren't. Open models are served by many independent hosts competing on price. DeepSeek V4.1 Flash is available from about 30 providers on OpenRouter, including Together, Fireworks, DeepInfra and CoreWeave, and several charge less than DeepSeek's own API. These hosts pay for their own GPUs. DeepSeek itself said in March 2025 that, at list prices, serving its models would earn a theoretical 545% return on its GPU costs. What the price doesn't cover is training: the lab pays for that and gives the weights away.

Most labs that release open weights still keep their best model closed. OpenAI released gpt-oss on August 5, 2025, two days before GPT-5, and described its larger version as close to o4-mini, a smaller model from earlier that year. Google's open Gemma models are smaller than its closed Gemini models. Meta follows the same pattern: on our exam, Muse Spark Open scored 38 of 43, while Meta's closed Muse Spark 1.3 scored 40 and Muse Wonder 41. The main exceptions are Chinese labs like DeepSeek, Moonshot and Z.ai, which publish the weights of their flagship models.

The rise of sovereign AI

The newest trend is sovereign AI: citizens and companies in many countries want models built and trained by AI companies in their own country, and some national funding for this is starting to appear. The Sovereign AI Index from the Center for a New American Security counted a single government-backed sovereign AI project in January 2023, and 184 projects across 67 countries by mid-2026. Many of the models are open-weight:

• South Korea. The Ministry of Science and ICT's Independent AI Foundation Model project gives competing teams NVIDIA B200 GPUs, about 768 so far with plans for about 1,000. LG AI Research, SK Telecom and Upstage are still in the running. Motif Technologies was eliminated in August 2026, and the final two teams are due to be picked in early 2027.
• Japan. METI's GENIAC program pays for part of the computing Japanese companies need to train models. Rakuten AI 3.0, a model of about 700 billion parameters released under the Apache 2.0 license in March 2026, was trained partly with GENIAC support.
• India. The IndiaAI Mission chose Sarvam in April 2025 to build a foundation model and gave it government-supported GPUs. Sarvam unveiled Sarvam 30B and Sarvam 105B, trained from scratch in India, at the AI Impact Summit in Delhi in February 2026, and has released the weights.
• Switzerland. ETH Zurich, EPFL and the Swiss National Supercomputing Centre trained Apertus, a fully open model with published training data and code, on the public Alps supercomputer. Apertus 1.5 came out on July 24, 2026.
• United Arab Emirates. The Institute of Foundation Models at MBZUAI, Abu Dhabi's AI university, released K2 Horizon, six open models from 0.9 billion to 375 billion parameters, on September 3, 2026.
• Germany. Aleph Alpha released Kolibri, a 78-billion-parameter open-weight model for government and industry, on October 3, 2026. Aleph Alpha is privately funded, with Lidl owner Schwarz Group among its backers, and is merging with Canada's Cohere.
• Canada. The federal government committed up to C$240 million to Cohere, the country's main developer of large language models, toward a C$725 million data centre project in Canada under its Sovereign AI Compute Strategy, and finalized the investment in March 2025. After the Aleph Alpha merger, the combined company will be headquartered in both Toronto and Berlin.
• France. Mistral raised €3 billion in September 2026 in a round led by Samsung. President Macron said the round reflected France and South Korea's goal of "building a third way in AI."
• United Kingdom. The government launched a Sovereign AI Unit in April 2026 with up to £500 million to invest in British AI companies.

Most of these models aren't on OpenRouter yet, so we haven't been able to benchmark them.

Open vs. closed models today

The price gap has closed for smaller models. Closed providers now sell small models at open-model prices. GPT-6 Luna and Claude Haiku 5.5 cost the same per token as DeepSeek V4.1 Flash, and on our exam they matched or beat it:

Running on your own hardware is compelling, but slow for interactive use. On a desktop, Qwen 3.8 27B runs on a 64 GB Mac mini and Muse Spark Open on two 512 GB Mac Studios. From measured hardware speeds and the tokens each model used on our exam, we estimate both would take about 3½ minutes for a typical question, against under a minute through a hosted provider. If answering a question takes more than 20 to 30 seconds, people tend to switch to something else. Local models fit analysis that runs in the background better than questions someone is waiting on.

Privacy and independence still favor open models. Running a model yourself means your data stays in your network and you aren't tied to one provider's subscription, pricing or terms. For many Plotly customers in regulated industries, that matters more than a few points of accuracy.

Fine-tuning is becoming feasible for more companies. Fireworks' Ember-1 shows what it can do: it's a fine-tune of the open-weight Kimi K3, trained to reason with fewer tokens, and on our exam it scored 39 of 43, against 35 for Kimi K3, catching all three planted data quality problems where Kimi K3 caught one. Fine-tuning is also getting much easier, because AI agents can now do much of the work themselves: preparing the training data, running jobs on services like Thinking Machines' Tinker, and evaluating the results. We expect to see many more companies tuning open models for their own kinds of analysis.

Hardware options for running open models

Try open models through a hosted provider first. You don't need your own hardware to try an open model. OpenRouter lists most popular open models, each served by several independent hosts, so you can test one on your own questions for a few dollars before buying anything. Not every open model is there yet: Aleph Alpha's Kolibri and models from Motif, SK Telecom and K2 Horizon still aren't available on OpenRouter, so we haven't been able to benchmark them. Reflection's Beam, which wasn't available when we published DataAnalyticsBench, was added on October 16, 2026.

If you want to run an open model yourself, the hardware you need depends on the size of the model and how many people will use it at once. The hardware can be tricky to price out, especially as usage patterns are changing so much. Here are some estimates from a few sources.

• A Mac on your desk. Apple's unified memory lets a Mac load models that would need several GPUs elsewhere. A Mac mini with 64 GB runs Qwen 3.8 27B, which scored 38 of 43 on our exam. At the high end, Apple's machine learning research team published results on October 15, 2026 for Muse Spark Open running on M5 Ultra Mac Studios with MLX, Apple's open source machine learning framework. At 4-bit precision the model needs about 493 GB, more than a single 512 GB Mac Studio can hold once you leave room for inference, so Apple splits it across two or four machines connected over Thunderbolt 5, using a new RDMA driver in macOS 26.2. With an 8,000-token prompt, two Mac Studios generated about 25 tokens per second and four generated about 37. Long prompts take a while to read: with 65,000 tokens of context, four machines took about 40 seconds before they started answering. The catch is supply: AI demand and a global memory shortage have pushed Mac mini and Mac Studio delivery times out by months this year, and the 512 GB Mac Studio configuration isn't shipping yet.
• NVIDIA's desktop AI computers. The DGX Spark is a small desktop box with 128 GB of memory, now about $6,950 after a recent price increase, with a 64 GB version at $4,999. It fits models like Nemotron 3.5 Super VL. The much larger DGX Station has up to 748 GB of memory, enough to hold Muse Spark Open in a single machine, and sells through hardware vendors for roughly $90,000 to $160,000.
• An on-prem server. To serve a team instead of one person, you need a GPU server. An 8-GPU server with NVIDIA H200 or B200 GPUs costs roughly $350,000 to $500,000 and can run the largest open models for many people at once. Servers built on NVIDIA's RTX PRO 6000 Blackwell cards, with 96 GB each, are a cheaper middle option. You can also rent the same GPUs by the hour from GPU clouds or in your own AWS, Azure or Google Cloud account.

How many servers you need depends on how people use them. Inference software batches many people's requests together, so one 8-GPU server ran DeepSeek R1 for up to 128 requests at once, and DeepSeek's own service averages about 700 users per server. Questions asked during the day come in bursts: 1,000 people asking 10 a day keep only about 30 in flight, which a couple of servers can handle. Through OpenRouter that costs about $300,000 a year with Muse Spark Open, but $7,500 with GPT-6 Luna, $22,000 with Claude Haiku 5.5 and $150,000 with GPT-6.1 Sol, which scores 43 of 43, so owning hardware is hard to justify for day-to-day analysis. Background agents are different. They run all day, so 1,000 people with an agent each need about 1,000 requests running at once, roughly 10 servers, and the same work with Muse Spark Open through OpenRouter would cost around $10 million a year. When servers stay busy like that, owning them makes sense.

Getting started with open models in Plotly Studio

Plotly Studio works with any model, and supports open-weight and custom models, including ones you host yourself, on the Enterprise tier. If you're weighing open and closed models for analytics in your organization, get in touch and we'll help you test them on your own data.

For the full benchmark results, including cost per person per year and how each model handles ambiguity, data quality and statistics, see Introducing DataAnalyticsBench.

Frequently asked questions

Are open models as accurate as closed models for data analytics?

Not yet. On our data analytics benchmark, as of October 16, 2026, the best closed models (GPT-6.1 Sol and GPT-6 Astra) score 43 of 43, and the best open-weight models (Qwen 3.8 27B, GLM 5.3 Flash and Muse Spark Open) score 38. Most models, open and closed, handle straightforward SQL well. The gap shows up on data quality problems, statistics and hidden patterns in the data.

What does your benchmark measure?

We ran every model through DataAnalyticsBench, our benchmark for data analytics. Each model answers the same 43 questions against the same 20M-row data warehouse, a synthetic wind-energy fleet, with the same agent and tools: listing tables, running SQL queries and making charts and tables. We grade every answer against a key. The questions cover straightforward analytics, ambiguous and impossible requests, data quality problems, statistics traps and hidden patterns in the data.

These results cover short-horizon data analytics tasks: one question, several queries, then an answer. That's different from world knowledge tasks, where a model answers from what it learned in training, and from large software engineering projects, where an agent works for hours across a big codebase. A model that does well on our exam may rank differently on those.

Are open models cheaper than closed models?

Not necessarily. Small closed models now cost the same per token as small open ones: GPT-6 Luna, Claude Haiku 5.5 and DeepSeek V4.1 Flash all cost $0.10 per million input tokens and $0.50 per million output tokens on OpenRouter. What a question costs also depends on how many tokens a model reads and writes. GPT-6 Luna ran our whole exam for $0.13, and the cheapest open models cost $0.46.

What is the best open-weight model for data analytics?

As of October 16, 2026, Qwen 3.8 27B, GLM 5.3 Flash and Meta's Muse Spark Open are tied on our benchmark at 38 of 43. Qwen 3.8 27B is the smallest and runs on a 64 GB Mac mini. Muse Spark Open is the best open-weight model we've tested from a US lab.

Can I run an open model on my own computer for data analysis?

Yes. Qwen 3.8 27B runs on a Mac with 64 GB of memory, Nemotron 3.5 Super VL fits on a 128 GB Mac or an NVIDIA DGX Spark, and Muse Spark Open needs two 512 GB Mac Studios. Expect minutes per question rather than seconds, which suits background analysis better than interactive questions.

Do open models keep my data private?

Only if you run them yourself. An open model on your own hardware or in your own cloud account keeps your data inside your network. If you call an open model through a hosted provider, your data goes to that provider, just as it would with a closed model.

What's the difference between open-weight and open-source models?

An open-weight model's trained weights can be downloaded and run, but the training data and code usually aren't released. Licenses vary: some allow any commercial use, while Meta's Muse Model License for Muse Spark Open has limited commercial terms. Check the license before you deploy.

Does Plotly support open models?

Yes. The Plotly and Dash Python libraries are well supported by both open and closed models, and Plotly Studio supports open and custom models on the Enterprise tier.