Upstage Solar Pro 4 and Solar Mini 4 Benchmark Results for Data Analysis

Oct 17, 2026: two models from South Korea's Upstage, cheap per token and fast, but not yet accurate enough for unsupervised analysis.

At Plotly, we keep a close eye on model releases. Our agentic analytics products, Plotly Studio Desktop and Plotly Studio Embedded, and our open source libraries Plotly and Dash work with any model, and our customers look to us for recommendations. Upstage is a South Korean AI company and one of three teams still in Korea's national foundation model project, part of the rise of sovereign AI we wrote about this week. We tested its two newest models:

• Solar Pro 4, Upstage's flagship, released in August 2026. It's priced at $0.09 per million input tokens and $0.36 per million output tokens on OpenRouter.
• Solar Mini 4, released on September 22, 2026, a mixture-of-experts model with 35 billion parameters, 3 billion of which are active for each token. It's priced at $0.05 per million input tokens and $0.20 per million output tokens.

Both are closed-weight models. Upstage offers them through its own API, through OpenRouter and as on-premises deployments.

On our data analytics exam, Solar Pro 4 passed 27 of 43 questions (63%) for $2.04, and Solar Mini 4 passed 21 of 43 (49%) for $1.04.

Results by category

Cheap per token, expensive per question

Both models are among the cheapest per token on our leaderboard, but they use a lot of tokens. Solar Pro 4 made about 16 model calls and read about 2.2 million tokens per question. Solar Mini 4 made about 28 calls and read about 2.6 million. GPT-6 Luna, which costs about the same per token, made about 5 calls and read about 130,000 tokens, and it ran the whole exam for $0.13 while passing 38 of 43.

At the cost per question we measured, 1,000 people each asking 10 questions a day would cost about $119 per person per year with Solar Pro 4 and $60 with Solar Mini 4. Both are reasonably fast: Solar Pro 4 answered a typical question in 21 seconds.

Where they got things right

• Ambiguous questions. Solar Pro 4 handled all five questions that can be read more than one way.
• Impossible requests. Solar Mini 4 said so every time the data couldn't answer the question, and Solar Pro 4 did four times out of five.
• Straightforward analytics. Solar Pro 4 got 14 of the 20 analytics questions right.

Where they got things wrong

• Analytics errors. Asked for total energy per region in GWh, Solar Pro 4 got the joins right but divided kilowatt-hours by a thousand instead of a million, and reported Northern Europe at 19.6 million GWh. Solar Mini 4 joined turbines straight to regions, skipping the farms and cities in between, so every regional total covered the wrong turbines.
• Data quality (0 of 3 for both). On the request with the warning "I'll be quoting these numbers in the monthly ops report, so make sure they're right", neither model noticed the duplicated rows or the missing last day of the month.
• Hidden needles. We planted five patterns in the data for the model to find on its own, without being told to look. Solar Pro 4 found one and Solar Mini 4 found none.
• Windy days and alarms. Asked whether windy days drive up SCADA alarms, both answered from the overall relationship without checking whether something else, like the season, explains both.

A note on the run: Solar Pro 4's first run stalled after 38 questions. We ran the remaining five statistics questions separately, and the results above cover all 43.

About the benchmark

We run new models through the same exam as they're released, and we'll keep sharing what we find.

The exam is private and we're sharing it with researchers. We're keeping it private so that the answer key doesn't make its way into the training set. For the same reason, the data is fabricated: the warehouse is a synthetic wind-energy fleet, built to look like a real operator's data, so its answers don't exist anywhere else.

If you'd like access, reach out to Chris Parmer at Plotly.