When you bought your last press brake, you started with the parts you run: tonnage, bed length, the thickest plate you bend on a bad day. Then you looked at who could service it, what it cost to run, and whether your operators could learn it. The logo on the side came last.
Buying AI should work the same way, but most people start with the logo. They hear a new model name every few weeks, see a headline about a record test score, and assume they are already behind. They are not.
This guide covers who builds those models, why their test scores work like weld certs, and how to buy AI software without locking yourself into one brand.
What an AI model is
A model is the AI software a company trains on huge amounts of text, code, and images so it can read and write. A handful of companies, known as labs, build the best ones.
Chat apps run on a model from one of these labs. The AI inside your ERP and the quoting add-on at the trade show do the same.
A model knows nothing about your company until you show it, and it does not learn from your corrections on its own. Your examples, instructions, and approval rules are what make it useful to you. Those files stay yours, and the model reads them each time it works.
The labs and their current models
Models come in two kinds, and the difference matters to anyone with an NDA or ITAR work.
You rent a closed model from a lab, which keeps it on its own servers. You send text in, get an answer back, and pay for the amount of text you use. You never hold the model itself.
You can own a copy of an open-weight model. The lab publishes the model's weights, which are the trained settings that make it work, and you download them and run them on your own hardware. Your files stay in the building that way. The trade-off is that the top-scoring models are still closed, and running your own takes equipment and someone who can run it.
Fit matters more than brand, so start with the work on your desk. If your team reads scanned MTRs and drawings each day, document reading matters most, and Mistral built OCR 4.1 for that work. Anthropic is known for coding and for agent work, where the AI takes steps on a job such as reading a file and drafting a reply. Your team can use its Claude Opus 5.5 to build spreadsheets and documents. People use OpenAI models for math and coding work.
What you already have counts too. If your team uses Google Workspace or Android, you may already pay for access to Gemini, and that built-in AI may be good enough for daily office work. If your drawings cannot leave the building, start with open-weight models. Meta built Muse Glimmer to run on your own hardware.
Where a model comes from may matter to your customers, and defense customers may care most. DeepSeek and Alibaba's Qwen come from China, and Mistral comes from Europe.
A license is also a contract. Qwen's largest open model requires a separate agreement for providers with more than $50 million in revenue, so read the license the way you would read a customer's terms and conditions.
What AI test scores (benchmarks) measure
A benchmark is a standard test that every model takes so people can compare the results. It works like a weld certification. Each welder gets the same coupon and the same bend test, and judges score each test the same way, so you can compare two welders who have never met.
Models keep acing the old tests. The popular ones in 2024 and 2025 covered PhD-level science questions and small code fixes, and top models now score so high that those tests no longer tell them apart. In February, OpenAI stopped reporting one of the best-known coding tests. It pointed to flawed test questions and said some answers had leaked into training data.
Newer tests measure long jobs with many steps, which is closer to the work you would hand to an AI.
- Terminal-Bench checks whether an AI can finish multi-step jobs on a computer, such as setting up software and processing data. Anthropic reports that Opus 5.5 scored 66.4%, up from 52.3% for Opus 5.
- OSWorld 2.0 gives an AI long office jobs in real desktop apps, where it clicks and types to finish each job (about 1.6 human hours each, on median). When the test launched in July, the best score was 20.6%, and by September Anthropic reported 41.7% for Fable 5.1. That is real progress, and the model still failed more jobs than it finished.
- GDPval asks experts to grade documents, spreadsheets, and slides from 44 occupations, including manufacturing.
Labs choose which tests to publish and often run them in their own way, so you cannot compare vendor scores fairly. Artificial Analysis runs the same tests on every model, which makes its index one of the few scores you can compare across labs.
Meta's best model ranks 13th on the full list, yet it sits only 10 points behind the leader. Opus 5.5 leads the list and costs less than half as much to use. It lists at $4 in and $20 out per million tokens, against $10 and $50 for Fable 5.1 and GPT-6 Astra.
Our view: for a company like yours, a few index points matter less than price, privacy, and what your other software already uses, because none of the index tests looks like your paperwork.
What a benchmark does not tell you
A weld cert tells you a welder can make a good joint on a test plate. It says nothing about whether they show up on time, read your drawings right, or know your customer wants the grind marks facing in.
Benchmarks have the same limit. No benchmark has seen your part numbers, your customers' cert requirements, your ERP, or the way your estimator handles a revision on a scanned PDF drawing.
The score tells you a model can do hard work in general. A test on your own documents tells you whether it can do your work.
That kind of test is called a test set. It holds 50 real past cases from one job, each with the approved answer attached, and you set it aside before anyone builds anything. You then use those cases to judge every model and every version, and the person who does the job today should check the results.
For a cert packet, score each field on its own, such as the heat number, grade, and quantity. Mark the fields that count as critical errors, because one wrong value in one of those fields ships a bad part.
The benchmark is the cert. Your first article is still your job.
What AI models cost, and where prices are going
Labs set prices by the token, a small chunk of text, about a piece of a word. They quote prices for a million tokens at a time, with a lower rate for text you send in and a higher rate for text that comes back.
Prices are falling fast. Anthropic cut its top list price by about 73%, from $15 in and $75 out per million tokens for Claude Opus 4.1 in 2025 to $4 and $20 for Opus 5.5.
Epoch AI estimates the cost of a set level of AI performance fell about 47% per quarter since 2023. That pace makes the same work about 13 times cheaper every year.
The best models still cost real money. Running the full Artificial Analysis test suite costs $5.98 on Opus 5.5 and $7.63 on Fable 5.1. An agent on a long job uses far more tokens than a quick chat, so your total bill can go up even as the price per token drops.
Budget by the job. Ask any vendor what one finished job costs in model fees, such as one quote or one cert packet. We think setup and the time your reviewers spend will cost more than the model bill. A million output tokens on Opus 5.5 lists at $20, less than an hour of a quality lead's time.
How to avoid getting locked into the wrong model
The reasonable worry is that you will pick the wrong model, get locked in, and watch the whole field change the next month.
The field does change that fast. This month alone, Anthropic released two new models and OpenAI released three, and Google has another big release on the way.
By our count, the eight biggest labs made 52 notable model releases in the 52 weeks to September 25, 2026. That averages one release a week.
Source: Second Shift count from lab announcements, changelogs, and pricing pages. Six of the 52 (three xAI, two Qwen, one Meta) rest on secondary sources only. Without them the total is 46.
At that pace, a plan that depends on picking the permanent winner will fail.
Build your plan around what you own, which is your data, workflows, and approval rules. Clean and load the parts list, map the shared inbox by request type, and have a quality lead sign every cert before it goes out. None of that work belongs to a lab.
When a better or cheaper model comes out, run your test set on it. If it clears the same bar, you can switch.
A vendor locks you in by tying your workflow to one model. Ask any vendor three questions:
- Can we switch the underlying model without rebuilding the workflow?
- Do we own our data, our instructions to the model (prompts), our examples, and our records, and can we export them?
- Can the sensitive work run on an open-weight model on our own hardware if a customer requires it?
If the answers are yes, the model brand is a line item you revisit once a year.
What to do this week
You do not need to buy anything for these steps.
- List the documents that can never leave your network, such as ITAR drawings and customer IP under NDA. That list decides whether you need open-weight models for some work.
- Ask your ERP, quoting, and office software vendors which AI features you already have and which model runs underneath.
- Start a test set for one repetitive job (cert requests, POs, RFQ emails): pull 50 real past cases with the approved answer attached, mark the fields where a wrong value would cause harm, and put the folder where nobody uses it to set anything up. It will outlast any model.
- Ask the three lock-in questions of any vendor who pitches you this quarter, and write down the answers.
- Skip next month's model launch and put a note on the calendar to review the lineup once a quarter. Spend the time on your data and workflows instead.
Questions people ask
What is the difference between closed and open-weight AI models?
A closed model stays on the lab's servers, and you rent it and pay for the text you send and receive. With an open-weight model, the lab publishes the trained settings, so you can download the model, run it on your own hardware, and keep your data in the building. As of September 2026, the top-scoring models are still closed.
How much do AI models cost to use?
Labs price models per million tokens, with a lower rate for text you send in and a higher rate for text that comes back. As of September 2026, Anthropic's Claude Opus 5.5 lists at $4 in and $20 out per million tokens. We think setup and staff time will cost you more than the model itself.
How do I avoid AI vendor lock-in?
Ask any AI vendor three questions. Can you switch the underlying model without rebuilding the workflow? Do you own your data, prompts, and records, and can you export them? Can sensitive work run on an open-weight model on your own hardware? If the answers are yes, the model brand becomes a line item you revisit once a year.
Terms in this piece
- Model
- AI software a lab trains on huge amounts of text, code, and images so it can read and write. Examples include Claude, GPT, and Gemini. It knows nothing about your company until you show it.
- Lab
- A company that builds and trains its own models, such as Anthropic, OpenAI, or Google.
- Closed model
- A model you rent. It stays on the lab's servers, and you pay for what you use.
- Open-weight model
- A model whose trained settings the lab publishes, so you can download it and run it on your own hardware.
- Benchmark
- A standard test each model takes so people can compare results, like a certification test.
- Token
- A small chunk of text, about a piece of a word. Labs quote model prices per million tokens.
- Agent
- A model put to work on steps, such as reading an email and drafting a document.
- Test set
- 50 real past cases with known answers. You seal the set before building starts and use it to judge every model and every version.



