Small language models · New or adapted
We build small language models that run on your own servers.
Many organisations we work with can't send their data to an outside AI service. So the model is installed with them: on their servers, in their private cloud, sometimes on machines with no internet connection at all. We help you decide how to build it, then build it, test it on your own work and hand it over.
Three ways to build
What “from scratch” means.
Clients often ask us to “build an SLM”, and that can mean quite different things. Here are the three ways, and when each one makes sense.
| Training a new model | Continued pretraining on an open base model | Fine-tuning and instruction tuning | |
|---|---|---|---|
| What it is | Every weight is learned from data you choose, with a tokenizer built for your text, so you know exactly what the model has read. | We take an open-weight model whose licence and origin we have checked, and train it further on your documents. | We train an existing model on worked examples of your tasks, reviewed by your experts. |
| Tokenizer | New, trained on your text. | The base model's, extended with your terms if needed. | The base model's. |
| Data you need | Far more than the other two: a large body of text in your domain and languages, plus general text so the model still reads and writes well. | A good body of your own documents. | Worked examples of the task, checked by your experts. |
| Control | Full. You decide the architecture, the size and every document it learns from. | High. You choose the training data; the base model's design and licence still apply. | Shapes behaviour and output format. Most knowledge comes from the base model. |
| Cost and time | The most expensive by a distance, and the longest. | Much less data and compute. | The quickest. |
| Right when | Your language or notation is poorly served by existing models, every source the model learnt from has to be accounted for, or the job needs a very small model that does one narrow thing. | Most domain work, such as law, tax, clinical guidelines or internal policy. It is the default in Jungroo AI Studio. | You need one task done well, such as extraction, classification or drafting in a set format. |
We help you choose. Most projects that start as a request for a brand-new model are better served by adapting an open model, at much lower cost. If yours really needs a new model, we scope it with you first: the data mix, the tokenizer and the compute. We also keep an adapted model in the comparison, so you can see what the extra cost bought.
How we build
How a project runs.
The order of work is much the same every time. Fine-tuning projects skip stage 4.
- 01
We build the test first
Real cases from your work, each with the answer it should get, checked by someone who knows the domain. We run it on whatever you use today, so there is a baseline before anything changes.
- 02
We sort out the data
We check what you are allowed to train on, then collect, clean and version it, removing duplicates and personal data where needed. In our experience, a lot of accuracy problems start here, not in the model.
- 03
We choose the starting point, the size and the tokenizer
The model has to suit the hardware it will run on; one that needs a GPU cluster is no use to a client with one server. For a new model we train a tokenizer on your text. For an open model we check how well its tokenizer handles your terms.
- 04
Pretraining
The model learns your language and domain from your documents, from random weights or from the open base model. We test checkpoints as training runs, so problems show up early.
- 05
Instruction tuning and alignment
We teach the model your tasks with examples your experts write or review, then align it with Direct Preference Optimization (DPO): your experts pick the better of two answers, and the model learns from those choices. When the output is structured, such as fields pulled from a form, we give the model an output layer built for that job instead of having it write free text.
- 06
We compare before go-live
The new model is tested on the same test as general-purpose models and your current service, and we report the results whichever way they go.
- 07
Inference optimisation
We make the model fast and cheap to run on your hardware, with quantisation to int8 or FP8, speculative decoding and caching of repeated prompts. Each change is tested again, so accuracy holds.
- 08
We install it inside your network
Through Jungroo AI Studio, on-premise, in a private cloud or air-gapped, with role-based access, every action logged and a person signing off decisions.
- 09
We keep it current
Content and search indexes are refreshed on a schedule, and the model is retrained when your documents or rules change. Each new version is checked against the current one before release.
Proof
Models we have built.
ReDefiNER is open source, so you can test it yourself. The others are client work, so names are left out. Accuracy figures come from test sets we built for each project, unless we say they are public benchmarks.
GST Amicus: drafting help for tax officers
Before a GST officer in India can issue an adjudication order, they read the case file, check provisions that change often and go through past judgments. In the state we work with, that took 48 to 72 hours per notice.
We trained an open model further on Indian GST law, with the legal thinking from our tax partner, a practice with more than 37 years in GST, and built a drafting assistant around it. Every citation opens the document it came from, so the order holds up on appeal, and the officer edits and signs, so it stays their order. It now takes about 30 minutes per notice, on government servers.
ReDefiNER: our open-source extraction model
You describe what you want in a plain-English rulebook, and the model returns the exact pieces of your text, each with a type and a confidence score. To change what it extracts, you edit the rulebook; nothing is retrained.
We started from the open Qwen3.5-0.8B model, adapted it with LoRA and replaced text generation with our own span and label heads. It runs on an ordinary CPU. On 13 public test sets, with GoLLIE's own guidelines, it averages 59.8 F1 against 58.1 for GoLLIE 34B, a research model about 40 times its size, and leads on 9 of the 13. Free under Apache 2.0.
Plain-English questions over company data
People want answers from their own database without writing SQL or waiting for IT. They ask in plain English, and the model sees only the structure of the database, its tables, columns and relationships, never the records.
Every query it writes is checked for syntax and against the user's role before it runs, and the answer comes back as a table or a chart. We have built this with a large model that reads only the schema, and with a small open model adapted for SQL that runs entirely inside the client's network.
A student help assistant for a US college
A US education technology company needed an assistant that answers students' questions from a college's own web pages, with student data kept on its own server.
We built the test first, then compared five retrieval models and three language models on it, and chose Granite Embedding R2 and Gemma 4 26B, running on the client's GPU server. On our test sets it finds the right page 98.5% of the time and gives a correct answer 95.3% of the time.
Who it's for
Who we build for.
Two kinds of teams ask us to build small language models.
Your data can't go to an outside AI service.
Government departments, banks and insurers, hospitals, colleges and companies that handle sensitive records. The model, the data and the logs stay inside your environment.
You need an SLM build partner for a client pursuit.
We can support your bid with a feasibility and data assessment, then build, test and deploy the model with your team.
Working with us
How an engagement runs.
Four steps, from the first call to a model running inside your infrastructure.
Discovery call
We talk through your use case, your data and your constraints, and which of the three ways looks right.
Feasibility and data assessment
We look at your data and the rules around it, then recommend a route, a model size and the hardware, and agree a plan with you.
Build
We build the test and the data, train the model and compare it on your tasks. You see the results at every stage.
Deployment
We install the model inside your infrastructure through Jungroo AI Studio, hand it over and train your team to run it.
Before we start, we pin down a few things
- What the model should do, and what a good answer looks like
- Which languages and kinds of text it has to handle
- How much data there is, who owns it, and whether it can be used for training
- Where the model has to run, and on what hardware
- How accurate it needs to be, and who decides that
- Whether starting from an open model is acceptable, or every training source must be accounted for
The answers usually settle which of the three ways fits. If you want to see results before committing, we can run a free two-week benchmark on one of your tasks, inside your environment.
Questions
Small language models, explained.
Plain answers to what clients and partners ask us most.
What is a small language model?
A language model small enough to run on your own servers, sometimes on a single GPU server or even an ordinary CPU, and trained or tuned for one kind of work. It does fewer things than a large general model, but it can do your job well, keep your data inside and cost the same however much you use it.
Can you build a small language model from scratch?
Yes, when the job calls for it: a language or notation that existing models handle badly, a need to account for every source the model learnt from, or a very small model for one narrow task. It is the most expensive way by a distance, so we scope the data mix, the tokenizer and the compute with you first, and keep an adapted open model in the comparison so you can see what the extra cost buys.
What is the difference between pretraining from scratch, continued pretraining and fine-tuning?
Pretraining from scratch learns every weight from data you choose, with a tokenizer built for your text. Continued pretraining takes an open model and trains it further on your documents, which suits most domain work. Fine-tuning trains an existing model on worked examples of one task, and is the quickest.
How much data do we need?
It depends on the route. Fine-tuning needs worked examples of the task, checked by your experts. Continued pretraining needs a good body of your own documents. A new model needs far more text than either, plus general text. In the feasibility assessment we look at what you have and tell you which route it supports.
Can the model run without an internet connection?
Yes. We install it inside your network through Jungroo AI Studio: on your servers, in your private cloud or on machines with no internet connection at all. Access is role-based and every request is logged inside your network.
What hardware does a small language model need?
It depends on the model's size and how many people use it. As a reference, a 30-billion-parameter-class model in FP8 runs on a single 80 GB GPU server, and small task models such as our open-source ReDefiNER run on an ordinary CPU. We size the model to the hardware you have.
How do you show the model is good enough?
We build the test first, from real cases your experts check, and run it on what you use today. Before go-live, the new model is compared with general-purpose models and your current service on the same test, and you see the results whichever way they go.
Do you work with systems integrators?
Yes. We can support a bid with a feasibility and data assessment, then build, test and deploy the model with the integrator's team inside the client's environment.
Next step
Tell us what the model needs to do.
Book a call with our team, or write to hello@jungroo.com. If you are a systems integrator preparing a bid, send us the client's brief and your timeline.
Contact
Tell us what you need.
We partner with a limited number of enterprises at a time, so every engagement gets senior attention. Tell us what you want to build, and our leadership will reply.