LLL Inc. - Private LLM Infrastructure
Why a Private LLM Platform
Most AI adoption today runs through a public API: a prompt leaves your network, goes to a provider’s servers, and a response comes back. That’s the right tradeoff for a lot of work. It stops being the right tradeoff once the material going into the prompt is something you’d rather keep inside your own infrastructure — internal documents, customer data, source code, anything under a confidentiality obligation you can’t hand to a third party.
A private LLM platform is the alternative: a large language model running on hardware you control, or that LLL operates on your behalf, instead of a shared public service.
LLL builds and operates private LLM infrastructure as part of our own in-house environment — the same environment we use to demonstrate the platform to prospective clients before they commit to a build of their own. This is an extension of infrastructure we already run, not a new practice area we’re describing on paper for the first time.
Three Ways to Run It
There isn’t one “correct” deployment form. Each of the three has a usage level where it’s the cheapest option — none of them wins at every scale.
| Form | What it means |
|---|---|
| Self-hosted (on your premises) | GPU hardware is installed in your own building. The equipment is your capital asset from day one. |
| Managed hosting | LLL houses and operates the hardware on your behalf — power, cooling, UPS, connectivity, and monitoring included in a monthly fee. |
| Shared infrastructure | No hardware purchase at all. You rent GPU/VRAM capacity, metered by the unit, on infrastructure LLL already operates. |
Self-hosted tends to be cheapest at sustained, heavy, predictable usage, provided your building can support the electrical and cooling load (see Equipment Market Reality, below). Managed hosting removes the facilities burden without removing hardware ownership. Shared infrastructure removes both, at the cost of paying a per-unit rate rather than owning the underlying capacity. Which of the three actually pencils out for a given workload is exactly the question a 30-day PoC is built to answer — not something we recommend by default before seeing your numbers.
What’s Included in a Full Build
A full build for a single use case is scoped around four elements:
- Environment build: provisioning and configuring the inference stack on the chosen deployment form
- RAG (Retrieval-Augmented Generation): connecting the model to your own documents and knowledge base, so answers are grounded in your material rather than the model’s general training
- Integration with existing systems: wiring the platform into the tools your team already uses, rather than shipping a standalone chat window nobody opens
- Training: bringing your team up to speed on how to use and, where relevant, operate the platform
Recommended Model Configuration
Weight figures below are FP8 — see the quantization note underneath the table.
| Class | Format | Weights (FP8) | Positioning |
|---|---|---|---|
| 32B-class | Dense | ~32GB | Practical floor for business use |
| 70B-class | Dense | ~70GB | Default and recommended — one machine, sized to approximate a company-wide ChatGPT-equivalent |
| 122B-A10B | MoE (10B active) | ~125GB | For higher concurrent usage |
| 400B-class | MoE (17B active) | ~400GB | Top of the range we currently deploy |
| 35B-A3B | MoE (3B active) | ~35GB | Triage / guardrail role only — not sized or positioned as an answering model |
Quantization floor: FP8. Our standard configurations do not go below FP8 (for example, 4-bit/INT4 quantization). In our testing, quantizing further measurably degrades accuracy on Japanese-language business tasks, so it isn’t something we ship as a default recommendation regardless of the cost savings it might offer on paper.
Investment
Prices below are in USD and reflect what we quote clients directly — not internal cost figures.
| Engagement | Price | Notes |
|---|---|---|
| 30-day PoC | $6,000 ($3,000 for teams of 50 or fewer) | Validates feasibility against your own workload before any hardware commitment. Fully credited toward the full build if you continue — see 30-Day Private LLM PoC |
| Full build (single use case) | $8,000–$22,000, scale-dependent | Environment build, RAG, integration with existing systems, and training (see above) |
| Ongoing maintenance & operations | $300–$1,300/month | Tiered by GPU count |
| Managed hosting | $300–$1,600/month | Tiered by power draw. Power, cooling, UPS, connectivity, and monitoring included |
| Shared infrastructure | From $300/month per 24GB VRAM unit | No hardware purchase |
Equipment Market Reality
GPU and server-memory pricing has been rising sharply, and on current market signals both are expected to stay tight through 2027 — driven by AI-related demand rather than a temporary shortage. We don’t publish a fixed equipment price on this page: hardware pricing is moving quickly enough right now that a specific number published today would already be dated by the time you read it, and would vary by region and procurement channel regardless.
As a rough sense of scale: a single-GPU or few-GPU setup — enough to run the 32B floor model described above — sits at a meaningfully different order of magnitude than a full 8-GPU build sized for the 70B-class default, which sits in the low six figures, in USD, as a complete build. A multi-node configuration sized for the 400B-class tier is a further step up again. Each tier moves the equipment investment into a materially higher bracket, which is part of why the 30-Day PoC exists: to establish which tier your workload actually needs before pricing any of them.
Memory is the second-largest line item after the GPUs themselves — a system built around 768GB of VRAM typically needs more than 1TB of system memory to support it, which is part of why memory pricing has moved about as fast as GPU pricing has. For current equipment pricing against your actual sourcing region and configuration, get in touch rather than relying on a number that may already be stale.
What We’re Honest About Upfront
Power, cooling, UPS, backup power, monitoring, and noise control vary enormously from one building to another, so we don’t quote them as a fixed number in a proposal — they’re priced after a site visit, individually.
As a general reference point: air cooling is workable up to roughly 20kW per rack; beyond that, liquid cooling or facility changes typically become necessary. Even changes that look minor on paper — adding electrical distribution capacity, for instance — have run into real, non-trivial cost on some sites. We’d rather tell you that a facilities budget line exists and needs a site visit than hand you a number we can’t stand behind.
Where Your Data Actually Goes
“Private” describes where inference runs, not your network architecture. With a self-hosted or managed deployment, the prompt and the generated response are processed on servers inside your own infrastructure — they are not sent to a public, general-purpose AI service. That’s a statement about where the computation happens.
It isn’t a claim about your network perimeter. VPNs, firewalls, access control, and everything else about how that infrastructure is secured on the network layer is a separate design decision, one we can advise on but don’t bundle into this service by default.
Model, Data, and Tuning Result Ownership
Infrastructure projects raise the same ownership question application projects do, extended to what’s specific to an LLM platform: not just the surrounding code, but the tuned model artifacts, the retrieval index, and the evaluation results.
- Tuning and RAG artifacts are yours: any fine-tuned weights, prompt/retrieval configuration, and evaluation datasets produced during a build are handed over as deliverables — not retained on infrastructure only LLL controls, which would make them a dependency rather than an asset.
- No model lock-in: builds are designed around a standard, OpenAI-compatible inference interface (see below), so the underlying model can be swapped — a different open-weight model, a different GPU vendor, a different hosting arrangement — without rewriting your integration layer.
- No provider lock-in: the same standard inference stack is used across self-hosted, managed, and shared deployments. Moving from one deployment form to another later doesn’t require re-architecting how your applications talk to the model.
- Your data stays governed by you: documents and prompts used for evaluation and RAG remain under your control throughout — consistent with the data-locality note above.
This is the same anti-lock-in principle LLL applies across every engagement: model-agnostic design, a standard inference stack, and a build you can move to a different vendor if you ever choose to, not one that depends on LLL staying involved to keep running.
Choosing the Right Deployment Form
| Question | Points toward |
|---|---|
| Do you need the hardware to be a capital asset you own — for audit, security policy, or accounting reasons? | Self-hosted |
| Do you want the capability without taking on facilities work (power, cooling, UPS, monitoring)? | Managed hosting |
| Is usage light, unpredictable, or still being proven out? | Shared infrastructure |
| Is usage heavy and sustained, and can your facility support the load? | Self-hosted often wins at that scale |
None of these is a universal answer — they’re starting points. The right one for a specific workload, along with the right model size, is what the 30-day PoC below is designed to determine with your own numbers rather than a rule of thumb.
Getting Started
Committing a six-figure equipment budget, or a multi-year managed or shared contract, before knowing whether a private LLM actually meets your accuracy and latency bar is a real risk — one a proof of concept exists specifically to remove.
The 30-Day Private LLM PoC runs on infrastructure LLL already operates, tests accuracy and latency against your own documents and workload (not a generic benchmark), and ends with a written report: which deployment form fits, which model tier fits, and a cost estimate for the full build. If you continue, the PoC fee is credited in full toward that build.
FAQs
Q: Do we have to buy hardware to find out if this works for us? A: No. The 30-day PoC runs on infrastructure LLL already builds and operates in-house. The report it produces tells you what to buy — or rent, or have us host — if you decide to continue.
Q: Is 70B always the right model size? A: It’s our default and recommended starting point, sized to approximate a company-wide ChatGPT-equivalent on one machine. Workloads with lower accuracy requirements can run on the 32B floor; workloads with heavy concurrent usage may need the 122B-A10B or 400B-class MoE tiers. The PoC report recommends a specific tier with reasoning, not a default.
Q: Can we quantize below FP8 to save on hardware? A: You can, but we don’t recommend it as a default. Our testing shows a measurable accuracy drop on Japanese-language business tasks below FP8, so our standard configurations hold the line there even where it would lower the hardware bill.
Q: Are we locked into LLL, or into a specific model or GPU vendor, once this is built? A: No, by design. The inference layer is a standard, OpenAI-compatible interface, so the model and the hosting arrangement underneath it can change without rewriting your integration — see Model, Data, and Tuning Result Ownership, above.
Q: What’s not included in the prices on this page? A: Facility-specific work — power, cooling, UPS, backup, monitoring, and noise control — which varies too much by building to price without a site visit. See “What We’re Honest About Upfront,” above, for the reference numbers we can share before that visit.
Related Resources
- 30-Day Private LLM PoC — the fixed-price way to start
- AI-Powered Development Services — application-layer AI work that can sit on top of this infrastructure
- AI Integration — adding an AI feature to an existing product
Start the Conversation
Ready to find out whether a private LLM platform fits your workload?
LLL Inc. - Private LLM Infrastructure from Malaysia Self-Hosted • Managed • Shared — No Lock-In