Private ChatGPT Server for Companies: 10, 50 & 200 Users

Quick answer: A private ChatGPT-style server needs enough GPU memory to hold the model plus a separate KV cache for every request being answered at the same moment. Using published model configurations, that cache is about 1.5 GiB (Qwen2.5 14B), 2 GiB (Qwen2.5 32B) or 2.5 GiB (Llama 3.3 70B) per 8,192-token request. As a memory-fit estimate, one 48 GB card holds a 32B model at 4-bit quantization with room for about 11 simultaneous requests, two 48 GB cards hold a 70B model for about 17, and four 48 GB cards for about 51. Team size matters less than how many people are mid-request at once, so measure that in a pilot before you size for 200 users.

This guide is for IT managers, CTOs and founders in India who want an internal AI assistant running on hardware the company owns. Nothing here is a ProStation benchmark: ProStation Systems publishes no tokens-per-second figures and no list prices, so the numbers below are memory arithmetic from published model files and configurations. The mapping from team size to simultaneous requests is an illustrative assumption you should replace with your own measurement. Sources are linked at the end. Last updated 20 October 2026.

What Is a Private ChatGPT (Private LLM Server)?

A private ChatGPT is an AI chat assistant that runs on a server you control instead of a hosted service. Staff open a web page, sign in and chat, but the model, the prompts and any uploaded documents stay on your hardware. It has five parts:

• The model. An open-weight large language model such as Qwen, Llama or DeepSeek, which is a file you download.
• The model server. Software that loads the model onto the GPUs and answers requests, for example Ollama or vLLM.
• The chat interface. A web front end with logins and permissions, for example Open WebUI.
• Document search (optional). Retrieval over company files so answers can use internal documents.
• The hardware. GPUs for the model, and the CPU, ECC memory, fast storage, power and cooling around them.

Open-weight models are not the same product as the hosted models behind ChatGPT. They are different models with different strengths, so test them on your own tasks before you commit to hardware. For the model memory side in detail, see our guide to the best GPU for local LLMs.

Why Do Indian Companies Run Their Own LLM Server?

The usual reason is control over where prompts and documents go. Four points are worth knowing:

• Data location. Prompts, uploads and chat history sit on your own machine, so the question "where does our data go?" has a simple answer.
• The DPDP Act. India's Digital Personal Data Protection Act, 2023 sets penalty ceilings in its Schedule: up to 250 crore rupees for failing to take reasonable security safeguards (section 8(5)) and up to 200 crore rupees for failing to give notice of a personal data breach (section 8(6)). These are maximums that the Data Protection Board decides after a hearing, not fixed fines. A private server does not make a company compliant by itself, because you still have to secure and govern the system. This is not legal advice; ask your counsel how the Act applies to your data.
• Cost shape. A private server has no per-token charge from a provider. You pay for hardware, power and administration time instead, so it suits steady daily use better than occasional use. Our on-prem AI server versus cloud GPU guide explains the break-even logic.
• Model control. You choose and pin the model version, so the assistant's behaviour does not change underneath you.

How Many Users Can One GPU Serve? The Sizing Method

Named users is the wrong unit. A model only needs extra memory for the requests being answered right now, so size for concurrent requests: the number of people who have a question in flight at the same moment. The memory rule is:

GPU memory needed = model weights + (KV cache per request x concurrent requests) + headroom.

The weights are the model's download size. Ollama's library lists Q4_K_M sizes of 9.0 GB for Qwen2.5 14B, 20 GB for Qwen2.5 32B and 43 GB for Llama 3.3 70B, and Q8_0 sizes of 16, 35 and 75 GB. The KV cache stores attention keys and values for every token of every active request. Per token it is 2 x layers x KV heads x head size x 2 bytes at the default f16 setting, and each model's published configuration supplies the numbers:

Model	KV cache per token (f16)	Per request at 4,096 tokens	Per request at 8,192 tokens	Per request at 32,768 tokens
Qwen2.5 14B (48 layers, 8 KV heads, head size 128)	192 KiB	0.75 GiB	1.5 GiB	6 GiB
Qwen2.5 32B (64 layers, 8 KV heads, head size 128)	256 KiB	1 GiB	2 GiB	8 GiB
Llama 3.3 70B (80 layers, 8 KV heads, head size 128)	320 KiB	1.25 GiB	2.5 GiB	10 GiB

Ollama's FAQ says it uses a 4,096-token context window by default and that parallel request processing increases the context size by the number of parallel requests. It also says OLLAMA_KV_CACHE_TYPE defaults to f16 and can be set to q8_0, which uses about half the memory with very little quality loss. A long context is therefore expensive: a single 32,768-token request on a 70B model needs about 10 GiB of cache. Most office chat and document tasks fit comfortably in 8,192 tokens, so set the limit deliberately.

For the capacity table below I made three assumptions, stated so you can change them: an 8,192-token context per request, an f16 cache, and 10% of total GPU memory kept as headroom for the runtime. I also treated Ollama's GB figures as GiB, which slightly overstates the weights, the safe direction.

What Hardware Do 10, 50 and 200 Users Need?

This table shows how many simultaneous 8,192-token requests fit in memory. It says nothing about speed: when many requests are generated at once, each one slows down, and ProStation publishes no tokens-per-second figures. Treat the numbers as an upper limit on memory and test speed on the real machine.

GPU memory in the server	Qwen2.5 14B at Q4_K_M (9.0 GB)	Qwen2.5 32B at Q4_K_M (20 GB)	Llama 3.3 70B at Q4_K_M (43 GB)	Llama 3.3 70B at Q8_0 (75 GB)
1 x 24 GB card	about 8	under 1 (weights fit, almost no room for a request)	does not fit	does not fit
1 x 48 GB card	about 22	about 11	under 1 (weights leave almost nothing)	does not fit
2 x 48 GB cards (96 GB)	about 51	about 33	about 17	about 4
4 x 48 GB cards (192 GB)	about 109	about 76	about 51	about 39

The multi-GPU rows assume the runtime splits the model and its cache across the cards, as Ollama does when a model does not fit on one GPU and as vLLM does with tensor parallelism. Real overhead and uneven splits will lower these figures.

From team size to concurrent requests. As an illustrative assumption of mine, not an industry statistic, take 1 in 5 staff as having a request in flight at the busiest moment. That gives about 2 concurrent requests for 10 users, 10 for 50 users and 40 for 200 users. Your real ratio may be lower or higher, which is what the pilot below measures.

10 users (about 2 concurrent requests)

One 24 GB-class card running a 14B model at 4-bit has memory for about 8 requests, well above 2. If staff need the stronger 32B model, step up to one 48 GB card, which fits about 11 requests. Starting here is a low-risk way to learn what your people really ask. For shared multi-person GPU use in a smaller team, the shared GPU workstation guide covers the access side.

50 users (about 10 concurrent requests)

One 48 GB card with a 32B model fits about 11 requests, which is just above 10 with almost no slack. Two 48 GB cards give about 33 requests with a 32B model or about 17 with a 70B model. Above 50 users, check the Open WebUI branding terms described below.

200 users (about 40 concurrent requests)

Four 48 GB cards fit about 51 requests with a 70B model at 4-bit or about 76 with a 32B model. Four GPUs is the top of ProStation's published AI and ML configuration, so beyond that point you would add a second server and spread users across them. At this size, consider vLLM, which is built around serving many requests together, and decide whether one server being unavailable is acceptable. Our server downtime cost guide shows how to put a number on that.

Ollama or vLLM for Multiple Users?

Both are model servers. This comparison uses their own documentation and compares features, not speed.

Question	Ollama	vLLM
What it is	A local model runner with simple setup	An HTTP server with OpenAI-style endpoints such as /v1/chat/completions
Handling several users	Runs parallel requests per model; OLLAMA_NUM_PARALLEL defaults to 1 and parallel requests grow the context memory by that number; queues up to 512 requests by default, then rejects	Continuous batching of incoming requests and PagedAttention memory management, per its README
More than one GPU	Loads on a single GPU if the model fits there, otherwise spreads it across all available GPUs	Tensor, pipeline, data, expert and context parallelism for distributed inference
Best fit	A pilot, a small team, easy administration	Larger teams and multi-GPU servers where many requests arrive together
Watch out for	Set the parallel value on purpose and recompute memory	More settings to learn; read the documentation for your version

A common path is to pilot on Ollama and move to vLLM if concurrency grows. The chat interface does not have to change when you switch, because Open WebUI lists Ollama and any OpenAI-compatible provider as connections, and vLLM serves an OpenAI-compatible API.

Which Chat Interface and Logins Should Staff Use?

Open WebUI is an open-source, self-hosted chat interface. Its features page lists roles, groups and per-resource permissions, the ability to restrict models to specific users or groups, federated sign-in through SSO, OIDC and LDAP, and knowledge bases that let the AI retrieve information from uploaded files.

Read its licence before you deploy. The licence page says that from version 0.6.6 (19 April 2025) a branding-protection clause applies: you may not alter, remove or obscure the Open WebUI branding, except for deployments of 50 or fewer users in any 30-day window or under an enterprise licence. Deploying to any number of internal users with the branding intact is allowed. That makes the 50-user line relevant to the sizing above. Other chat front ends exist, and each has its own licence to check.

Which Models Can a Company Use Commercially?

Licences differ by model and by version. This table summarizes what each publisher states; read the full text for the exact model you download, and ask your counsel about internal use.

Model	Licence as stated by the publisher	What to check before company use
Qwen2.5 32B Instruct	Apache-2.0 (model card)	Standard Apache terms; keep the licence notices
Qwen2.5 72B Instruct	The model card tags the licence as Qwen	Read the Qwen licence text itself before commercial use
DeepSeek-R1	MIT for code and weights (model card)	The distilled models inherit their base licences: Qwen-based ones are Apache 2.0, and the 70B distill comes from Llama 3.3 70B Instruct under the Llama 3.3 licence
Llama 3.3 70B	Llama 3.3 Community License (Meta)	A 700 million monthly active user threshold; Built with Llama attribution when distributing or making the materials available; models you distribute built from it must start with Llama; the Acceptable Use Policy applies

How Do You Keep a Private LLM Server Secure?

Ollama's FAQ says it binds to 127.0.0.1 port 11434 by default and that you change this with OLLAMA_HOST. It gives no guidance on authentication for network exposure, so do not open the model port to the office network or the internet. The following is standard good practice rather than anything specific to one vendor:

• Keep the model server on localhost or a private network segment, so staff reach only the chat interface.
• Use company sign-in (SSO or LDAP) where the chat interface supports it, and limit models and document collections by group.
• Put HTTPS in front, even on the office LAN, and use a VPN for remote staff instead of exposing the service publicly.
• Decide how long chats are kept and who may read them, because prompts can contain personal data.
• Back up the chat database and the document index, and plan updates for the model server, the interface and the operating system.
• Protect the machine against power loss; our UPS sizing guide explains how to size a UPS.

What Else Does the Server Need Besides GPUs?

• Power. NVIDIA lists 300 W maximum power for the RTX A6000, so four of them draw 1,200 W before the CPUs, drives and fans. NVIDIA's page for a single GeForce RTX 4090 lists a minimum 850 W system power supply. See our dual power supply guide for redundancy.
• Cooling. Several GPUs in one chassis put out a lot of heat; the cooling and thermal throttling guide covers airflow.
• System memory. ECC DDR4 keeps a long-running service stable. System RAM helps only as overflow: Ollama's ollama ps command reports how much of a model runs on the CPU instead of the GPU, and that part is slower than GPU memory.
• Storage. Model files are large, at 43 GB for one 70B model at Q4_K_M and 75 GB at Q8_0, and a team often keeps several models plus a document index. Fast NVMe storage suits this; see the server storage guide.
• Network. Chat text itself is small, so this is our reasoning rather than a measured figure: 1GbE is usually enough for the chat, and faster links matter mainly for document shares and backups. The networking guide compares the options.
• CPU and PCIe. Multi-GPU builds need enough PCIe lanes and a balanced CPU; the CPU versus GPU balance guide explains how to check.

Which ProStation Build Fits Each Size?

ProStation Systems builds brand-new custom workstations and tower servers in India. These are the tiers as published on the servers page:

ProStation tier	CPU	ECC DDR4 RAM	GPU (as published on the site)	Fit for a private LLM server
Starter	Intel Xeon E	16 to 64 GB	Optional	Not the model host; suited to other office services
Pro	Intel Xeon Scalable	64 to 256 GB	Optional NVIDIA RTX	A one-GPU pilot or the 10-user case
Ultra	Dual Intel Xeon Scalable	256 to 512 GB	NVIDIA RTX / A-series / Tesla; the AI and ML configuration lists 1 to 4 GPUs	The 50-user and 200-user cases

All tiers use Intel Xeon processors and ECC DDR4 memory, ship in about 4 working days after configuration, and are burn-in tested for 48 hours or more (see how every build is stress-tested). Warranty is one year on hardware with extended cover available as a paid option, and the warranty page has the terms. Support is available 24/7.

What ProStation does and does not publish matters here. The site lists the RTX, A-series and Tesla GPU families but no specific model or price, and it publishes no benchmark figures. The exact GPU model, including any 48 GB-class card, is confirmed in a written quote. The practical advantage of a configurable build is that the GPU memory can be chosen from your model, context length and concurrency figures instead of from a fixed catalogue. Tell us those figures through free consulting, or build a configuration in the configurator. See also the AI and ML use case page, the AI and ML labs page and, for regulated or data-sensitive teams, the healthcare, fintech and enterprise and SMB pages.

If budget is the constraint, ProStation's sister brand Serverwale sells tested refurbished GPU servers. The new versus refurbished page compares the routes; ask for warranty terms in writing either way. For GPU choice by workload, read the GPU server spec guide and the AI and ML workstation GPU guide, and for fine-tuning rather than serving, the fine-tuning VRAM guide.

When Is a Private LLM Server the Wrong Choice?

• Only a handful of occasional users. A pay-per-use service may cost less than owning a server; compare the two in the on-premise, colocation and cloud guide.
• You need the quality of the strongest hosted models. Open-weight models may not match them on every task.
• Nobody will own it. Updates, access control, backups and logs need a named owner.
• The data is not sensitive and usage is unpredictable. The control benefit is smaller then.

A simple test: if you cannot name the person who will administer the server, start with a pilot rather than a purchase for 200 users. If you are deciding where the machine will sit, the tower versus rack guide compares the two form factors.

How Should You Run a Pilot Before Buying for 200 Users?

1. Pick two or three real tasks, such as policy questions, drafting or code help, and the candidate models. Check each licence.
2. Start with the smallest GPU setup that holds the model plus 4 to 8 requests, using the table above.
3. Give it to a pilot group of 10 to 20 people and keep simple usage logs, counts and timestamps, within your privacy rules.
4. Measure peak simultaneous requests, the typical prompt and answer length, and how long answers take in the busiest hour.
5. Recompute the memory with the sizing method using your measured context length and concurrency.
6. Add GPUs or a second server only where the measurements show a need.

Frequently Asked Questions

Q1. How much GPU memory does a private ChatGPT need for 50 employees?
It depends on how many people are mid-request at once, not on headcount. If 1 in 5 of 50 staff has a request in flight at the busiest moment (an illustrative assumption, not a measured figure), that is about 10 concurrent requests. A Qwen2.5 32B model at Q4_K_M has 20 GB of weights, and each 8,192-token request needs about 2 GiB of KV cache, so 10 requests need about 40 GiB before headroom. One 48 GB card fits that narrowly; two 48 GB cards leave comfortable room. Run a pilot to replace the assumption with your own measurement.

Q2. Can I run Llama 3.3 70B for my company on-premise?
Yes, if you have the memory and accept the licence. The Q4_K_M download is 43 GB on Ollama, so a single 48 GB card is left with almost no room for requests. Two 48 GB cards fit about 17 simultaneous 8,192-token requests by our memory arithmetic. The Llama 3.3 Community License has a 700 million monthly active user clause, a Built with Llama attribution requirement when you distribute or make the materials available, and an Acceptable Use Policy. Ask your counsel whether purely internal use triggers the attribution rule.

Q3. Should I use Ollama or vLLM for many users?
Ollama is simpler to set up and suits a pilot or a small team. Its documentation says OLLAMA_NUM_PARALLEL defaults to 1 and that parallel requests increase the context memory by the number of parallel requests. vLLM's README lists continuous batching of incoming requests, PagedAttention memory management and tensor parallelism, which are features aimed at serving many requests on one or more GPUs. This compares documented features, not speed; test both on your own hardware.

Q4. Is Open WebUI free to use in a company?
Its licence page says you may deploy it to any number of internal users as long as the Open WebUI branding stays in place. Branding can be removed only for deployments of 50 or fewer users in any 30-day window, or with an enterprise licence. This applies from version 0.6.6 (19 April 2025); earlier code is BSD-3-Clause. Check the current licence page before you deploy, especially above 50 users.

Q5. Does a private LLM server make us compliant with India's DPDP Act?
No, not by itself. The Digital Personal Data Protection Act, 2023 sets penalty ceilings in its Schedule, up to 250 crore rupees for failing to take reasonable security safeguards and up to 200 crore rupees for failing to notify a breach, decided by the Data Protection Board after a hearing. A private server keeps prompts and documents on hardware you control, but you still have to secure it, control access and decide how long chats are kept. This is not legal advice, so confirm with your counsel.

Q6. Does ProStation publish tokens-per-second benchmarks or prices for AI servers?
No. The ProStation site publishes configuration tiers and component families, not benchmark figures or list prices. That is why this guide uses memory arithmetic and asks you to measure speed in a pilot. Tell us the model, quantization, context length and the number of simultaneous users through free consulting and we size the GPU memory and quote in writing, including the exact GPU model.

Q7. What if our usage outgrows one server?
Add a second server and spread users across them, or move to a serving setup built for many requests. The AI and ML configuration on ProStation's use-case page lists 1 to 4 NVIDIA RTX or A-series GPUs per build, so four cards is the practical ceiling for one machine. Also decide in advance whether one server being unavailable is acceptable, because a shared assistant becomes something staff rely on.

Final Recommendation

Size a private ChatGPT from concurrent requests, not headcount. Take the model's weights, add about 1.5 to 2.5 GiB of KV cache for each simultaneous 8,192-token request, keep 10% headroom, and pick the GPU memory that covers it. In our memory-fit arithmetic that is one 24 GB to 48 GB card for about 10 users, one or two 48 GB cards for about 50, and up to four 48 GB cards for about 200. Pilot first, measure the real concurrency and answer times, then scale. If you want help matching the memory to your models, free consulting is the quickest route, or contact us with your model, context length and user count.

Sources: Ollama FAQ (context default, KV cache types, parallel requests, GPU placement, OLLAMA_HOST) and Ollama library tag pages for Qwen2.5 and Llama 3.3 (download sizes, retrieved 19 to 20 October 2026); vLLM README and vLLM online serving documentation; Open WebUI features and licence; model configurations and cards on Hugging Face for Qwen2.5 14B Instruct, Qwen2.5 32B Instruct, Qwen2.5 72B Instruct, Llama 3.3 70B Instruct (a mirror of Meta's configuration) and DeepSeek-R1; the Llama 3.3 Community License; the Schedule of the Digital Personal Data Protection Act, 2023 as reproduced at dpdpa.com (confirm against the official Gazette text); NVIDIA product pages for the RTX A6000 and GeForce RTX 4090. The KV cache and capacity figures are our own arithmetic from these configurations, using the assumptions stated in the sizing section. Nothing in this post is legal advice.