Running your own AI chatbot on a Hong Kong VPS eliminates per-token OpenAI API costs for high-volume applications, keeps sensitive conversations off third-party servers, and delivers responses to mainland Chinese users via CN2 GIA without routing queries through US API endpoints. This guide combines Ollama for local LLM inference, pgvector for knowledge base retrieval, and LangChain for orchestrating a production-ready RAG chatbot.
Architecture
<code">
User message → FastAPI endpoint → LangChain orchestration
│
┌─────────────────┼─────────────────┐
│ │ │
pgvector Ollama Redis
(knowledge (local LLM: (session
retrieval) Qwen2.5-7B) history)
│ │ │
└─────────────────┼─────────────────┘
│
Response streamed
to user
Step 1: Install Ollama
<code">curl -fsSL https://ollama.com/install.sh | sh systemctl enable --now ollama # Pull a bilingual model (English + Chinese) ollama pull qwen2.5:7b # 4.7 GB — excellent for 8+ GB RAM VPS # ollama pull qwen2.5:3b # 2.0 GB — for 4 GB RAM VPS # ollama pull llama3.2:3b # 2.0 GB — English-focused alternative # Test ollama run qwen2.5:7b "用中文介绍香港VPS的优势" # Expose Ollama API to localhost only (default) # /etc/systemd/system/ollama.service.d/override.conf: cat > /etc/systemd/system/ollama.service.d/override.conf << 'EOF' [Service] Environment="OLLAMA_HOST=127.0.0.1:11434" EOF systemctl daemon-reload && systemctl restart ollama
Step 2: Knowledge Base with pgvector
<code"># Setup from existing pgvector guide — if not already installed:
sudo -u postgres psql -c "CREATE EXTENSION vector;" myapp_db
# Create chatbot knowledge table
sudo -u postgres psql myapp_db << 'SQL'
CREATE TABLE IF NOT EXISTS knowledge_base (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
source TEXT,
content TEXT NOT NULL,
embedding vector(768), -- nomic-embed-text dimension
metadata JSONB,
created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_kb_embedding
ON knowledge_base USING ivfflat (embedding vector_cosine_ops)
WITH (lists = 50);
SQL
# Ingest your knowledge base (product docs, FAQs, policies)
pip install langchain langchain-community langchain-postgres \
psycopg2-binary ollama --break-system-packages
python3 << 'PYTHON'
from langchain_community.embeddings import OllamaEmbeddings
from langchain_postgres import PGVector
from langchain_core.documents import Document
# Use nomic-embed-text for embeddings (runs locally on Ollama)
embeddings = OllamaEmbeddings(
model="nomic-embed-text",
base_url="http://localhost:11434"
)
# First pull the embedding model
import subprocess
subprocess.run(["ollama", "pull", "nomic-embed-text"])
store = PGVector(
embeddings=embeddings,
collection_name="knowledge_base",
connection="postgresql://myapp_user:pass@localhost/myapp_db",
)
# Ingest documents
docs = [
Document(
page_content="Server.HK provides Hong Kong VPS with CN2 GIA routing to mainland China at 30-50ms latency.",
metadata={"source": "product_info", "category": "network"}
),
Document(
page_content="All VPS plans include NVMe SSD storage, DDoS protection, and 24/7 support.",
metadata={"source": "product_info", "category": "features"}
),
# Add more documents...
]
store.add_documents(docs)
print("Knowledge base ingested successfully")
PYTHONStep 3: RAG Chatbot with LangChain
<code"># chatbot.py
from langchain_community.llms import Ollama
from langchain_community.embeddings import OllamaEmbeddings
from langchain_postgres import PGVector
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.messages import HumanMessage, AIMessage
from langchain.chains import create_retrieval_chain
from langchain.chains.combine_documents import create_stuff_documents_chain
import redis
import json
# Redis for conversation history
r = redis.Redis(host='127.0.0.1', port=6379, decode_responses=True)
# Ollama LLM
llm = Ollama(
model="qwen2.5:7b",
base_url="http://localhost:11434",
temperature=0.7,
num_ctx=8192, # Context window
)
# Embedding model
embeddings = OllamaEmbeddings(model="nomic-embed-text")
# Vector store retriever
store = PGVector(
embeddings=embeddings,
collection_name="knowledge_base",
connection="postgresql://myapp_user:pass@localhost/myapp_db",
)
retriever = store.as_retriever(
search_type="similarity",
search_kwargs={"k": 4, "score_threshold": 0.7}
)
# Prompt template
system_prompt = """You are a helpful assistant for Server.HK, a Hong Kong VPS provider.
Answer questions based on the context below. Be concise and helpful.
If the context doesn't contain the answer, say so politely.
Context:
{context}
Previous conversation is provided for reference. Respond in the same language the user writes in."""
prompt = ChatPromptTemplate.from_messages([
("system", system_prompt),
MessagesPlaceholder(variable_name="chat_history"),
("human", "{input}"),
])
# Build RAG chain
qa_chain = create_stuff_documents_chain(llm, prompt)
def get_history(session_id: str, max_turns: int = 10) -> list:
"""Retrieve conversation history from Redis."""
raw = r.lrange(f"chat:{session_id}", -max_turns * 2, -1)
messages = []
for i in range(0, len(raw), 2):
if i + 1 < len(raw): messages.append(HumanMessage(content=raw[i])) messages.append(AIMessage(content=raw[i + 1])) return messages def save_to_history(session_id: str, human: str, ai: str): """Save conversation turn to Redis.""" key = f"chat:{session_id}" r.rpush(key, human, ai) r.expire(key, 86400) # 24 hour session TTL async def chat(session_id: str, user_message: str) -> str:
"""Main chat function with RAG and conversation history."""
# Retrieve relevant context
docs = retriever.get_relevant_documents(user_message)
context = "\n\n".join([d.page_content for d in docs])
# Get conversation history
history = get_history(session_id)
# Generate response
response = qa_chain.invoke({
"input": user_message,
"context": context,
"chat_history": history,
})
# Save to history
save_to_history(session_id, user_message, response)
return responseStep 4: FastAPI Streaming Endpoint
<code"># api.py — streaming chat API
from fastapi import FastAPI, HTTPException
from fastapi.responses import StreamingResponse
from pydantic import BaseModel
from langchain_community.llms import Ollama
import httpx
app = FastAPI(title="HK VPS AI Chatbot API")
class ChatRequest(BaseModel):
session_id: str
message: str
stream: bool = True
@app.post("/api/chat")
async def chat_endpoint(request: ChatRequest):
if request.stream:
return StreamingResponse(
stream_response(request.session_id, request.message),
media_type="text/event-stream"
)
else:
response = await chat(request.session_id, request.message)
return {"response": response, "session_id": request.session_id}
async def stream_response(session_id: str, message: str):
"""Stream tokens as they are generated."""
docs = retriever.get_relevant_documents(message)
context = "\n\n".join([d.page_content for d in docs])
history = get_history(session_id)
full_response = ""
async for chunk in llm.astream(
prompt.format_messages(
input=message,
context=context,
chat_history=history,
)
):
full_response += chunk
yield f"data: {json.dumps({'token': chunk})}\n\n"
save_to_history(session_id, message, full_response)
yield f"data: {json.dumps({'done': True})}\n\n"
@app.delete("/api/chat/{session_id}")
async def clear_history(session_id: str):
r.delete(f"chat:{session_id}")
return {"cleared": session_id}
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="127.0.0.1", port=8000)Step 5: Frontend Chat Widget
<code"># Simple HTML + JavaScript streaming chat widget const chatWidget = `
`
Resource Requirements
| Model | VPS RAM | Response speed | Quality |
|---|---|---|---|
| Qwen2.5 3B | 4 GB | 15–30 tokens/sec | Good (basic queries) |
| Qwen2.5 7B | 8 GB | 8–15 tokens/sec | Very good (production) |
| Qwen2.5 14B | 16 GB | 4–8 tokens/sec | Excellent |
| Llama 3.2 3B | 4 GB | 20–40 tokens/sec | Good (English focus) |
Conclusion
A self-hosted AI chatbot on a Hong Kong VPS with Ollama and LangChain delivers conversational AI for Asia-Pacific users at zero per-token cost — Qwen2.5-7B runs entirely on the VPS, responds to Chinese and English questions equally well, and retrieves relevant knowledge from your pgvector database for grounded, accurate answers. CN2 GIA routing ensures mainland Chinese users experience the streaming response without proxy latency. For customer support bots, internal knowledge assistants, or product recommendation chatbots, self-hosting on a HK VPS is commercially viable at moderate traffic volumes without cloud AI API costs.
Run your own AI: Browse Server.HK Hong Kong VPS plans — an 8 GB plan handles Qwen2.5-7B inference, pgvector, and FastAPI comfortably for a production chatbot workload.