• Home
  • Cloud VPS
    • Hong Kong VPS
    • US VPS
    • Japan VPS
  • Dedicated Servers
    • Hong Kong Servers
    • US Servers
    • Japan Servers
    • Singapore Servers
  • Company
    • Contact Us
    • Blog
logo logo
  • Home
  • Cloud VPS
    • Hong Kong VPS
    • US VPS
    • Japan VPS
  • Dedicated Servers
    • Hong Kong Servers
    • US Servers
    • Japan Servers
    • Singapore Servers
  • Company
    • Contact Us
    • Blog
ENEN
  • 简体简体
  • 繁體繁體
Client Area

AI Chatbot on Hong Kong VPS: LangChain and Ollama Self-Hosted (2026)

October 9, 2026

Running your own AI chatbot on a Hong Kong VPS eliminates per-token OpenAI API costs for high-volume applications, keeps sensitive conversations off third-party servers, and delivers responses to mainland Chinese users via CN2 GIA without routing queries through US API endpoints. This guide combines Ollama for local LLM inference, pgvector for knowledge base retrieval, and LangChain for orchestrating a production-ready RAG chatbot.


Architecture

<code">
User message → FastAPI endpoint → LangChain orchestration
                                          │
                        ┌─────────────────┼─────────────────┐
                        │                 │                 │
                  pgvector              Ollama           Redis
                  (knowledge          (local LLM:       (session
                  retrieval)          Qwen2.5-7B)       history)
                        │                 │                 │
                        └─────────────────┼─────────────────┘
                                          │
                                    Response streamed
                                    to user

Step 1: Install Ollama

<code">curl -fsSL https://ollama.com/install.sh | sh
systemctl enable --now ollama

# Pull a bilingual model (English + Chinese)
ollama pull qwen2.5:7b          # 4.7 GB — excellent for 8+ GB RAM VPS
# ollama pull qwen2.5:3b        # 2.0 GB — for 4 GB RAM VPS
# ollama pull llama3.2:3b       # 2.0 GB — English-focused alternative

# Test
ollama run qwen2.5:7b "用中文介绍香港VPS的优势"

# Expose Ollama API to localhost only (default)
# /etc/systemd/system/ollama.service.d/override.conf:
cat > /etc/systemd/system/ollama.service.d/override.conf << 'EOF'
[Service]
Environment="OLLAMA_HOST=127.0.0.1:11434"
EOF
systemctl daemon-reload && systemctl restart ollama

Step 2: Knowledge Base with pgvector

<code"># Setup from existing pgvector guide — if not already installed:
sudo -u postgres psql -c "CREATE EXTENSION vector;" myapp_db

# Create chatbot knowledge table
sudo -u postgres psql myapp_db << 'SQL'
CREATE TABLE IF NOT EXISTS knowledge_base (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    source TEXT,
    content TEXT NOT NULL,
    embedding vector(768),   -- nomic-embed-text dimension
    metadata JSONB,
    created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_kb_embedding
    ON knowledge_base USING ivfflat (embedding vector_cosine_ops)
    WITH (lists = 50);
SQL

# Ingest your knowledge base (product docs, FAQs, policies)
pip install langchain langchain-community langchain-postgres \
  psycopg2-binary ollama --break-system-packages

python3 << 'PYTHON'
from langchain_community.embeddings import OllamaEmbeddings
from langchain_postgres import PGVector
from langchain_core.documents import Document

# Use nomic-embed-text for embeddings (runs locally on Ollama)
embeddings = OllamaEmbeddings(
    model="nomic-embed-text",
    base_url="http://localhost:11434"
)

# First pull the embedding model
import subprocess
subprocess.run(["ollama", "pull", "nomic-embed-text"])

store = PGVector(
    embeddings=embeddings,
    collection_name="knowledge_base",
    connection="postgresql://myapp_user:pass@localhost/myapp_db",
)

# Ingest documents
docs = [
    Document(
        page_content="Server.HK provides Hong Kong VPS with CN2 GIA routing to mainland China at 30-50ms latency.",
        metadata={"source": "product_info", "category": "network"}
    ),
    Document(
        page_content="All VPS plans include NVMe SSD storage, DDoS protection, and 24/7 support.",
        metadata={"source": "product_info", "category": "features"}
    ),
    # Add more documents...
]

store.add_documents(docs)
print("Knowledge base ingested successfully")
PYTHON

Step 3: RAG Chatbot with LangChain

<code"># chatbot.py
from langchain_community.llms import Ollama
from langchain_community.embeddings import OllamaEmbeddings
from langchain_postgres import PGVector
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.messages import HumanMessage, AIMessage
from langchain.chains import create_retrieval_chain
from langchain.chains.combine_documents import create_stuff_documents_chain
import redis
import json

# Redis for conversation history
r = redis.Redis(host='127.0.0.1', port=6379, decode_responses=True)

# Ollama LLM
llm = Ollama(
    model="qwen2.5:7b",
    base_url="http://localhost:11434",
    temperature=0.7,
    num_ctx=8192,  # Context window
)

# Embedding model
embeddings = OllamaEmbeddings(model="nomic-embed-text")

# Vector store retriever
store = PGVector(
    embeddings=embeddings,
    collection_name="knowledge_base",
    connection="postgresql://myapp_user:pass@localhost/myapp_db",
)
retriever = store.as_retriever(
    search_type="similarity",
    search_kwargs={"k": 4, "score_threshold": 0.7}
)

# Prompt template
system_prompt = """You are a helpful assistant for Server.HK, a Hong Kong VPS provider.
Answer questions based on the context below. Be concise and helpful.
If the context doesn't contain the answer, say so politely.

Context:
{context}

Previous conversation is provided for reference. Respond in the same language the user writes in."""

prompt = ChatPromptTemplate.from_messages([
    ("system", system_prompt),
    MessagesPlaceholder(variable_name="chat_history"),
    ("human", "{input}"),
])

# Build RAG chain
qa_chain = create_stuff_documents_chain(llm, prompt)

def get_history(session_id: str, max_turns: int = 10) -> list:
    """Retrieve conversation history from Redis."""
    raw = r.lrange(f"chat:{session_id}", -max_turns * 2, -1)
    messages = []
    for i in range(0, len(raw), 2):
        if i + 1 < len(raw): messages.append(HumanMessage(content=raw[i])) messages.append(AIMessage(content=raw[i + 1])) return messages def save_to_history(session_id: str, human: str, ai: str): """Save conversation turn to Redis.""" key = f"chat:{session_id}" r.rpush(key, human, ai) r.expire(key, 86400) # 24 hour session TTL async def chat(session_id: str, user_message: str) -> str:
    """Main chat function with RAG and conversation history."""
    # Retrieve relevant context
    docs = retriever.get_relevant_documents(user_message)
    context = "\n\n".join([d.page_content for d in docs])

    # Get conversation history
    history = get_history(session_id)

    # Generate response
    response = qa_chain.invoke({
        "input": user_message,
        "context": context,
        "chat_history": history,
    })

    # Save to history
    save_to_history(session_id, user_message, response)

    return response

Step 4: FastAPI Streaming Endpoint

<code"># api.py — streaming chat API
from fastapi import FastAPI, HTTPException
from fastapi.responses import StreamingResponse
from pydantic import BaseModel
from langchain_community.llms import Ollama
import httpx

app = FastAPI(title="HK VPS AI Chatbot API")

class ChatRequest(BaseModel):
    session_id: str
    message: str
    stream: bool = True

@app.post("/api/chat")
async def chat_endpoint(request: ChatRequest):
    if request.stream:
        return StreamingResponse(
            stream_response(request.session_id, request.message),
            media_type="text/event-stream"
        )
    else:
        response = await chat(request.session_id, request.message)
        return {"response": response, "session_id": request.session_id}

async def stream_response(session_id: str, message: str):
    """Stream tokens as they are generated."""
    docs = retriever.get_relevant_documents(message)
    context = "\n\n".join([d.page_content for d in docs])
    history = get_history(session_id)

    full_response = ""
    async for chunk in llm.astream(
        prompt.format_messages(
            input=message,
            context=context,
            chat_history=history,
        )
    ):
        full_response += chunk
        yield f"data: {json.dumps({'token': chunk})}\n\n"

    save_to_history(session_id, message, full_response)
    yield f"data: {json.dumps({'done': True})}\n\n"

@app.delete("/api/chat/{session_id}")
async def clear_history(session_id: str):
    r.delete(f"chat:{session_id}")
    return {"cleared": session_id}

if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="127.0.0.1", port=8000)

Step 5: Frontend Chat Widget

<code"># Simple HTML + JavaScript streaming chat widget
const chatWidget = `

 

`


Resource Requirements

ModelVPS RAMResponse speedQuality
Qwen2.5 3B4 GB15–30 tokens/secGood (basic queries)
Qwen2.5 7B8 GB8–15 tokens/secVery good (production)
Qwen2.5 14B16 GB4–8 tokens/secExcellent
Llama 3.2 3B4 GB20–40 tokens/secGood (English focus)

Conclusion

A self-hosted AI chatbot on a Hong Kong VPS with Ollama and LangChain delivers conversational AI for Asia-Pacific users at zero per-token cost — Qwen2.5-7B runs entirely on the VPS, responds to Chinese and English questions equally well, and retrieves relevant knowledge from your pgvector database for grounded, accurate answers. CN2 GIA routing ensures mainland Chinese users experience the streaming response without proxy latency. For customer support bots, internal knowledge assistants, or product recommendation chatbots, self-hosting on a HK VPS is commercially viable at moderate traffic volumes without cloud AI API costs.

Run your own AI: Browse Server.HK Hong Kong VPS plans — an 8 GB plan handles Qwen2.5-7B inference, pgvector, and FastAPI comfortably for a production chatbot workload.

Leave a Reply

Your email address will not be published. Required fields are marked *

Recent Posts

  • NGINX Unit on Hong Kong VPS: Universal Application Server (2026)
  • CockroachDB on Hong Kong VPS: Distributed SQL for Asia-Pacific (2026)
  • Matrix and Element Chat Server on Hong Kong VPS: Self-Hosted Slack (2026)
  • MediaWiki on Hong Kong VPS: Self-Hosted Wiki for Asia-Pacific (2026)
  • Temporal Workflow Engine on Hong Kong VPS: Durable Code for Asia (2026)

Recent Comments

  1. Cheap Proxies on Php.ini Configuration: session.sid_length
  2. full88vnz on Laravel Octane on Hong Kong VPS: 10x Faster PHP with Swoole (2026)
  3. airport transfer dubai airport to hotel on How to Set Up a WordPress Site on a Hong Kong VPS with aaPanel (Step-by-Step 2026)
  4. Alejandra768 on VPS Backup Strategy for Hong Kong: Restic, B2, and Encryption (2026)
  5. Valerie4779 on CockroachDB on Hong Kong VPS: Distributed SQL for Asia-Pacific (2026)

Knowledge Base

Access detailed guides, tutorials, and resources.

Live Chat

Get instant help 24/7 from our support team.

Submit Sales Ticket

Get custom quotes or bulk pricing from our sales team.

logo
Alipay Cc-paypal Cc-stripe Cc-visa Cc-mastercard Bitcoin
Cloud VPS
  • Hong Kong VPS
  • US VPS
  • Japan VPS
Dedicated Servers
  • Hong Kong Servers
  • US Servers
  • Japan Servers
  • Singapore Servers
More
  • Contact Us
  • Blog
  • Legal
© 2026 Server.HK | Hosting Limited, Hong Kong | Company Registration No. 77008912
Telegram
Telegram @ServerHKBot