WordPressAIdeveloper

WordPress Plugin Build: Add a Local-AI Chat to Your Site (Pi or Cloud)

UUnknown

2026-02-12

10 min read

Developer tutorial: build a WordPress plugin that uses a Raspberry Pi local AI and falls back to a cloud endpoint with performance and security tuning.

Launch a privacy-first chatbot on WordPress that switches between a local Raspberry Pi inference engine and a cloud AI endpoint

Hook: You want a fast, low-cost AI chat on your WordPress site, but you’re worried about cloud costs, latency, privacy and uncertain hosting resources. This tutorial shows developers how to build a WordPress plugin that uses a local Raspberry Pi inference server when available and automatically falls back to a cloud endpoint — with tuning for performance, caching, and safe deployment on shared hosts.

By 2026, small on-prem inference devices (Raspberry Pi 5 + AI HAT+2) and local LLM runtimes have become practical for micro-apps — but you still need robust fallback and host-aware performance controls.

Why build hybrid (Pi + Cloud)? A short, practical case

Hybrid inference blends the strengths of both worlds:

Local inference reduces latency and keeps sensitive conversations on-premises.
Cloud inference provides scale, large models and high availability when the local device is offline or overloaded.
Fallback logic is essential to avoid downtime on shared hosting, unreliable networks, or when the Pi is updating.

Example (real-world scenario): a small clinic used a Raspberry Pi 5 with an AI HAT+2 to run a quantized medical FAQ model for privacy-sensitive queries. When the Pi was under load or offline, their WordPress plugin routed traffic to a cloud endpoint for continued service — reducing cloud costs by ~70% for low-volume queries while preserving uptime.

What you’ll build — architecture overview (inverted pyramid first)

At the core is a WordPress plugin exposing a REST endpoint that the frontend chat widget calls. The plugin tries the local Pi endpoint first with a short timeout and, on failure or slow response, transparently calls the cloud API. Results are cached and rate-limited.

Components

Raspberry Pi 5 running an LLM server (Ollama / text-generation / custom Flask server) behind Nginx or a tunnel.
WordPress plugin (PHP) with admin settings and a REST proxy that implements the switch logic.
Frontend JS chat widget that streams or polls responses.
Performance & caching layers (transients, queue for long tasks, concurrency limits).

Prerequisites (developer checklist)

WordPress 6.4+ with developer access
Raspberry Pi 5 (recommended) plus AI HAT+2 or equivalent — or a small local x86 box
Local LLM server (Ollama, text-generation-server, or similar) running on Pi and reachable from the web host (via public IP, reverse proxy, or Cloudflare Tunnel)
Cloud AI endpoint (OpenAI-compatible or any LLM REST API) and API key
Familiarity with PHP, WP REST API, JavaScript (fetch), and basic Linux server administration

Set up the Raspberry Pi inference server (summary)

By late 2025 and into 2026, Pi 5 + AI HAT+2 offers usable local inference for small quantized models. For production-like responsiveness, run a small optimized model (e.g., a quantized Llama-style model) via an inference server.

Recommended Pi server stack

OS: Raspberry Pi OS (64-bit) or Ubuntu 22.04/24.04
LLM runtime: Ollama, text-generation-server, or a small Flask/Node server wrapping llama.cpp / ggml
Reverse proxy: Nginx with TLS (Let’s Encrypt) or a Cloudflare Tunnel for secure access
Service manager: systemd for auto-start and restart

Quick example: systemd service for a Python text-generation server

[Unit]
Description=Local LLM Server
After=network.target

[Service]
User=pi
WorkingDirectory=/home/pi/llm-server
ExecStart=/usr/bin/python3 -m server.app --host 0.0.0.0 --port 5000
Restart=on-failure

[Install]
WantedBy=multi-user.target

Then secure it with Nginx and Let's Encrypt or use a Cloudflare Tunnel. Keep an authentication token to avoid open access.

Plugin structure — files and responsibilities

wp-content/plugins/local-ai-chat/
├── local-ai-chat.php          # main plugin bootstrap
├── includes/class-api.php     # REST logic & switch + caching
├── includes/class-admin.php   # settings page
├── assets/js/chat.js          # frontend widget
├── assets/css/chat.css
└── templates/shortcode.php

Install: Create the folder and the main plugin file

/*
Plugin Name: Local AI Chat
Description: Hybrid local + cloud AI chat with Pi fallback
Version: 1.0
Author: Your Name
*/

defined('ABSPATH') || exit;

require_once __DIR__ . '/includes/class-api.php';
require_once __DIR__ . '/includes/class-admin.php';

LocalAI\API::init();
LocalAI\Admin::init();

Core REST proxy — switch logic and timeouts

This is the central part: try the Pi first with a short timeout; if it fails, call cloud. Always validate and sanitize. Cache successful small queries with transients.

namespace LocalAI;

class API {
  public static function init() {
    add_action('rest_api_init', function () {
      register_rest_route('local-ai-chat/v1', '/query', [
        'methods' => 'POST',
        'callback' => [__CLASS__, 'handle_query'],
        'permission_callback' => '__return_true',
      ]);
    });
  }

  public static function handle_query($request) {
    $body = json_decode($request->get_body(), true);
    $prompt = sanitize_text_field($body['prompt'] ?? '');
    if (!$prompt) {
      return new \WP_Error('no_prompt', 'No prompt', ['status'=>400]);
    }

    $cache_key = 'local_ai_' . md5($prompt);
    $cached = get_transient($cache_key);
    if ($cached) {
      return rest_ensure_response(['source'=>'cache','response'=>$cached]);
    }

    // Load settings
    $local_url = esc_url_raw(get_option('local_ai_pi_url', 'http://192.168.1.100:5000/generate'));
    $cloud_url = esc_url_raw(get_option('local_ai_cloud_url', 'https://api.openai.com/v1/chat/completions'));

    // Try local first with a short timeout
    $local_args = [
      'timeout' => 2, // short local timeout (seconds)
      'body' => wp_json_encode(['prompt'=>$prompt,'max_tokens'=>get_option('local_ai_max_tokens',150)]),
      'headers' => ['Content-Type'=>'application/json', 'Authorization'=>'Bearer '.get_option('local_ai_pi_token','')]
    ];

    $local_resp = wp_remote_post($local_url, $local_args);
    if (!is_wp_error($local_resp) && wp_remote_retrieve_response_code($local_resp) === 200) {
      $body = wp_remote_retrieve_body($local_resp);
      set_transient($cache_key, $body, 60*5); // short cache
      return rest_ensure_response(['source'=>'local','response'=>json_decode($body, true)]);
    }

    // Fallback to cloud with a longer timeout and API key stored in config
    $cloud_args = [
      'timeout' => 10,
      'headers' => ['Content-Type'=>'application/json', 'Authorization'=>'Bearer '.get_option('local_ai_cloud_key','')],
      'body' => wp_json_encode([
        'model'=>get_option('local_ai_cloud_model','gpt-4o-mini'),
        'messages'=>[['role'=>'user','content'=>$prompt]],
        'max_tokens'=>get_option('local_ai_max_tokens',150)
      ])
    ];

    $cloud_resp = wp_remote_post($cloud_url, $cloud_args);
    if (is_wp_error($cloud_resp)) {
      return new \WP_Error('ai_error','No AI available', ['status'=>503]);
    }
    $body = wp_remote_retrieve_body($cloud_resp);
    set_transient($cache_key, $body, 60*10);
    return rest_ensure_response(['source'=>'cloud','response'=>json_decode($body, true)]);
  }
}

Notes about the code

Short local timeout: set to 1–3s so the plugin fails over quickly on flaky connections.
Caching: inexpensive queries are cached for 5–10 minutes to save CPU and API calls.
Non-blocking UI: For longer cloud responses, use background workers or streams on the client side.

Keep the widget simple and resilient. Use fetch to call the WP REST route and show a loading state. Support streaming later, but start with normal responses.

document.getElementById('ai-send').addEventListener('click', async () => {
  const prompt = document.getElementById('ai-input').value;
  const res = await fetch('/wp-json/local-ai-chat/v1/query', {
    method: 'POST',
    headers: {'Content-Type':'application/json'},
    body: JSON.stringify({prompt})
  });
  const data = await res.json();
  document.getElementById('ai-output').innerText = data.response.choices ? data.response.choices[0].message.content : JSON.stringify(data.response);
});

Admin settings — host-aware performance controls

Expose a small admin page with fields for:

Pi URL and token
Cloud API URL and key
Local timeout (1-5s), cloud timeout (5-30s)
Max tokens, model choice, caching TTL
Enable/disable local-first vs cloud-first strategies
Host profile: low-memory / shared host / VPS — toggles defaults (max concurrency, queue method)

Host-aware defaults: for shared hosts, set concurrency to 1–2 and avoid long synchronous requests (offload to a background queue such as Action Scheduler or other workers). For VPS or dedicated hosts, use streaming and allow higher concurrency. See Beyond Serverless patterns for guidance.

Advanced performance patterns

1. Transient caching & signature hashing

Cache by prompt hash plus model parameters. Use get_transient / set_transient with TTL tuned to content freshness — a common micro-app pattern described in micro-apps playbooks.

2. Rate limiting and queuing

Use an in-plugin queue (option + cron or Action Scheduler) for requests expected to take >5s. Enqueue, return an immediate job id, and let the frontend poll a results endpoint.

3. Batching and short prompts locally

Run brief prompts locally (FAQ lookups, intent classification) and send longer context to the cloud. Use a small classifier model on the Pi to decide routing — this kind of local routing and lightweight inference is discussed in edge bundle reviews like affordable edge bundles.

Security & deployment best practices

Never store cloud API keys in plain options: prefer constants in wp-config.php or environment variables. Provide an admin helper to copy a secure key into WP but encourage env storage — see cloud-native security guidance.
Secure the Pi: restrict Nginx to accept only token-authenticated requests, or expose it through a Cloudflare Tunnel and use Access policies.
CORS and CSRF: the WP REST API handles nonces; ensure your JS obtains a nonce for admin-only endpoints or use cookie-based authentication.
Input sanitization: sanitize_text_field for prompts when storing, but forward raw prompt content to AI service only over TLS.
Auth & delegation: consider an authorization provider for service-to-service tokens (see reviews of auth services like NebulaAuth).

Monitoring, logging, and observability

Log local vs cloud hits (source), latency, and error codes to a debug log (rotate frequently).
Expose health endpoint on Pi (/health) and use the plugin to check it periodically. If local health fails, mark it unhealthy for a cooldown period.
Track cloud usage and cost — log tokens used per request so you can add quota enforcement in admin.

Testing & fallback validation

Bring Pi offline and verify the plugin routes to cloud within the local timeout window.
Simulate slow local responses and tune the local timeout until failover behaves as expected without dropping short local replies.
Test under host constraints (simulated CPU/memory limits) to ensure fallback remains predictable.

Migration and upgrade path (local -> cloud -> hybrid)

Design your plugin so model names and endpoints are configurable. When traffic grows or you need stronger models, flip cloud-first, or move to a dedicated inference host. Steps:

Run analytics for local vs cloud usage and latency.
Gradually increase local model size if Pi resources permit (quantized models are friendly).
Introduce model versioning in your request signature so cached results remain correct. Use IaC templates when you automate the deployment of inference hosts and their networking.

2026 trends and why this pattern matters now

Two trends accelerated in late 2025 and early 2026:

Edge inference maturity: Raspberry Pi 5 and AI HAT+2 make sub-second responses realistic for small quantized models, powering micro-apps and private local experiences — an evolution summarized in reviews of affordable edge bundles.
Hybrid deployment adoption: Developers increasingly run micro-apps locally for privacy and cost reasons while keeping cloud as a safety net — a pattern mirrored by local AI browsers and micro-app frameworks in 2026. This approach also intersects with conversations about autonomous agents and when to gate automated behaviors.

These changes mean a WordPress site can now offer private, low-cost AI experiences while retaining reliability through cloud fallback.

Troubleshooting quick guide

No local responses: check Pi server logs, token, and /health endpoint. Use curl locally to test.
Time outs: reduce local timeout and increase cloud timeout or use queued processing.
High costs: increase caching and classify queries to route only big-context requests to cloud.
Slow shared host: offload long-running operations to Action Scheduler or external job worker; see cloud-native guidance at Beyond Serverless.

Actionable takeaways

Start local-first with a short timeout (1–3s) and a cloud fallback for reliability.
Use caching and small models on the Pi for FAQs and classification; escalate to cloud for long-context replies.
Add host profiles (shared, VPS, dedicated) to set sane concurrency and queue defaults.
Secure endpoints with token auth, TLS and, where possible, environment-stored cloud keys.

Future-proofing & final notes

By building configurability and observability into the plugin now, you’ll be ready for new local runtimes, on-device acceleration, and specialized LLMs coming through 2026 and beyond.

Learn by doing: minimal checklist to get this working in 1 day

Spin up Pi server with a small model and simple /generate endpoint.
Create plugin folder and paste the core API code above.
Add admin options for local URL and cloud key.
Wire a minimal JS widget and test failure scenarios (bring Pi offline).
Tune timeouts, caching, and start low-traffic production trials.

Closing thoughts

Hybrid local+cloud AI for WordPress is practical in 2026. It saves cost, improves privacy, and provides low-latency experiences — but only if you architect fallback, caching, and host-aware settings correctly. This tutorial gave you a working blueprint and concrete code patterns to implement a robust hybrid plugin.

Call to action: Ready to prototype? Download the starter plugin skeleton from our Git repo, try it with a Pi 5 or a small cloud VM, and share your performance results. If you want a tailored walkthrough or a pre-built plugin configured for shared hosts, request a dev audit from our team — we’ll help you tune timeouts, security, and migration strategy.

Unknown

Contributor

Senior editor and content strategist. Writing about technology, design, and the future of digital media. Follow along for deep dives into the industry's moving parts.