Skip to content
Get Started

Serving an Osaka AI Service Through a Tokyo VPS

A team in Osaka runs speech recognition and translation models on its own hardware. Its users are in Japan and mainland China, so serving the models was a separate problem from running them.

The team put a small Riven Cloud KVM VPS in Tokyo in front of the Osaka system. Inference stayed in Osaka. The Tokyo VPS provides the public IP address, TLS endpoint, reverse proxy, and carrier-specific routing to mainland China.

This is a Riven Cloud customer case study. The customer is not publishing the product name or specific model versions. The customer operates the proxy, tunnel, application authentication, and models. Riven Cloud provides the Tokyo VPS and its network routes.

Customer background

The project provides cross-language voice input and translation. Its model runtime needs local compute, storage, and processes that remain resident between requests. The public edge needs a stable address, a certificate, request controls, and reliable paths to its users.

An always-on cloud GPU instance could have handled both inference and delivery, but the team already had suitable compute in Osaka. Keeping inference there avoided renting another GPU merely to obtain a public HTTPS endpoint.

The Tokyo VPS handles a much narrower workload. It is where users enter the service, not where the models run.

The delivery problem

Four constraints had to hold at the same time:

  • The inference service exposes no listening port to the public internet.
  • Users receive one stable HTTPS entry point even if the Osaka site’s address changes or stops accepting connections.
  • Mainland China traffic uses a carrier-aware path instead of generic international transit.
  • The model service can be restarted or updated without rebuilding the public-facing layer.

A reverse proxy can terminate TLS and apply routing rules, but it still needs a path to an origin without a public listener. The outbound tunnel supplies that path. The proxy handles the public certificate and request controls.

The project runs both functions at the Tokyo edge.

Riven Cloud’s role in the architecture

Riven Cloud supplies the Tokyo VPS and its network position. The customer operates the delivery components on that VPS:

  1. A reverse proxy accepts public HTTPS traffic and applies the customer’s routing and access rules.
  2. A customer-managed gateway forwards approved requests to the private inference service.
  3. An outbound tunnel connects the Tokyo edge to Osaka. Because the Osaka side initiates the connection, the inference host does not need an inbound public port.
  4. For mainland China users, the Tokyo VPS uses Riven Cloud’s carrier-specific paths: CTGNet (AS23764) with China Telecom’s CN2 premium segment (AS4809), China Unicom Premium (CUP, AS9929) through China Unicom Global (AS10099), and China Mobile International N2 (CMIN2, AS58807).

The Tokyo server holds the public service layer for the customer’s own application. It does not run the models.

Riven Cloud supplies the unmanaged KVM VPS and its network routes. The customer is responsible for the proxy configuration, tunnel identity, application authentication, monitoring, and model operations.

For background on the carrier paths, read Riven Cloud’s China-optimized routing guide and network overview.

How a request reaches the Osaka model

The request path keeps the inference service behind the customer-managed tunnel:

User in Japan or mainland China
  -> public HTTPS endpoint on a Riven Cloud Tokyo VPS
  -> customer-managed reverse proxy and gateway
  -> outbound tunnel from Osaka
  -> private speech and translation service in Osaka
  -> response returns through the same controlled path

Users only see the Tokyo HTTPS endpoint. The Osaka inference host has no public listening port for this service. The public edge can keep its address, certificate, and routing rules while the team updates the local runtime behind the tunnel.

The customer reported 9-10 ms round-trip latency between the Tokyo VPS and the Osaka inference host during tests in July 2026. The exact test date and raw samples were not retained with this case study, so the figure is a case-specific observation rather than a network benchmark. The customer reported no perceptible change to the complete voice and language workflow.

For mainland China users, the Tokyo edge does not treat the three carriers as one route. China Telecom, China Unicom, and China Mobile each use a different carrier-specific path. This does not guarantee one latency for every province or access line. Last-mile conditions, time of day, and carrier policy still matter, so customers should test from the networks they plan to serve.

Japan-based users use the same endpoint without needing to know where the models run. Public traffic ends at the edge; private inference stays in Osaka.

What the split actually buys

The existing Osaka hardware covers inference. The Tokyo VPS handles TLS termination, request proxying, and the tunnel endpoint.

The split also makes ordinary operations less disruptive. The team can restart or update inference in Osaka without changing the public DNS name, TLS endpoint, or routing policy. The edge and the model runtime are monitored as separate systems because their failures look different and need different responses.

The cost depends on local hardware, request volume, monthly transfer, and the operational work the team is willing to own. For this project, separating delivery from inference avoided an always-on cloud GPU bill.

When this pattern fits

This pattern suits teams that already have local or private model compute but need a stable public endpoint for users in Japan and mainland China. It also works when the origin is behind NAT or changes independently of the public service layer.

It is a poor fit for teams that want a fully managed AI API gateway, have nobody to maintain a Linux server and tunnel, or need the model itself to scale across several cloud GPU regions. A managed platform or regional GPU deployment will be less work in those cases.

This edge design is a poor fit for sustained heavy traffic. A voice and text request moves little data compared with video or large-file delivery. High-volume services may exhaust the plan’s monthly transfer allowance or trigger Riven Cloud’s sustained bandwidth limits, both described in the FAQ.

If you are evaluating a Tokyo edge, test the path from the networks that will actually use the service. Riven Cloud’s Tokyo Looking Glass is a useful starting point, and the current VPS plans show the available edge-server sizes.

For this Osaka project, the models stay on existing local compute while the Tokyo VPS handles public delivery. Teams considering the same design should test the edge path first and account for the work of operating the proxy and tunnel.

Share