Deploy Prio on Google Cloud Marketplace

1. Open Google Cloud Console. Sign in with the account you use for your Google Cloud project. Complete any password, passkey or organization verification Google requests.
2. Check the account avatar at the top right. If you have several Google accounts, make sure the selected account has access to your deployment project.
3. Click the project selector beside the Google Cloud logo. Search for your project by name or ID, then select it. If you need a new project or billing account, have your administrator prepare it before continuing.
4. Open the Navigation menu (☰) at the top left and select Marketplace. You can also open Marketplace directly; recheck the selected project afterward.

If you are starting on a Compute Engine page, use the same navigation menu. Prio deploys to Google Kubernetes Engine (GKE); you do not create a standalone Compute Engine VM first.

1. Click Find products or ask questions inside the Marketplace page.
2. Type Prio Model Serving Platform and press Enter. You do not need AI Mode or the Container images filter: this offering is listed as a Kubernetes app.
3. In the product results, select Prio Model Serving Platform from Renice AI, Inc. Check both the name and publisher.

You can also open the Prio Marketplace listing directly. If it does not load, follow the listing and account access checks.

1. Confirm the title Prio Model Serving Platform and publisher Renice AI, Inc.
2. Read Overview, Pricing, Documentation and Support. Review the applicable product terms and your organization’s purchase requirements.
3. If Google shows Purchase or Get, have an authorized purchaser complete the displayed billing, order and terms steps first. Return to the listing once the order is available for your billing account. If Configure is already shown, continue directly.
4. Click Configure. The next page is titled Deploy Prio Model Serving Platform. Confirm that your selected project and release are still correct.

Google documents Purchase, Configure and Get as possible Kubernetes listing actions. The purchase step is conditional; it was not exercised with a new billing account in these captures.

Configuring the form does not create a cluster. Resource creation starts when you submit Deploy after completing the remaining steps.

Confirm your project in the top bar and select Click to Deploy on GKE. Follow this tab throughout the guide. Google may also display a command-line tab; you do not need it for this installation.

Standard installations also need CPU capacity for system services. Reusing a cluster still provisions the model's compute, including a dedicated N2 model pool for Qwen.

Choose New Cluster or Existing Cluster first. After changing a mode or model, recheck dependent fields before submitting.

For Qwen CPU, there is no TPU zone or reservation to choose. A new Standard CPU cluster places its model pool within the selected region.

TPU capacity example

The GKE form uses a private API. Authentication controls who can call it; network access controls who can reach it. You need both.

Generate bearer credential — simplest first run

The token is generated inside Kubernetes, not displayed in the form. After deployment, step 10 reads it with kubectl and sends a request.

Google identity (OIDC) — one identity per caller

1. Select Google identity (OIDC).
2. Set ID token audience to the exact identifier your callers request, for example https://prio.example.com. This does not create a hostname or public endpoint.
3. Add the exact caller service-account emails under Admitted service accounts. These need not be the deployment service account.
4. For manual testing, an administrator must authorize you to mint an ID token by impersonating an admitted account. Step 10 shows the mint command; it includes the audience and email claim. A Google access token is not interchangeable.

Keep the defaults unless your network requires changes. You do not need to add your computer’s IP address: kubectl reaches a new cluster through its DNS endpoint.

Inference API access only governs direct calls from your network. It does not affect kubectl and does not expose the model endpoint. Diagnose connectivity problems.

1. Recheck project, release, model, cluster, namespace, reservation and its zone if applicable, network and authentication.
2. Review terms and pricing, then select Deploy once.
3. Open Solutions → Solution deployments in the same project. Select your deployment and inspect its status and View Logs.
4. A CPU deployment stays Deploying while the model loads and shows Deployed once its API is ready. A TPU deployment shows Deployed once its API is ready or after about 45 minutes, whichever comes first, so its model may still be loading. Either way, continue with step 10: its commands wait for the model.

Checkpoint download, verification, restore and preparation can take time, especially for DeepSeek. Follow logs rather than assuming a fixed startup time or clicking Deploy again.

The model API is private. From Cloud Shell or your own terminal, connect to the cluster, open a local tunnel and call the API. You only need the names you entered in the form.

1. Connect to your cluster

Use the cluster name and location from the form, and your project ID. The location is the cluster's region, or its zone for a zonal existing cluster.

gcloud container clusters get-credentials YOUR_CLUSTER --location=YOUR_LOCATION --project=YOUR_PROJECT --dns-endpoint

For an existing cluster, its administrator enables this once with gcloud container clusters update YOUR_CLUSTER --location=YOUR_LOCATION --enable-dns-access. If you already reach that cluster another way, omit --dns-endpoint.

2. Open the tunnel and load your API key

Set your deployment name and namespace (the form's default namespace is llm-serving). The commands wait until the model is ready, open a tunnel on a free local port in the background, and load the generated key into APP_API_KEY. If the tunnel does not start, they print its log and leave APP_BASE_URL unset, so no request is sent.

DEPLOYMENT_NAME=prio-team-test
NAMESPACE=llm-serving
kubectl -n $NAMESPACE rollout status deployment/$DEPLOYMENT_NAME-tokenfront --timeout=5h
unset APP_PORT APP_BASE_URL; APP_TUNNEL_LOG="$(mktemp)"
kubectl -n $NAMESPACE port-forward service/$DEPLOYMENT_NAME-tokenfront :8080 --address=127.0.0.1 > "$APP_TUNNEL_LOG" 2>&1 & PF=$!
for i in $(seq 60); do APP_PORT="$(awk '/^Forwarding from 127.0.0.1:/ { split($3, a, ":"); print a[2]; exit }' "$APP_TUNNEL_LOG")"; [ -n "$APP_PORT" ] && break; kill -0 $PF 2>/dev/null || break; sleep 1; done
if [ -n "$APP_PORT" ]; then export APP_BASE_URL="http://127.0.0.1:$APP_PORT/v1"; else cat "$APP_TUNNEL_LOG"; echo "The local tunnel did not start; see the log above."; fi
export APP_API_KEY="$(kubectl -n $NAMESPACE get secret $DEPLOYMENT_NAME-api-key -o go-template='{{ index .data "api-key" }}' | base64 --decode)"

3. Call the API

MODEL_ID=qwen3p5-0p8b-int8
curl -s $APP_BASE_URL/models -H "Authorization: Bearer $APP_API_KEY"; echo
curl --fail-with-body $APP_BASE_URL/chat/completions \
  -H "Authorization: Bearer $APP_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"model":"'"$MODEL_ID"'","messages":[{"role":"user","content":"Say hello in one sentence."}],"max_tokens":128}'; echo

/models returns the API model ID and max_prompt_length, the prompt limit including chat-template tokens. Set MODEL_ID to that ID; the model table lists it too. Expect JSON containing choices. For streaming, add "stream":true to the JSON and -N to curl.

For an OpenAI-compatible client, use $APP_BASE_URL as its base URL and $APP_API_KEY as its API key, for example OPENAI_BASE_URL=$APP_BASE_URL OPENAI_API_KEY=$APP_API_KEY python your_app.py. When you finish, close the tunnel with kill $PF and run unset APP_API_KEY.

The tunnel listens only on 127.0.0.1, and kubectl reaches the cluster over authenticated TLS. The model API still checks its configured credential. Never share tokens in tickets or screenshots. How Kubernetes port-forwarding works.

1. Stop clients, close the tunnel with kill $PF (or close that terminal) and clear the key with unset APP_API_KEY.
2. In the correct project's Solution deployments, select your deployment, verify its name and choose Delete.
3. Choose resource deletion if offered a choice. Retaining or abandoning resources does not stop their charges. Keep the deployment account and permissions available until deletion finishes.
4. Wait for completion. Check GKE, Compute Engine disks and load balancers for this deployment's resources, including TPU nodes. A removed application or list entry alone does not prove compute is gone.
5. If deletion fails, resolve the dependency or permission in its logs before retrying. Do not delete shared resources as a shortcut. Cleanup troubleshooting.