Running AI models efficiently in production requires tuning model architecture, compute, memory, and the serving runtime together. Teams also need a repeatable way to deploy and operate that system within their own infrastructure.
Today, Renice AI is announcing its Google Cloud Marketplace offering for inference, Renice-Prio, on Google Kubernetes Engine (GKE). The container-based package brings together deployment automation, verified model checkpoints, and OpenAI-compatible inference endpoints in the customer’s own Google Cloud project.
Customers can create a new managed GKE cluster or deploy into a compatible existing cluster. They retain ownership of their project, cluster, network, and accelerator capacity.
GPT-OSS 120B benchmark: approximately 2.1× the peak output throughput
In an evaluation on October 12, 2026, Renice-Prio reached approximately 3,100 output tokens per second, compared with vLLM-TPU configuration’s peak of roughly 1,500. This represents approximately 2.1× the peak output throughput on the same eight-chip Google Cloud TPU v6e host.
Each request used 8,192 input tokens and 1,024 output tokens, with offered concurrency ranging from 4 to 512 requests.
These measurements apply to the historical software versions shown in the chart.
Model and hardware profiles
The Renice Marketplace offering is accessible through a defined deployment package. It combines Terraform, Helm, containers pinned by image digest, verified model checkpoints, configurable endpoints, and workload-level Marketplace metering.
Instead of assembling these components independently, customers start with a model and hardware profile, then deploy it into their Google Cloud environment.
The initial offering supports these configurations:
| Model | Weight format | Hardware | Topology |
|---|---|---|---|
| GPT-OSS 120B | MXFP4 | Cloud TPU v6e, 8 chips | 2×4 |
| DeepSeek V4.1 Flash-1017 | Native MXFP4 | Cloud TPU v5p, 4 chips | 2×2×1 |
Each profile specifies the model, weight format, and hardware configuration together, giving teams a defined starting point for deployment and evaluation.
For TPU profiles, customers can use on-demand capacity or target a specific customer reservation. Deployments remain subject to compatible infrastructure, quota, and available capacity.
Long context and multi-turn conversations
Renice-Prio supports multi-turn conversations through an OpenAI-compatible chat interface. With each follow-up request, your application sends the previous user and assistant messages so the model has the conversation history.
Context limits vary by model:
- GPT-OSS 120B: The Marketplace profile supports up to 262,144 tokens, using the same serving configuration as the benchmark with 8,000 input tokens and 1,000 output tokens. Requests beyond the model’s 131,072-token trained context rely on a RoPE extension. Evaluate response quality at these longer lengths for your intended workload.
- DeepSeek V4.1 Flash-1017: Supports 1,048,576 tokens shared between input and output. Both the 8k/1k benchmark and longer conversations use the same v5p serving configuration.
Your application controls reply length with max_tokens, which cannot exceed the context space remaining after the input.
The input includes the entire formatted conversation: system instructions, user messages, previous replies, tool results, and chat-formatting overhead. The number of supported turns therefore depends on their combined length.
Longer requests also use spare capacity in the conversation cache (KV cache), reducing the number of requests that can run at the same time.
From deployment configuration to an inference endpoint
The package brings the deployment components into one workflow:
- Go to the Marketplace listing page.
- Select a profile. Choose the supported model and hardware configuration that fits your workload.
- Choose the cluster, namespace and network.
- Choose your API authentication preference.
- Deploy the workload. Terraform and Helm configure the deployment using digest-pinned containers and verified model checkpoints.
- Connect your application. Use the OpenAI-compatible inference interface, checking compatibility with the API features your application requires.
Pinning container images by digest makes the deployed software version explicit. Verified checkpoints provide a defined model artifact for each profile. Together, these choices support repeatable deployments and clearer troubleshooting.
Customer control and clear billing boundaries
Customers retain ownership of the Google Cloud resources supporting their deployment. This allows teams to manage inference within their own project, network, and infrastructure processes.
The offering includes workload-level Marketplace metering for Renice software usage. Google Cloud infrastructure is billed separately, including the underlying cluster and compute or accelerator resources.
This distinction lets customers evaluate software and infrastructure costs together while maintaining control over their capacity choices.
Research informed by production experience
Renice’s research team works across model execution, memory use, and serving infrastructure. We focus on improving hardware efficiency while balancing model quality, throughput, and response time. Production requirements guide our approach to deployment, capacity management, and operational reliability.
The Marketplace package provides a concrete foundation for this work: supported configurations, reproducible deployment artifacts, and endpoints that customers operate within their own environment.
Get started
Find Renice AI on Google Cloud Marketplace to review the supported profiles, deployment requirements, and commercial terms.
About Renice AI
Renice AI builds an integrated stack for AI inference, optimizing everything from chips and memory to the shape of the model itself. We help AI labs, cloud providers, and enterprises turn infrastructure into secure, reliable token factories that scale to meet demanding workloads.
Our team brings experience building AI systems and cloud services for Anthropic, Gemini, and AWS SageMaker and Bedrock. We combine AI systems research with production engineering to improve hardware utilization, simplify deployment, and help customers get more inference capacity from their infrastructure.
To discuss workload fit, capacity requirements, or deployment planning, contact us at [email protected]. You can learn more about Renice at renice.ai.