SparkInfer has merged two new ways for local AI software to talk to its inference server.
Alongside the existing OpenAI-compatible /v1 routes, the server now exposes selected LM Studio endpoints under /api/v0 and selected Ollama endpoints under /api. The compatibility layers translate requests and responses around the generation handlers SparkInfer already uses.
That architecture is the point. SparkInfer did not add three inference engines. It added three protocol dialects around one core.
The change could reduce the amount of custom integration work needed to place SparkInfer behind software that already expects LM Studio or Ollama behavior. It does not prove that every client, endpoint or model-management workflow is compatible. The code is merged on the main branch, but it is not yet presented as a tagged SparkInfer release or verified production deployment.
What the LM Studio surface includes
The LM Studio compatibility layer implements model listing, model detail, chat completions and text completions. Responses add the stats, model_info and runtime blocks expected by that surface.
Streaming uses server-sent events, like the OpenAI path, but the LM Studio dialect can attach its statistics to the usage chunk.
Embeddings return HTTP 501 rather than pretending to exist. That is a small but useful design choice. A specific unsupported response lets a client distinguish a deliberately absent capability from a broken route.
The same restraint appears throughout the Ollama layer.
What the Ollama surface includes
SparkInfer now implements Ollama-style version, model tags, running-model state, model detail, chat and generate routes.
The streaming format changes from OpenAI server-sent events to newline-delimited JSON. Each line carries an Ollama-shaped message or completion record, and the stream ends with a done=true object rather than an OpenAI [DONE] marker.
Embeddings and model-management endpoints return HTTP 501 with a reason. SparkInfer is not claiming that clients can pull, create, copy or delete models through this server.
That limit is important for search intent. “Ollama API compatibility” can sound like a drop-in replacement for the entire Ollama daemon. The merged code supports a defined inference subset, not every operation in the Ollama product.
One generation loop, three stream dialects
The pull request says all stream writers already passed through one function. The new implementation assigns a stream dialect to each request at that boundary, while leaving the model generation loop unchanged.
For the existing /v1 path, OpenAI responses are expected to remain byte-identical to their previous form. LM Studio uses the same event framing but adds its own metadata. Ollama receives newline-delimited objects with its expected message and completion fields.
This design avoids a common compatibility trap. Reimplementing generation for every client surface can produce different sampling, cancellation or error behavior depending on which URL a user calls. Translating at the edge keeps the compute path shared.
It also concentrates risk in the translators. A nullable field, wrong finish signal or misleading HTTP 200 can make a stream unusable even when the model generated tokens correctly.
SparkInfer says testing with real clients exposed six bugs that curl-based checks did not catch. One nullable field caused the server to terminate during streaming. Another mapping returned HTTP 200 from /api/generate while producing output the client could not use.
The repository includes regression tests for those cases. Tao Outsider did not independently run the official Ollama CLI or the OpenAI SDK against the merged server.
What the project measured
The pull request reports live incremental output across all three surfaces. Its example observations include 36 Ollama chunks distributed over roughly 145 to 273 milliseconds, 38 LM Studio chunks carrying statistics, and 33 OpenAI chunks without LM Studio metadata leaking into the response.
Those observations support the project’s claim that the streams were incremental in its test environment. They are not latency benchmarks. Chunk count and spacing depend on the model, hardware, prompt, sampler, client and buffering path.
The PR reports a clean build and passing server unit tests. It does not report users, production request volume, revenue or demand created by the new routes.
The same merge changes how performance patches are judged
Pull request 1015 also changes SparkInfer’s evaluation bots.
The Muse matrix expands from two axes to 12, measuring decode and prefill at 128, 512, 4,000, 16,000, 32,000 and 64,000 tokens. It adds 32,000-token no-regression checks on Qwen3.6 and a ModelOpt Qwen3.8 checkpoint.
The bots now automatically close a pull request only for a measured REJECT. A none result means the bot did not measure a relevant axis, not that the code failed. SparkInfer says the previous rule nearly closed a Muse prefill improvement because a different evaluator had no applicable result.
Outage handling also changes. A dead evaluation machine had degraded to labels-only mode and exited successfully, which allowed repeated failures to look routine. The wrappers now count and escalate that condition.
These are repository-governance changes, not product features. They matter because public performance claims are only as reliable as the harness that decides which patches survive.
The Gittensor context remains a separate claim
SparkInfer is public software from Gittensor AI Lab. TaoSwap listed Gittensor SN74 with 11 active miners, an emission_value of 0.000005474 and miner burn of about 63.95% at block 9,037,275.
That status does not show that SN74 miners run this merged revision. It also says nothing about live LM Studio or Ollama clients, or whether the compatibility work affects subnet rewards.
The next proof should be a tagged release, a reproducible client matrix and deployment evidence that links the public server behavior to the subnet’s actual serving path. Until then, the accurate description is narrower: SparkInfer’s main branch now speaks more of the local AI software ecosystem without duplicating its inference core.
Sources
Was this article useful?
One tap feedback helps us improve each post.
