Accessing Apple's on-device AI from Kotlin using apfel

If you own a modern Mac with an Apple Silicon chip and Apple Intelligence enabled, then you already have access to a local foundation model. There is no separate model to download, no API key to create, and no token bill to worry about.

macOS Tahoe ships with Apple’s on-device foundation model, roughly a 3B parameter LLM, and apfel gives it a very convenient front door. It wraps Apple’s FoundationModels framework and exposes the model in three ways:

1. A UNIX-style command-line tool.
2. An OpenAI-compatible HTTP server.
3. An interactive chat mode.

In this post I will show how to use apfel from the command line, then call the same local model from a Kotlin script using the OpenAI-compatible API.

Installing apfel

The installation is a single Homebrew command:

brew install apfel

apfel requires Apple Silicon, macOS Tahoe, and Apple Intelligence to be enabled. Once those pieces are in place, there is no extra configuration needed.

The important part is that the model runs on your Mac. For short prompts, small text transformations, command explanations, shell one-liners, translations, and local scripting tasks, that is very appealing. You get a useful local assistant without sending the prompt to a cloud model.

Prompting from the terminal

After installing apfel, you can prompt the model directly:

apfel --stream "Can you write some Kotlin code to print Hello World to the screen?"

You can also use it in the normal UNIX way by redirecting output:

apfel "Write a Kotlin script to print Hello World to the console without any explanation or anything else besides the script." > hello.kts

The model is still a small local model, so you should inspect what it writes, but the output is easier to pipe into a file.

Starting the OpenAI-compatible server

The most interesting part for Kotlin is the server mode. apfel can expose the local model as an OpenAI-compatible API:

apfel --serve

By default, this starts a server on:

http://127.0.0.1:11434

If that port is already busy, or you just prefer a different port, you can specify one:

apfel --serve --port 11435

The server exposes the familiar chat completions endpoint:

POST /v1/chat/completions

It also supports useful OpenAI-style features such as streaming, GET /v1/models, tool calling, response_format: json_object, temperature, max_tokens, seed, and CORS for browser clients.

For this post, we only need the basic chat completions endpoint.

Calling apfel from Kotlin

Because the server speaks the OpenAI API shape, calling it from Kotlin is straightforward. The model name is apple-foundationmodel, and the API key can be any placeholder value because the local server does not require authentication by default.

Here is a complete Kotlin script:

@file:DependsOn("tools.jackson.core:jackson-databind:3.0.3")

import tools.jackson.databind.json.JsonMapper
import java.net.URI
import java.net.http.HttpClient
import java.net.http.HttpRequest
import java.net.http.HttpResponse

val baseUrl = "http://localhost:11434/v1"
val apiKey = "unused"

val body = """
    {
      "model": "apple-foundationmodel",
      "messages": [
        {
          "role": "user",
          "content": "Translate to Korean: Thank you"
        }
      ],
      "temperature": 0.2,
      "max_tokens": 100
    }
""".trimIndent()

val request = HttpRequest.newBuilder(URI.create("$baseUrl/chat/completions"))
    .header("Content-Type", "application/json")
    .header("Authorization", "Bearer $apiKey")
    .POST(HttpRequest.BodyPublishers.ofString(body))
    .build()

runCatching {
    val response = HttpClient.newHttpClient()
        .send(request, HttpResponse.BodyHandlers.ofString())

    check(response.statusCode() in 200..299) {
        "HTTP ${response.statusCode()}: ${response.body()}"
    }

    val root = JsonMapper.shared().readTree(response.body())
    val content = root.path("choices").get(0)
        ?.path("message")?.path("content")?.asString()
        ?: error("Unexpected response shape: ${response.body()}")

    println(content)
}.onFailure { System.err.println("Error: ${it.message}") }

Save this as apfel.main.kts. With Kotlin on your path, you can run it directly:

kotlin apfel.main.kts

Here is the output I get:

감사합니다.

If you started apfel on the alternate port shown earlier, change the script to:

val baseUrl = "http://localhost:11435/v1"

That is the main benefit of the OpenAI-compatible API. The code is ordinary HTTP client code. If you already have a client abstraction that talks to OpenAI, you can often point it at http://localhost:11434/v1, use the apple-foundationmodel model name, and keep most of the rest of the code the same.

Checking the available model

The server also exposes a models endpoint, which is useful when you want to verify that the server is running:

curl http://localhost:11434/v1/models

For apfel, the important model id is:

apple-foundationmodel

There is no model picker here in the same sense as a cloud API or a local model manager such as Ollama. apfel is intentionally wrapping the Apple model that is already available through macOS.

What this is good for

The local model is not meant to replace larger cloud-based models. It has a smaller context window, around 4,096 tokens for input and output combined, and it is not the tool I would choose for long documents, complex code generation, or tasks that need strong factual recall.

Where it becomes interesting is in small, local, everyday tasks:

1. Explaining a shell command.
2. Generating a small regex.
3. Rephrasing a short paragraph.
4. Translating a phrase.
5. Summarising a short file.
6. Turning command output into something easier to read.
7. Building a small local Kotlin utility that needs just enough language understanding.

The local context is the part I like most. Cloud models cannot see the output of df -h, lsof, git log, or ls unless you explicitly send that data to them. With apfel, you can pipe local command output into a local model and keep the whole loop on your machine.

For example:

df -h | apfel "Which volume is closest to full? Reply in one sentence."

or:

git log --oneline -10 | apfel "Summarise these commits in three bullets."

That makes apfel feel less like a replacement for a big hosted model and more like a small language-aware UNIX tool.

Final thoughts

For me, the nice thing about apfel is not only that it provides CLI access to Apple’s on-device model. It is that it exposes the model through interfaces developers already know: standard input and output for shell scripts, and an OpenAI-compatible API for application code.

That makes it easy to experiment from Kotlin. Start apfel --serve, point a small Kotlin script at http://localhost:11434/v1, send a normal chat completions request, and you have a private local model call running on your Mac.

It will not replace cloud models for every use case, and it should not be treated as if it has the same capabilities as a larger hosted model. But for quick local automation, small text transforms, and developer workflows where privacy and zero setup matter, it is a very useful tool to have installed.