Build a Local-First Note App with the Gemini 3.7 Flash API

Q3 2026 setup notes before you build

Last verified: August 25, 2026, UTC. This guide is for using Gemini 3.7 Flash API to build a note taking app with local Markdown files, auto-summaries, auto-tags, and semantic search. The current model ID used here is gemini-3.7-flash, and Google lists Gemini 3.7 Flash with a 1,048,576-token input limit, a 65,536-token output limit, and support for text, image, video, audio, and PDF inputs on the model page updated August 13, 2026.[1]

Google’s current getting-started path shows the Gen AI SDK installation commands — pip install -U google-genai or npm install @google/genai — and the first-call shape using client.interactions.create with model='gemini-3.7-flash'.[2] That matters because older note-app tutorials still floating around the web often use gemini-pro or 2.x-era examples. They may still be useful as product sketches, but not as API references for this build.

The build flow

The through-line is simple: the Markdown file is the note; the database is an index and a recovery aid. PostgreSQL is used here because it makes metadata, job state, and vectors inspectable with ordinary SQL. That is a preference, not a law. SQLite plus a vector extension, a file-based index, or another local database can be the better fit for a smaller desktop app. The design constraint is that a note saved today should still be readable if the Gemini key is revoked tomorrow.

1. Create the API boundary, then keep it boring

Start by treating the Gemini key as server-side configuration. In a desktop app, that may mean a local encrypted settings store or an OS keychain. In a web app, it means the key belongs behind your own API route, never in browser JavaScript. The note files do not need to know that Gemini exists.

npm install @google/genai

# .env.local or your server-side environment manager
GEMINI_API_KEY="replace-with-your-key"

The first call should not summarize a whole vault. It should send a small string and fail loudly if auth, dependency installation, or the model ID is wrong. Google’s getting-started documentation shows the SDK path and the Interactions API call shape for Gemini 3.7 Flash.[2]

import { GoogleGenAI } from "@google/genai";

const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });

export async function smokeTestGemini() {
  const response = await ai.client.interactions.create({
    model: "gemini-3.7-flash",
    input: "Reply with: ok"
  });

  return response;
}

The exact response-object accessors can change across SDK versions, so keep this test isolated. The useful thing is not the string “ok.” It is having one tiny endpoint or script that proves your installed SDK, environment variable, API key, and model name agree before you wire the result into note storage.

2. Make Markdown the source of truth

For a local-first note app, the lowest-drama file format is still a directory of Markdown files. Keep the note body in a plain file and put only stable, user-meaningful fields in front matter. AI-produced fields can be stored in the database or in a clearly separated sidecar file. Mixing regenerated tags into the canonical Markdown body is how a harmless feature becomes a merge-conflict machine.

---
id: 8b3b8a80-7a1f-4c1c-81f1-example
created_at: 2026-08-25T12:00:00Z
updated_at: 2026-08-25T12:00:00Z
title: "Example architecture note"
---

Write the human note here.

The file should remain useful even if summaries, tags, embeddings,
and the Gemini API are unavailable.

A minimal database schema can track the file path, content hash, AI metadata, embedding status, and job errors. The content hash is not decorative. It tells the app whether a summary or embedding still belongs to the current version of a note.

create table notes (
  id uuid primary key,
  path text not null unique,
  title text,
  body_hash text not null,
  updated_at timestamptz not null
);

create table note_ai_metadata (
  note_id uuid primary key references notes(id) on delete cascade,
  body_hash text not null,
  summary text,
  tags text[] not null default '{}',
  model_id text not null,
  generated_at timestamptz not null,
  error text
);

This separation also gives you a clean re-run path. If revised instructions produce better tags, you can regenerate the metadata table without rewriting the Markdown files. If a user edits the note while a background job is running, the job can compare hashes before saving and discard stale output.

If you are deciding whether this much hardening is worth it, FlowDesk’s AI note-taking agents security test is a useful companion because it focuses on what note assistants do with private material rather than on feature demos.

3. Generate summaries and tags as structured output

Summaries and tags are clerical features, which is exactly why they should be boring to store. Do not ask the model for “a nice summary and useful tags” and then scrape text with regular expressions. Google’s structured-output documentation supports constraining model responses to a JSON schema using Pydantic or Zod, and it also documents streaming partial JSON.[3]

import { z } from "zod";

export const NoteAiSchema = z.object({
  summary: z.string().max(900),
  tags: z.array(
    z.string()
      .min(1)
      .max(40)
      .regex(/^[a-z0-9][a-z0-9- ]*[a-z0-9]$/i)
  ).max(8),
  confidence_notes: z.string().max(500).optional()
});

export type NoteAiResult = z.infer<typeof NoteAiSchema>;

The schema should reflect how your app behaves, not what looks impressive in a demo. If your sidebar cannot use more than eight tags, cap it at eight. If tags are used as filters, prevent newline characters and other formatting surprises. If you will show the summary in a narrow list view, set a practical maximum.

export async function summarizeAndTagNote(note: {
  id: string;
  title: string;
  markdownBody: string;
  bodyHash: string;
}) {
  const response = await ai.client.interactions.create({
    model: "gemini-3.7-flash",
    input: [
      {
        role: "user",
        content: [
          {
            type: "text",
            text: `Summarize this Markdown note and propose tags.\n\nTitle: ${note.title}\n\n${note.markdownBody}`
          }
        ]
      }
    ],
    config: {
      responseMimeType: "application/json",
      responseSchema: NoteAiSchema
    }
  });

  const parsed = NoteAiSchema.parse(response.output);

  return {
    note_id: note.id,
    body_hash: note.bodyHash,
    summary: parsed.summary,
    tags: normalizeTags(parsed.tags),
    model_id: "gemini-3.7-flash",
    generated_at: new Date().toISOString()
  };
}

function normalizeTags(tags: string[]) {
  return [...new Set(tags.map(t => t.trim().toLowerCase()))].sort();
}

Treat the code above as an integration pattern, not a pasted guarantee. The important parts are the same even if your SDK response accessor differs: a fixed model ID, a schema, validation after the model returns, tag normalization, and a stored body_hash. If validation fails, the app should preserve the note, record the failure, and leave the previous summary in place until a successful regeneration replaces it.

The last row is easy to skip and annoying to retrofit. If a user corrects “postgres” to “postgresql,” the next background run should not quietly undo the correction. A small tag_source field or a separate user tag table is often enough.

4. Store embeddings without turning search into a cloud dependency

Auto-tags help navigation when the right word is already known. Semantic search is for the messier case: “that note about rotating tokens after a vendor outage” when the file was actually titled “ops cleanup.” Google’s embeddings documentation identifies gemini-embedding-2 for embeddings and describes Matryoshka dimension truncation, with 768 dimensions recommended for many retrieval use cases.[4]

The database side is straightforward if you use pgvector: add a vector(768) column, query by cosine distance, and add an HNSW index once the table is large enough to justify approximate nearest-neighbor search. Google’s pgvector guide shows the pattern of storing embeddings in PostgreSQL and querying them with pgvector operators, including cosine search and HNSW indexing.[5]

create extension if not exists vector;

create table note_embeddings (
  note_id uuid primary key references notes(id) on delete cascade,
  body_hash text not null,
  embedding vector(768) not null,
  embedding_model text not null,
  embedded_at timestamptz not null,
  status text not null default 'ready',
  error text
);

create index note_embeddings_hnsw_cosine_idx
on note_embeddings
using hnsw (embedding vector_cosine_ops);

For indexing, send the part of the note that should be discoverable. In a personal note app that is usually title plus body text, not the YAML front matter and not every regenerated summary. Embedding AI-generated summaries can work, but it also lets one model’s wording influence what another search retrieves. Start with the human note, then add derived fields deliberately if recall is poor.

export async function embedNoteForSearch(note: {
  id: string;
  title: string;
  markdownBody: string;
  bodyHash: string;
}) {
  const textForEmbedding = `${note.title}\n\n${note.markdownBody}`;

  const response = await ai.models.embedContent({
    model: "gemini-embedding-2",
    contents: textForEmbedding,
    config: {
      outputDimensionality: 768
    }
  });

  return {
    note_id: note.id,
    body_hash: note.bodyHash,
    embedding: response.embedding.values,
    embedding_model: "gemini-embedding-2",
    embedded_at: new Date().toISOString()
  };
}

The query path mirrors the indexing path: embed the user’s query, then ask PostgreSQL for the nearest vectors. Use the same embedding model and dimensionality for both note vectors and query vectors. Mixing dimensions is not a runtime mystery; it is a schema error waiting for the first search request.

select
  n.id,
  n.title,
  n.path,
  m.summary,
  e.embedding <=> $1::vector as cosine_distance
from note_embeddings e
join notes n on n.id = e.note_id
left join note_ai_metadata m on m.note_id = n.id and m.body_hash = n.body_hash
where e.body_hash = n.body_hash
order by e.embedding <=> $1::vector
limit 12;

A local-first app still calls Gemini to create the query embedding unless you add a local embedding model. That is an honest dependency. The recoverable part is that existing Markdown notes and stored embeddings remain inspectable; the app can still show files, exact-text search, and previously indexed results while the AI service is unavailable.

For a broader look at which AI note features usually justify the complexity, FlowDesk’s AI PKM apps value-versus-hype setup guide is closest to this build’s feature set: semantic search and auto-tagging, not a chatty second-brain performance.

5. Add a job table before you add a prettier UI

The first prototype can call Gemini during save. The second prototype should not. Saving a note should write the Markdown file and database row first, then enqueue AI work. Summaries, tags, and embeddings are enhancements; they should not hold the note hostage.

create table ai_jobs (
  id bigserial primary key,
  note_id uuid not null references notes(id) on delete cascade,
  body_hash text not null,
  job_type text not null check (job_type in ('summary_tags', 'embedding')),
  status text not null default 'queued' check (status in ('queued', 'running', 'done', 'failed', 'retry_wait')),
  attempts int not null default 0,
  run_after timestamptz not null default now(),
  last_error text,
  created_at timestamptz not null default now(),
  updated_at timestamptz not null default now()
);

create index ai_jobs_ready_idx
on ai_jobs (status, run_after, created_at);

This table is where maintenance stops being theoretical. It lets you pause the worker after repeated failures, re-run only embeddings for notes whose hash changed, and explain to a user why a tag list is stale. Without job state, every failure becomes either invisible or a support thread.

• On note save: update the Markdown file, calculate a new body hash, upsert the note row, and enqueue one summary/tag job plus one embedding job.
• Before a worker writes output: compare the job body hash with the current note body hash.
• On validation failure: mark the job failed, keep the previous metadata, and make the error visible in a developer log.
• On rate-limit failure: move the job to retry_wait with a future run_after value instead of looping.
• On API key rotation: stop workers, update configuration, run the smoke test, then resume queued jobs.

6. Price the feature as arithmetic, not vibes

Google lists introductory Gemini 3.7 Flash pricing at $0.75 per 1 million input tokens and $3.75 per 1 million output tokens through December 31, 2026.[6] Google’s launch post also states that pricing doubles on January 1, 2027, to $1.50 per 1 million input tokens and $7.50 per 1 million output tokens.[7]

Here is the per-note math for one summary/tag call, labeled as arithmetic on those published rates rather than a vendor promise. For a roughly 1,000-token note with about 200 output tokens, the input side is 1,000 ÷ 1,000,000 × $0.75 = $0.00075. The output side is 200 ÷ 1,000,000 × $3.75 = $0.00075. Together, that is about $0.0015 per note at the introductory rates.[6]

That number excludes embeddings, retries, failed calls, longer notes, and any prompts that include extra context. It also assumes one generated summary/tag result per note version. If the app regenerates tags on every keystroke, the cost model is no longer “per note”; it is “per save event,” and the user should be told that.

A useful UI shows token-estimate ranges before batch jobs run: “index 2,400 notes,” “regenerate summaries for 318 changed notes,” or “embed 45 notes missing vectors.” It does not need accounting-grade precision. It needs enough visibility that a user can stop a batch before a weekend experiment becomes an unpleasant invoice. For external benchmarking context, FlowDesk’s AI note cost comparison is more relevant than generic productivity claims.

7. Rate limits shape the worker, not the note format

Google’s rate-limit documentation describes usage tiers and spend-based limits, including caps of $10, $50, and $200 per rolling 10 minutes depending on tier, and it identifies 429 RESOURCE_EXHAUSTED as the relevant error to handle when limits are exceeded.[8]

A note app should respond to that with boring controls: concurrency limits, exponential backoff, a batch pause button, and job resumption. Do not hide a 429 behind “AI unavailable.” The person maintaining the app needs to know whether the failure is auth, schema validation, a transient service issue, or a quota boundary.

async function runAiJobWithRetry(job: AiJob) {
  try {
    await markRunning(job.id);

    if (job.job_type === "summary_tags") {
      await runSummaryTagJob(job);
    } else {
      await runEmbeddingJob(job);
    }

    await markDone(job.id);
  } catch (error: any) {
    if (isRateLimitError(error)) {
      await markRetryWait(job.id, {
        attempts: job.attempts + 1,
        runAfter: backoffTime(job.attempts),
        lastError: "429 RESOURCE_EXHAUSTED"
      });
      return;
    }

    await markFailed(job.id, {
      attempts: job.attempts + 1,
      lastError: safeErrorMessage(error)
    });
  }
}

function isRateLimitError(error: any) {
  return error?.status === 429 || String(error?.message ?? "").includes("RESOURCE_EXHAUSTED");
}

For a personal app, one worker with low concurrency is usually less impressive and more useful than a clever queue that saturates limits. Batch import is the risky path: migrating a large Markdown vault, changing the embedding dimension, or revising the summary schema. Those operations should be resumable and cancelable.

8. Put the pieces behind a small local API

The app can be Electron, Tauri, a local web server, or a conventional web app pointed at a local sync directory. The useful boundary is the same: one path saves Markdown, one path reads notes, one path searches, and background workers handle AI tasks.

If you already live in an Obsidian-style Markdown vault, compare this build with a lighter assistant layer before committing to a full database-backed app. FlowDesk’s Claude organizing Obsidian notes profile is a useful contrast: sometimes the right answer is not a new app, but a controlled AI pass over files you already own.

Known issues log for this version

The current Gemini 3.7 Flash API makes the AI layer smaller than the surrounding app work: one documented interaction for structured summaries and tags, one embedding path for semantic search, and enough model capacity that ordinary personal notes are not the bottleneck. The prototype becomes maintainable only when the non-demo parts are treated as first-class: Markdown that survives without AI, metadata that can be regenerated, embeddings with stored hashes, retry behavior for 429s, and visible cost arithmetic before batch work runs.

References

1. Gemini 3.7 Flash, Google AI for Developers, 2026-08-13
2. Get started with the Gemini API, Google AI for Developers
3. Structured output, Google AI for Developers
4. Embeddings, Google AI for Developers
5. A guide to embeddings and pgvector, DEV Community / Google AI
6. Gemini API pricing, Google AI for Developers
7. Introducing Gemini 3.7 Flash, Google Blog
8. Rate limits, Google AI for Developers