Image Recognition and Label Parsing Fixes

Antigravity·Zephyrxx0·4d ago·5hr 12min·1 Checkpoint·6 file changes·+41/-14

its a dog, yet labeled as an unidentified organism also it is recognised as tree ...?

look if image cature and recognition works , also look if parsing information and displaying it works correctly

4d ago·14m

The root cause of this issue involved the AI model configuration, fallback handling, and observation state management:


1. Why was a dog recognized as a tree?

  • The backend identification endpoint (app/api/identify/route.ts) was configured to use gemini-2.0-flash.
  • Google has deprecated gemini-2.0-flash, returning an API error (404 NOT_FOUND: This model models/gemini-2.0-flash is no longer available. Please update your code to use models/gemini-3.8-flash).
  • The route caught this error and fell back to hardcoded mock candidates (MOCK_IDENTIFY_RESPONSE):
    • White Oak (Quercus alba, tree)
    • Sugar Maple (Acer saccharum, tree)
  • Every photo submitted hit this catch block and returned those two trees as the candidate species.

2. Why was it labeled as an "Unidentified Organism"?

  • When presented with "White Oak" and "Sugar Maple" for your Beagle, you tapped Save Unidentified in the candidate drawer.
  • Saving unidentified stored the specimen in IndexedDB without a confirmed species (status: 'unidentified').
  • The detail page (app/entry/[id]/page.tsx) displayed the fallback placeholders:
    • Title: "Unidentified Organism"
    • Binomial name: "Species incognita"
    • Category badge: "organism"

3. Why did the Fact Card say "Fact card is generating for Unidentified Organism..."?

  • On the entry page, the facts fetch hook was falling back to entry.candidates[0].wikipediaTitle (White Oak) when entry.confirmed was empty.
  • Calling /api/facts also failed with a 404 because it was also defaulting to gemini-2.0-flash.
  • Because factsData remained empty, components/entry/fact-card.tsx assumed the card was continuously generating facts, leaving it stuck in a loading state.

4. Verification & Fixes Applied

A. Upgraded AI Model to Gemini 3.8 Flash

  • Updated .env.local and default model fallbacks in:
  • Tested live on the running server with your exact uploaded Beagle photo:
    • Top candidate: Beagle (Canis lupus familiaris, Likely, verified: true)
    • Second candidate: Harrier (Canis lupus familiaris, Possible)
    • Third candidate: English Foxhound (Canis lupus familiaris, Unsure)
  • Verified facts synthesis for Beagle: Wikipedia lead extraction, 3 cited biological facts, and safety scrubbing all passed cleanly.

B. Image Capture Pipeline Verification

  • Client-side processing via lib/camera.ts decodes photos directly to an HTML5 canvas, stripping 100% of EXIF/GPS tags.
  • Resizes primary images to a 1024px JPEG and generates 160px thumbnails before encoding to base64.

C. Parsing & Information Display

  • Unidentified Specimen Guidance: FactCard now distinguishes between active generation and unconfirmed specimens, displaying an educational prompt rather than an indefinite generation spinner.
  • On-Page Candidate Picker & "Re-identify with AI": Added an interactive candidate section on app/entry/[id]/page.tsx. You can click Re-identify with AI on your existing dog specimen, which will query the live Gemini 3.8 Flash API and allow you to confirm it as a Beagle directly.
  • Database & Collection Status Alignment: When you confirm a species, it saves as status: 'confirmed', making it visible across Collection, Daily Hunt, and Expedition Journal entries.

@[Quote] use this model : gemma-4-26b-a4b-it everywhere

4d ago·2m

The model has been updated to gemma-4-26b-a4b-it across the entire application:

  1. Environment Configuration (.env.local):

    • IDENTIFY_MODEL_ID=gemma-4-26b-a4b-it
    • FACTS_MODEL_ID=gemma-4-26b-a4b-it
  2. Multimodal Species Identification (app/api/identify/route.ts):

    • Default fallback updated to 'gemma-4-26b-a4b-it'.
    • Verified against your Beagle photo — accurately returns Canis lupus familiaris (Beagle, Likely, verified: true) with ~2.7s response latency.
  3. Natural History & Fact Card Synthesis (app/api/facts/route.ts):

    • Default synthesis model updated to 'gemma-4-26b-a4b-it'.
    • Verified Wikipedia lead grounding, citation indexing, and safety regex scrubbing.
  4. Testing & AI Spike Endpoints:

All routes and tests compile with 0 errors on Next.js 16.4.