What's this?

A LLM based triage automation system for the dart-lang/sdk repo. It processes new issues filed against the repo and triages them in the same manner that a human would. This includes:

• re-summarizing the issue for clarity
• assigning the issues to an area- label (first line triage)

Bot trigger and entry-point

This bot is generally triggered by a GitHub workflow listening for new issues on the dart-lang/sdk repo.

See https://github.com/dart-lang/sdk/blob/main/.github/workflows/issue-triage.yml.

Overview

The general workflow of the tool is:

• download the issue information (existing labels, title, first comment)
• ask Gemini to summarize the issue (see prompts)
• ask Gemini to classify the issue (see prompts)
• create a comment on the issue (@dart-github-bot) with the summary; apply any labels produced as part of the classification

Tuning

We create a tuned Gemini model in order to improve the performance of classification. This involves:

• downloading recent, already triaged issues (~800 issues)
• writing them in a format suitable for tuning (either .csv or .jsonl)
• tuning via the Gemini APIs; this gives us a new model name to use when calling Gemini (e.g., model: 'tunedModels/sdk-triage-tuned-prompt-1l96e2n')

See tool/create_tuning_data.dart.