Class 5: Checking an agent’s evidence

1. Turn your question into an instruction

An agent is a system that uses a model and tools to carry out work. Today’s agent runs commands in the working folder and explains their results. Its explanations are not evidence from the text, so you check the commands and outputs yourself.

Rewrite your class 4 homework question as an instruction of this form. Change the folder and corpus filenames to match your author, and use your own question.

Use the prepared corpus akutagawa.lingcorpus, the works list works.csv, and the pattern file class4.lx in the working folder dh-b-2026/class4-akutagawa to investigate this question.

Question: In Akutagawa’s fiction, does 落ちる combine with 来る more often than with 行く?

Conditions: show every ling command and its full output. Give the unit counted with every count (tokens, works, or per 10,000 verbs). Quote a passage from the text, with the work’s title, for every claim. Do not change or create files. Ask me before running each command.

Choose one other student’s question from “Submitted questions”. Rewrite that question as an instruction of the same form. If no other question has been submitted yet, or the page is unavailable, use this question.

Question: In your author’s fiction, does 上る combine with 行く more often than with 来る?

2. Read the recorded investigation

The recording below is a conversation in which an agent investigated the example question from section 1 on the Akutagawa corpus. The recording dates from October 27, 2026 and uses Claude Code (model Claude Fable 5.1) and ling 0.5.0. Each command was approved one at a time. The play button shows the conversation in order. The arrow buttons move to the previous or next block. You can also open the recording in a new window.

For each command proposed in the recording, record the following.

Record the unit the agent stated separately from the unit the command actually counted. A summarize count is a token count, not a number of works.

Rerunning the recording’s commands in section 4 requires the Akutagawa folder class4-akutagawa. If your author differs, save that ZIP from the class 4 author table and extract it.

3. Investigate with your own agent as well (optional)

If an agent is available on your own computer, open a terminal, move to your author’s folder dh-b-2026/class4-<author>, start the agent there, and enter the instruction from section 1.

1. Write the agent or tool name, the model name if shown, and the date in your notes.
2. Read each proposed command before it runs, and approve it only after checking that it does not change or create files. Do not approve a proposal that writes to a file; ask for the output on screen instead.
3. Record the same five items as in the section 2 table for your own instruction.
4. Save the whole conversation as a text file. Leave out API keys and personal conversations.

If you do not use an agent, use the recording in section 2 for sections 4 onward.

4. Reproduce one claim

Choose one claim that includes a count.

1. Run the command behind the claim in your own terminal exactly as written, and compare the counts.
2. Display the quoted passage from the text with --where '@document.title = TITLE' and check that it matches the quotation.
3. Record the result as one of these three outcomes.

ling matches --corpus akutagawa.lingcorpus --where '@document.title = 疑惑' --pattern '[lemma=落ちる] [lemma=て] [lemma in {来る, 行く}].' --context 4 --color never
ling matches --corpus akutagawa.lingcorpus --where '@document.title = 疑惑' --pattern '[text ~ 梁].' --context 3 --color never

The first line shows the 落ちて来る and 落ちて行く passages in Akutagawa’s 「疑惑」 (Giwaku, “Doubt”) with four tokens of context on each side (3 matches in the distributed corpus). If the quotation does not match the pattern, search for a word from the quotation with [text ~ word], as in the second line.

Read the first matching paragraph, second matching paragraph, third matching paragraph in Soranoha.

A matching count does not establish the claim. As with あのぼんやり in class 3, a match may fall outside what the question asks. Read the quoted paragraph to decide.

5. Compare with your own investigation

Put the two commands from your class 4 homework beside the agent’s commands.

1. If the question is the same, write which condition makes the counts differ: the pattern, the set, the --where range, or the unit. If the question differs, adapt the agent’s command to your question and write the count it gives on your author’s corpus.
2. Write down one passage that the agent found and that you missed, or one that you read and that the agent did not quote.
3. Write out one sentence of the agent’s explanation that the commands and the text could not confirm.

Exit note

Record these four items in your notes (.txt or .md) and submit the notes at the submission page. If you saved a conversation in section 3, submit that file as well.

Homework: assemble the classes 1–5 investigation into a report

Write the report under the six items of the evidence assignment. The submission location and deadline are in the submissions table.

Your records up to class 4 (question, prediction, commands, and CSV passages) and today’s checking record cover the six items. In the agent item, write only the interactions and checks you actually carried out.