Class 4: Comparing authors and models
Tuesday, October 27, 2026
Have your class 3 class3-prepared folder, your homework CSV, and your saved worksheet dh-b-movement.json at hand. Choose one author from the table in section 1; two students may choose the same author.
| What you use | From class 3 | Today |
|---|---|---|
| Text | 「蜘蛛の糸」 (“The Spider’s Thread”) and 103 works by Akutagawa Ryūnosuke | Selected works for 13 authors; one author per student |
| Search | class3.lx |
class4.lx (the same lemma set, plus counting by part of speech) |
| Record | CSV rows 1–10 | CSV rows 11–12 and your author’s values checked against authors.csv |
1. Set up your author’s prepared corpus
If you have not yet done so, submit your class 2 worksheet dh-b-movement.json at the submission page. Keep the page open: sections 5 and 6 use it.
If you no longer have dh-b-movement.json, save the sample worksheet and load it with 「保存したワークシートを開く」 (open a saved worksheet) in the worksheet of class 2 section 2. Rewrite the evidence of all seven rows in your own words, change the labels of any row that you judge differently, then save and submit the file. Do not submit a file whose evidence is still the sample’s.
Save your author’s ZIP from the table below into the class 2 working folder dh-b-2026. When you extract the ZIP, it creates the author’s folder (such as class4-akutagawa) with akutagawa.lingcorpus and the other files. On a computer without ling, first follow Set up ling.
In the commands below, replace class4-akutagawa.zip and class4-akutagawa with your author’s ZIP and folder names from the table. Replace akutagawa.lingcorpus with the corpus filename inside that folder. Start your terminal in the folder containing dh-b-2026; if you have already extracted the ZIP with your file manager, skip the unzip line.
cd dh-b-2026
unzip class4-akutagawa.zip
cd class4-akutagawa
ling corpus inspect akutagawa.lingcorpusIf inspect reports the number of works in the table and finishes with complete:, continue to the next section.
| Author | Folder name | Works | ZIP |
|---|---|---|---|
| 森鴎外 (Mori Ōgai) | class4-mori |
31 | 13 MB |
| 泉鏡花 (Izumi Kyōka) | class4-izumi |
31 | 15 MB |
| 国木田独歩 (Kunikida Doppo) | class4-kunikida |
19 | 3 MB |
| 夏目漱石 (Natsume Sōseki) | class4-natsume |
14 | 29 MB |
| 岡本綺堂 (Okamoto Kidō) | class4-okamoto-kido |
43 | 10 MB |
| 谷崎潤一郎 (Tanizaki Jun’ichirō) | class4-tanizaki |
22 | 13 MB |
| 室生犀星 (Murō Saisei) | class4-murou |
15 | 3 MB |
| 芥川龍之介 (Akutagawa Ryūnosuke) | class4-akutagawa |
70 | 10 MB |
| 梶井基次郎 (Kajii Motojirō) | class4-kajii |
13 | 2 MB |
| 堀辰雄 (Hori Tatsuo) | class4-hori |
16 | 5 MB |
| 夢野久作 (Yumeno Kyūsaku) | class4-yumeno |
25 | 12 MB |
| 岡本かの子 (Okamoto Kanoko) | class4-okamoto-kanoko |
25 | 7 MB |
| 太宰治 (Dazai Osamu) | class4-dazai |
56 | 13 MB |
The course uses a fixed selection of 380 works from Soranoha, grouped by the 13 authors. The selection includes Kunikida’s translation of 「糸くず」 and Mori’s translations of 「冬の王」 and 「みれん」; works.csv records the original author and the selected author’s role. The texts and bibliographic information come from the Soranoha release recorded in README.txt. Akutagawa’s selection contains 70 works, compared with 103 in class 3.
The first_published field preserves Soranoha’s first-publication information when available; year is extracted from that field and is blank when no year is recorded. Check the work’s Soranoha page before comparing publication periods.
| In the ZIP | Contents |
|---|---|
<author>.lingcorpus |
The author’s selected works analyzed with the novel UniDic, as a prepared corpus |
works.csv |
The list of works, including author and translator roles, first-publication information, the work’s source page, and its reader link |
class4.lx |
Today’s pattern file |
README.txt |
Versions of the tools and dictionary, and the selection rule |
How do I follow along without a terminal?
Open the search and summary outputs in the table below, generated from the distributed corpora and pattern file. Although the patterns are fixed, you can compare authors and check passages against the source text. The README lists the commands and versions used. The table over all 13 authors is authors.csv.
| Author | Saved output |
|---|---|
| 森鴎外 | Summary · Counts by work · 落ちる hits · 来る/行く passages |
| 泉鏡花 | Summary · Counts by work · 落ちる hits · 来る/行く passages |
| 国木田独歩 | Summary · Counts by work · 落ちる hits · 来る/行く passages |
| 夏目漱石 | Summary · Counts by work · 落ちる hits · 来る/行く passages |
| 岡本綺堂 | Summary · Counts by work · 落ちる hits · 来る/行く passages |
| 谷崎潤一郎 | Summary · Counts by work · 落ちる hits · 来る/行く passages |
| 室生犀星 | Summary · Counts by work · 落ちる hits · 来る/行く passages |
| 芥川龍之介 | Summary · Counts by work · 落ちる hits · 来る/行く passages |
| 梶井基次郎 | Summary · Counts by work · 落ちる hits · 来る/行く passages |
| 堀辰雄 | Summary · Counts by work · 落ちる hits · 来る/行く passages |
| 夢野久作 | Summary · Counts by work · 落ちる hits · 来る/行く passages |
| 岡本かの子 | Summary · Counts by work · 落ちる hits · 来る/行く passages |
| 太宰治 | Summary · Counts by work · 落ちる hits · 来る/行く passages |
2. Count vertical movement lemmas in your author’s corpus
class4.lx combines the vertical and deixis definitions from class 3 with pos, which counts tokens by part of speech. Each [...] is one token, and brackets written in a row match consecutive tokens. The key: line above each definition specifies how summarize groups the counts.
Definition in class4.lx |
Matches | Group counts by |
|---|---|---|
pos := []. |
Every token | upos, the part of speech |
verbs := [upos=VERB]. |
Tokens whose part of speech is VERB |
lemma |
vertical := [lemma in {上る, …}]. |
Tokens whose lemma is one of the 12 in the set | lemma |
deixis := (?<verb> [upos=VERB]) [lemma=て] (?<deictic> [lemma in {来る, 行く}]). |
Three consecutive tokens: a verb, て, then 来る or 行く | The verb’s lemma and the 来る/行く lemma |
(?<name> ...) is a named capture: it gives the matched part a name that you can use later. In deixis, two parts have names.
| Named part | Lemma in key: |
|---|---|
verb: the first verb token |
$verb.lemma |
deictic: the final 来る/行く token |
$deictic.lemma |
key: [$verb.lemma, $deictic.lemma] groups counts by the combination of these two lemmas.
Does the pattern count compounds without て, such as 走り行く?
deixis counts matching sequences of three consecutive tokens: a verb, て, and 来る/行く. Compound verbs such as 走り行く and 帰り来る are analyzed as one token with a single lemma such as 走り行く, so neither deixis nor [lemma=行く] matches them.
The following pattern counts one-token verbs whose lemma ends in 来る or 行く. 出来る and 心行く are excluded because they do not express movement.
key: [lemma]
deixis_compound := [upos=VERB & lemma ~ /(来る|行く)$/ & lemma not in {来る, 行く, 出来る, 心行く}].
This pattern can include compounds that do not express movement. It also misses compounds that the analyzer splits into two tokens.
In the distributed corpora it matches 4 tokens for Akutagawa, 37 for Kunikida (deixis: 188), and 47 for Izumi (deixis: 525). State which pattern produced each count you report.
authors.csv summarizes the results for all 13 authors. Every author’s counts use class4.lx, with the same lemma set and the same way of counting total verbs for the rate.
Use the distributed class4.lx without modifying it. To search with the lemma set that you revised in class 3, copy class4.lx to class4-mine.lx and edit that copy. State that you used class4-mine.lx with every count obtained from it.
ling check --patterns class4.lx
ling summarize --corpus akutagawa.lingcorpus --patterns class4.lx --top all > summary.txt
grep -w pos summary.txt | grep VERB
grep -w vertical summary.txt
grep -w vertical summary.txt | awk '{s+=$1} END {print s}'The awk command sums and prints the counts from the first column, one per vertical lemma.
A vertical count is the number of tokens whose lemma is in the set, not the number of movements. In Akutagawa’s 「文章」 (Bunshō, “writing”; Akutagawa_R_Bunshou.md), 肉の落ちた (“thin”) matches 落ちる but describes a face, with no movement. The next command lists the first 40 落ちる hits with the file name, the sentence number within the work, and the surrounding text:
ling matches --corpus akutagawa.lingcorpus --tsv --context 8 --limit 40 --pattern '[lemma=落ちる].' | cut -f3,5,9-Because the selected works for each author differ in length, compare counts as a rate per 10,000 verbs. The rate is the vertical total divided by the VERB count, times 10,000. In Akutagawa’s 70 works, the vertical total is 406, the VERB count is 30,878, and the rate is 131.5.
- Calculate the
verticaltotal, theVERBcount, and the rate, and record them in your notes. - Write the top three
verticallemmas in your notes. - Find one
落ちるhit in your author’s corpus that describes no movement, and record its wording and file name in your notes. If you find none, record which hits you inspected. - Compare the three values with your author’s row in
authors.csv, columnsverbs,vertical, andvertical_per_10k_verbs. If a value differs, write down which command gave the different count.
| Your value | Column in authors.csv |
|---|---|
The VERB count |
verbs |
The vertical total |
vertical |
| The rate, to one decimal place | vertical_per_10k_verbs |
| 落ちて来る, 落ちて行く (section 3) | 落ちる_来る, 落ちる_行く |
| 上って来る, 上って行く (section 3) | 上る_来る, 上る_行く |
3. Check how often verbs combine with 来る and 行く
In Akutagawa’s 70 works, 落ちる combines with 来る 10 times and 行く once, and 上る combines with 来る 10 times and 行く 13 times. A pair absent from the output has a count of 0.
- Compare your author’s four counts with the
authors.csvcolumns in the section 2 table. - Does
落ちるalso combine more frequently with来るin your author’s corpus? Write one sentence.
4. Choose a work and read two passages
ling matches --corpus akutagawa.lingcorpus --tsv --pattern '[lemma in {上る, 落ちる, 下りる}] [lemma=て] [lemma in {来る, 行く}].' | cut -f3 | sort | uniq -c | sort -rn | head
ling matches --corpus akutagawa.lingcorpus --where '@document.title = 杜子春' --tsv --context 12 --pattern '[lemma in {上る, 落ちる, 下りる}] [lemma=て] [lemma in {来る, 行く}].' | cut -f3,5,9-The first command counts matching three-token sequences per file: 上る, 落ちる, or 下りる, followed by て and then 来る or 行く, as in Figure 3.
The second shows the passages of one work: the file name, the sentence number within the work, and the surrounding text. Replace akutagawa.lingcorpus with your corpus filename and 杜子春 with the title value of the chosen file in works.csv.
「杜子春」 (Toshishun) has four such passages. In Kunikida’s corpus, only 「少年の悲哀」 (Kodomo no kanashimi, “childhood sorrow”) (Kunikida_D_Kodomonokanashi.md) has two. When no work by your author has at least two such passages, run the second command without --where '@document.title = 杜子春' and choose passages from two works. Without a terminal, the saved 来る/行く passages file for your author lists the same output.
- Choose the file with the most passages, select two passages, and read the full paragraph containing each one: open the work’s
readerlink inworks.csvand search for the wording shown byling. The linked examples above open the relevant paragraph directly. - Classify them with the same three judgments as in class 2 and add them as CSV rows 11–12. Write the work’s title in
partand the wording shown bylinginexpression. The other column values are those in the class 3 homework table. - Write in your notes the file name and sentence number of each passage, who moves, and where the narrator watches from.
5. Compare the class’s results
authors.csv holds the section 2 and 3 values for all 13 authors.
The submission page shows the label distributions, deferrals, and pairwise agreement for rows 2–7, separately for direction and the movement judgment. Outlines mark the passages where your labels differ from the majority.
Write down the authors with the highest and lowest rates per 10,000 verbs, and one possible reason based on the kind of work or the period.
Write down any author whose works do not combine
落ちるwith来るmore often than with行く.Choose a row with the lowest agreement. Look in the submitted criteria list for a criterion that could resolve the disagreement, and record that criterion in your notes. If none fits, write your own in the worksheet’s 「修正した基準」 (revised criterion) field.
When you save and resubmit the file, the list updates within a minute. If the page is unavailable, or no submissions are listed yet, compare your labels in worksheet section 4 with a partner’s saved file or with the sample worksheet, and write a criterion for a row where the judgments differ in your notes. A comparison with the sample is agreement with one reading, not the class’s agreement rate. If every label in your comparison matches, choose a row and record the wording that supports your shared judgment.
This rate is calculated from lemma-match counts, which can include passages with no movement. The counts also depend on the analysis dictionary and on which lemmas are included in vertical.
Whenever you describe a difference between authors, state that the selection includes children’s fiction and three translations, and that first-publication information is incomplete.
6. Compare a model’s extractions with your labels
For the seven worksheet passages, this section compares the direction, movement judgment, and actor labels with the results from rules, the LLM Qwen3.5-4B, and Perplexity Decider. For extracting movement expressions from whole paragraphs, it compares rules, GLiNER2.5, and Qwen3.5-4B.
The notebook displays saved results, so no model download is required. Downloading Python or marimo requires a network connection. Its interface is in Japanese.
Unpack the extraction comparison notebook (ZIP) and keep all files together, including notebook-theme.css and the hidden .marimo.toml. Run this command from that folder and open the URL that it prints in your browser. Keep the submission page open in another tab or window and switch between them in the steps below.
If uvx is not recognized, first install uv by following Run a course notebook.
- On the submission page, choose one passage with a wavy underline in the 規則 (rules), Qwen, or Decider column. The underline marks a method’s label where it differs from the most frequent submitted label. If the page is unavailable or no passage is underlined, choose row 2,
どこまでものぼって行けば, or row 5,下りろ. In the saved results, Qwen’s movement judgment for these two rows differs from the sample worksheet’s labels. - Switch to the notebook window and load your
dh-b-movement.jsonwith 「第2回のワークシートを開く」 (open the class 2 worksheet). If you do not load a file, the sample worksheet becomes the reference. - In the notebook’s 「同じ箇所を分類する」 (classify the same passages), compare each method’s direction, movement judgment, actor, and supporting wording for that passage with your labels, and record where they agree or disagree.
- In 「段落から探す」 (find in the paragraph), record whether each method found the worksheet expression and what else it extracted.
The notebook calculates agreement between your labels and each method’s output. Agreement shows consistency, not correctness. Seven passages, with rules written specifically for them, cannot establish accuracy on other passages.
For one disagreement, read the paragraph and record which case in the table below applies. If your labels match every method’s output, use the difference between Qwen and the sample worksheet in row 2 or 5.
Qwen and Decider received instructions to use the narrator’s perspective, judge whether the movement itself occurs under negation or inability, and treat commands or conditions alone as insufficient evidence that movement happened. The rule-based method classifies by verb ending. 「方法の違い」 (differences between methods) at the end of the notebook summarizes each method.
Keep your worksheet file or notes open beside the notebook, because the notebook does not display the criterion that you revised in class 2.
| Case | What to record |
|---|---|
| Your criterion or the sample worksheet’s criterion differs from the model’s | The two criteria, and the wording to which each applies |
| The paragraph does not settle the judgment | The wording that leaves it open |
| One side misread the paragraph | The wording that decides it, and the label it supports |
How do I run the models myself?
To use 「選んだ段落で実行」 (run on the selected paragraph), close the notebook and reopen it from the same folder with the command below. The command opens an environment with the model libraries. In a notebook opened with --no-sandbox, 「実行する」 fails with an import error because the libraries are missing.
This command downloads a few hundred MB of libraries on first use. After you switch to 「選んだ段落で実行」 and press 「実行する」, the notebook downloads and runs the models; the first run downloads several GB. The steps for an NVIDIA or AMD GPU on Linux are the same as for the NER comparison, with movement-extraction.py in place of ner-comparison.py.
Outside class, optional: how can a model score yes/no answers?
These English resources from marimo show ways to score yes/no answers, including methods that require further training. They use Qwen3.5 base models, Jev, and GLiNER2. The molab notebook evaluates each method on the same 500 questions from BoolQ, a set of English passages with yes/no questions.
| Resource | What to look for |
|---|---|
| From Qwen to Jev, How Does That Even Work? (marimo, 2026, English): 15 minutes 10 seconds | How to make a text-generation model choose between yes and no. Compare its yes/no scores with the option probabilities that Perplexity Decider returns in the course’s extraction comparison notebook. |
| Encoders Are All You Need™ (marimo, 2026, English): 15 minutes 19 seconds | How GLiNER2 computes a score for each supplied label. Compare the supplied labels and the outputs: a yes/no choice for a passage and question in the video, and extracted spans and relations in the course’s extraction comparison notebook. That notebook’s 「方法の違い」 (differences between methods) describes what GLiNER2.5 extracts. |
| fine tuning a jev like model (molab notebook, English) | In the final table, compare each method’s accuracy with 0.596, the accuracy of always answering yes. Read the saved results, code, and explanations without running it. |
To run the molab notebook, you need an NVIDIA GPU, and its Jev cells use an API key. Its Decider is a method from AnyJev, not Perplexity Decider.
Accuracy here is the share of answers that match BoolQ’s reference answers. As with the seven passages above, it does not show how a method performs on other kinds of text, such as Japanese fiction.
Exit note
Record these five items in your notes (.txt or .md) and submit the notes at the submission page.
| What to save | Contents |
|---|---|
| Your author’s values | The three values and four counts, and whether they match authors.csv (sections 2 and 3) |
| Two passages | CSV rows 11–12 (section 4) |
| A difference between authors | One difference in rate or bias and a possible reason (section 5) |
| Criterion | A row where judgments differed and a criterion that resolves the difference (section 5) |
| A disagreement with a model | One passage, which of the three cases applies, and the supporting wording (section 6) |
What does a completed exit note look like?
This example entry uses Akutagawa’s corpus and shows one reading. Record your own author’s values and your own judgments.
| What to save | Example entry |
|---|---|
| Your author’s values | VERB 30,878; vertical 406; rate 131.5 (406 ÷ 30,878 × 10,000). 落ちて来る 10, 落ちて行く 1, 上って来る 10, 上って行く 13. All seven match the akutagawa row of authors.csv. Top three vertical lemmas: 上る 117, 落ちる 116, 下りる 49. 落ちる with no movement: 腑に落ちない (“was not convinced”), Akutagawa_R_Futatsunotegami.md, sentence 200. In these 70 works, 落ちる combines with 来る more often than with 行く (10 and 1), but 11 matches is a small number. |
| Two passages | 11,路上,登って来る,裾模様の令嬢,up,narrator,CT+,labelled and 12,路上,下りて行っ,俊助,down,narrator,CT+,labelled. Notes: Akutagawa_R_Rojou.md has the most passages (5, tied with 「素戔嗚尊」). Sentence 254: a young woman climbs onto the stage; the narrator watches with 俊助 from the audience, so she comes toward him. Sentence 726: 俊助 goes down the stone steps; the narrator follows 俊助 away from the entrance where 大井 stands. |
| A difference between authors | Highest rate: Kunikida Doppo, 163.5; lowest: Dazai Osamu, 59.6. Possible reason, unchecked: the period. Kunikida’s 15 works with a recorded first-publication year date from 1897–1908 and Dazai’s 56 works from 1933–1945. One work, 「わかれ」, has 26 of Kunikida’s 164 vertical matches, so a single work may raise the rate. 落ちる does not combine with 来る more often than with 行く for Dazai (0 and 3), Okamoto Kanoko (4 and 5), Yumeno Kyūsaku (4 and 7), and Kajii Motojirō (1 and 3). The selection includes children’s fiction and three translations, and first-publication information is incomplete. |
| Criterion | Row 4, 上へ上へ一心によじのぼって来る. Criterion: label the movement from the narrator’s perspective; when the passage states that a character notices it, as with 「ふと気がつきますと」, name that character in the evidence field and keep the narrator as the source. |
| A disagreement with a model | Row 5, 下りろ. The sample worksheet has down, 罪人たち, and “Not stated whether it happens” (Uu); in 「同じ箇所を分類する」, Qwen3.5-4B gives down, 罪人たち, and “Does not happen / did not happen” (CT-), with 【下りろ】 as its evidence. The rules and Decider give Uu. Case: one side misread the paragraph. 「下りろ。下りろ。」と喚きました states only Kandata’s shout and does not say whether the sinners climb down, which supports Uu. In 「段落から探す」, the rules and Qwen extracted 下りろ; GLiNER2.5 extracted only 犍陀多 as a person and no movement expression. |
Homework: decide the question for your class 5 investigation
Allow 20–30 minutes.
In class 5, you compare an agent’s commands and claims with your own investigation and the source. Record your own investigation first.
- Write one question about your author that corpus counts can answer, and your prediction, in the question and prediction fields of the evidence record. The submission page lists the questions and predictions from the submitted records. Example: in this author’s selected works, does
落ちるcombine with来るmore often than with行く? - Run two commands that answer the question, and record the commands, numeric results, and units (token counts, work counts, or rates per 10,000 verbs). Without a terminal, use the saved outputs, record their commands from the README, and mark the counts as saved results that you have not rerun.
- Read one passage from the output in its full paragraph, and write one sentence on what the counts show and one on what they do not.
- Submit the CSV, including rows 11–12, and the evidence record at the submission page before class 5.
What does a completed evidence record look like?
This example entry uses Akutagawa’s corpus and a question different from the one above, and shows one reading. Write your own question, prediction, commands, and counts.
grep -w deixis summary.txt | grep -F '"$verb.lemma":"上る"'
ling matches --corpus akutagawa.lingcorpus --tsv --pattern '[lemma=上る] [lemma=て] [lemma=来る].' | cut -f3 | sort -u | wc -l
ling matches --corpus akutagawa.lingcorpus --tsv --pattern '[lemma=上る] [lemma=て] [lemma=行く].' | cut -f3 | sort -u | wc -l| Field in the evidence record | Example entry |
|---|---|
| Question | In Akutagawa’s 70 works in akutagawa.lingcorpus, does 上る combine with 行く more often than with 来る? |
| Prediction before searching or running | 来る more often, as with 落ちる. |
| Work and URL | The 70 works in class4-akutagawa.zip (listed in works.csv); quoted work: 「路上」, https://soranoha.org/works/000132_000879/ |
| Version or file hash | Searched with ling 0.7.0. Corpus built with ling 0.5.0 and unidic-novel-202512 from Soranoha release c373bb085971c6a88d8aa1c164788c53ebf304ab501a6f5188c3077ac5482916 (from README.txt) |
| Commands, notebook, or tool, with its version | The three commands above, after the section 2 summarize command; ling 0.7.0 |
| Unit counted, and the rows or passages included | Token sequences 上る + て + 来る/行く; works containing at least one |
| Result | 行く 13 tokens in 11 works; 来る 10 tokens in 5 works. In these works, 上る combines with 行く more often, against my prediction. |
| Passages checked | Akutagawa_R_Rojou.md, sentence 794, 微笑が唇に上って来る: a smile appears on 俊助’s lips; no person or thing moves up. |
| Missed or misleading example | Four of the ten 来る matches are 唇に上って来る (a smile or words): 「路上」 794, 「素戔嗚尊」 256 and 1218, 「保吉の手帳から」 13. |
| What the result cannot establish | The counts do not show how many climbing movements the works describe, because a match can describe a smile or words. |