Uwazi ML
Labelling is the training data
Uwazi learned to read its own documents.
Juan M. Cornejo
Uwazi ML
A notebook
In Uwazi’s machine learning, the labels researchers already give are what the model learns from.
- Notes2
- Suggestions, sketched3
- Labelling is the training data4
- Two labels, six documents5
- Try it6
- The extractors7
- Labelling a value8
- From label to suggestion9
- In short10
Uwazi learned to read its own documents. Every suggestion is something a researcher can accept or correct.
1Uwazi ML
Notes
- Researchers fill properties by hand, such as a date, a court or a name, document after document.
- The model learns from the values they have already filled in. That work was done anyway, and now it is also the training data.
- A researcher can leave any value out of the training if it is a bad example.
- The model suggests values for the other documents, each with the sentence it was read in.
- Accept writes a suggestion into its record, and nothing is written before it. The exception is a run set to auto-accept, which this notebook does not show.
- There is no Reject. A suggestion that is not accepted stays a suggestion.
- The model on this page is a simulation. It is a rule that looks for the words before a date. No trained model runs here.
2Notes
Suggestions, sketched
Figure 1. The suggestions view, sketched in code. The date is marked in a few documents, and the others come back as suggestions.
3Uwazi ML
Chapter 1
Labelling is the training data
In Uwazi’s Information Extraction, a researcher fills in the date of judgment on a few documents, as they would anyway. Filling in a value by hand is called labelling, and those labels are what the model learns from.
The model then suggests the date for the other documents, each with the passage it was read in. The researcher accepts the suggestions that are right.
4Labelling is the training data
Two labels, six documents
| Document | Labelled | Suggested | Score | State |
|---|---|---|---|---|
| Labelled by hand: the training | ||||
| Mora v. State of Valdora | 21 March 2019 | 21 March 2019 | 83 | Matches |
| Ibarra and others v. Republic of Corin | 14 November 2020 | 14 November 2020 | 83 | Matches |
| Suggested | ||||
| Hale v. Northmarch | 8 July 2016 | 83 | Empty | |
| Teodoro v. State of Valdora | 26 October 2018 | 83 | Empty | |
| Association of River Fishers v. Corin | 3 February 2022 | 83 | Empty | |
| Quist v. Northmarch | none | Empty | ||
Two dates are marked by hand. The simulation learns the words that come before them: its, judgment, on, delivered. Every other document gets the date that follows the same words, with the sentence it came from. The score is how much of what the simulation learned appears before that date. The last document has no judgment yet, so it gets no suggestion.
State compares a document’s own value with the suggestion: Matches when they agree, Empty when the document has no value yet.
Table 1. One round on six invented judgments, using Uwazi’s suggestion states. The model is a rule-based simulation, not a trained model.
5Uwazi ML
Try it
Mark the date of judgment in two or three documents, then find suggestions.
Figure 2. Uwazi’s review loop, rewritten for this page, with a rule-based simulation in place of the model. Mark dates, find suggestions and accept them. Uwazi accepts the ones you select; [accept the pending ones] is this page’s shortcut and selects them all (Uwazi has no Accept all). Uwazi has no Reject, and neither does this page.
6Try it
The extractors
- 1An extractor: one property to learn
- 2Its property, Date published
- 3Where the value is read from
Plate 1. Uwazi’s extractors. Each is one property in a set of templates, and Document Publication learns the Date published property, the kind of value this notebook labels.
7Uwazi ML
Labelling a value
- 1The passage, marked in the document
- 2Search the field’s options
- 3A value ticked by hand: a label
- 4Options found in the text are highlighted
- 5Save writes the labels
Plate 2. Labelling in Uwazi. The researcher marks the value in the document and ticks it in the field. This property lists participants; the step is the same for a date.
8Labelling a value
From label to suggestion
Figure 3. The loop. Labels train the model, the model suggests values with their passages, and accepted values join the labels.
- 1A few values, filled in by hand, are the training data.
- 2The model suggests values for the other documents.
- 3What is accepted becomes a value, and can train the next round.
9Uwazi ML
In short
A researcher’s own labels train the model, and each suggestion comes back with the sentence it was read in. A suggestion changes a record only when the researcher accepts it, unless a run is set to auto-accept.
The case study covers the rest: Paragraph Extraction, which keeps a whole passage instead of a value, and how a suggestion is shown on its page.
another notebook Uwazi Preserve
10In short
