Lot acceptance

A quality check splits messages into lots by executor: maesense with a given model, or a person. With lot acceptance, nobody has to judge each lot by eye: a sampling plan from the ISO 2859-1 standard says how many messages to check and how many errors a lot may have, and the decision follows from what the reviewers find.

Acceptance works with the per-executor sample. With a simple or stratified sample, a person still decides on each lot, as described in Quality checks.

AQL in plain words

• AQL (acceptable quality limit) is the share of messages with errors, in percent, that you are ready to live with. A lot that is about this good is accepted almost every time.
• There are two classes of errors with their own AQL: defects (wrong sentiment, an extra topic; 2.5% by default) and minor defects (a missing topic, a wrong author; 6.5% by default).
• A lot is accepted only when both classes pass.
• AQL is not a promise that every accepted lot is that good: a sample never sees everything. The chart on the settings screen shows how likely a lot is to pass at each share of errors.

Inspection level and the plan

• The plan depends on how many messages the executor has in the check (the lot size) and on the inspection level. Level II is the usual choice; level I checks fewer messages, level III more; the special levels S-1…S-4 check very few and fit cheap, frequent checks.
• From the lot size and level comes a code letter, and from the letter and AQL the plan: how many messages to check (n), the most errors that still accept the lot (Ac) and the fewest that reject it (Re). For example, a lot of 1,500 messages at level II gets letter K; at AQL 2.5 the plan checks 125 messages and accepts the lot with up to 7 defects.
• If the plan asks for as many messages as the lot has, the whole lot is checked.
• The two classes may need samples of different size. The lot sample is the larger one; each class is judged on the first messages of the sample, as many as its plan needs. The order of the sample is random, so its beginning is a random sample too.

Single and double sampling

• Single: one sample; the lot is accepted when the errors found are not more than Ac.
• Double: a smaller first sample. Few errors accept the lot at once, many reject it, and in between a second sample of the same size is drawn from the messages not yet checked. Reviewers get these messages like any others. The errors of both samples together then decide. On average a double plan checks fewer messages.

The decision

• The lot is decided by the system “ISO 2859-1 acceptance” as soon as every message its plan needs has a final verdict. A disputed message waits for the expert, and so does the decision.
• A message deleted before its verdict counts neither way: the next message of the random order takes its place.
• The decision cannot be changed, but a user who decides on lots can comment on it.
• Corrections of the checked messages are written into the session either way, accepted or not.

When a lot is not accepted

People who decide on lots and the executor (if a person) get an email. The lot appears under Needs your action in the Quality section, and the sidebar shows how many lots wait.

• maesense lot: the page suggests retagging, with a warning: the same model with the same instruction repeats the same errors, so fix the instruction first. Retagging is an ordinary session start with its cost; nothing starts by itself. Choose Retag or Leave as it is.
• A person's lot: Return to the executor or Hand over to another member who may tag and gave no verdict in this lot. That person gets an email and finds the lot under To rework, with a link to the keyboard mode over exactly the lot's messages. After the rework, Check again creates a new check over the same messages.

Inspection state of an executor

Each executor has a state per class of errors, kept separately for defects and minor defects. The executor page shows it, the switching score, the latest lots and every change.

• Normal: the start.
• Tightened: when 2 of the last 5 lots or fewer were not accepted. Plans are stricter. Normal returns after 5 lots accepted in a row.
• Reduced: lighter plans. The system offers it when the switching score reaches 30 and production is steady; a user who decides on lots switches it on. A lot not accepted, or a change of the instruction, returns normal inspection.
• Discontinued: after 5 lots not accepted on tightened inspection. New lots are still checked, but a person decides on them, and the company owner gets an email. Once the cause is fixed, Resume starts again at tightened.

The switching score grows on normal inspection while lots pass comfortably and drops to 0 as soon as one does not. For maesense, “steady production” means the instruction has not changed since the score started; editing the instruction resets the score.

Only lots decided by a plan change the state. Lots decided by hand do not.

Reading the chart

The operating characteristic shows, for each share of messages with errors in a lot, how likely the plan is to accept it. Two marks help: the chance to accept a lot exactly at the AQL (should be high), and the share of errors at which a lot passes only one time in ten. A steeper curve tells good lots from bad ones more sharply but needs more messages.

Settings and roles

• Acceptance settings in the Quality section: on or off, level, single or double sampling, and the two AQL values. Saving publishes a new version of the review scheme; checks already created keep theirs. Needs the right to create checks (editors and the owner).
• Choosing what happens to a lot, switching to reduced inspection and resuming need the right to decide on lots.

Details: Quality checks, Roles and permissions.