eviStreams workflow guide
Independent review and consensus in systematic reviews
Updated
How are disagreements between reviewers resolved?
Reviewers check study data independently. An adjudicator then compares conflicting entries with their evidence and records a final decision. eviStreams supports this workflow while retaining the individual entries and the decision history for review.
Agree on the review process before starting
Define who extracts, who reviews, and who resolves disagreements. Decide when reviewers can see AI suggestions and one another’s entries. Configure the project’s review settings to match the protocol before work begins.
Independent entries are useful because a mismatch can expose a different interpretation of the field, a missed table, or a transcription error. Reviewers should apply the same definitions, units, and time points rather than guessing what another reviewer intended.
Resolve a disagreement against the paper
Use the disputed field as the starting point, then check the underlying evidence. Agreement counts can help organize the queue, but they do not replace the adjudicator’s judgment.
- Open the disagreement and compare the individual reviewer entries.
- Inspect the supporting source for each entry and confirm the arm, outcome, and time point.
- Check the extraction field’s definition against the review protocol.
- Record the supported consensus value or a different value justified by the source.
- Review the decision history when questions arise about how the final dataset was produced.
Use disagreements to improve field definitions
Repeated disagreements about denominators or follow-up times may indicate an ambiguous form rather than careless review. Clarify the instruction and decide which earlier entries need another check. Keep the change connected to the review process so the same rule is applied consistently.
An AI entry agreeing with one reviewer is still a suggestion. Two matching entries may share the same interpretation error. The final answer should be supported by the source and the agreed field definition.
Keep a traceable final dataset
The final dataset should distinguish reviewed decisions from proposals that still need checking. Before export or synthesis, inspect unresolved disagreements and missing values. Preserve the source context needed to revisit a decision later.
eviStreams brings extraction, independent review, and adjudication into one workspace. The published paper describes the review design; your team remains responsible for the protocol and the recorded decisions.
| Entry | What the adjudicator checks |
|---|---|
| Reviewer 1 records 12 months | Does the cited passage refer to the protocol-defined outcome? |
| Reviewer 2 records 18 months | Is this a different outcome or follow-up time? |
| Consensus decision | Record the time point supported by the paper and the field definition. |