Corpus Builder

1. What Corpus Builder is for

A corpus is a deliberately assembled collection of texts. Corpus Builder is ADAR's private source-management workspace: it helps you record where texts came from, inspect extraction, retain originals, correct working copies, and choose a defensible study.

It is connected to—but distinct from—Literary & Corpus Stylistics. Corpus Builder organizes the reusable library. Stylistics investigates selected targets and references, presents candidate findings, supports researcher review, and produces analytical reports.

Sign in to create or change a private collection. The public source catalogue is an introduction to available discovery services, not a list of texts automatically cleared for reuse. Collections are owned by the authenticated account; another user's corpus is not made public by appearing in your workspace.

Corpus Builder and local stylistics analysis do not invoke the main ADAR analysis pipeline or change its subscription allowances. Owner manuscript generation, where available, is a separate provider-backed action.

2. Create your first corpus

1. Open Corpus Builder. Use ADAR's home page or dashboard, or open the workspace.
2. Create a collection. Use the + control beside Corpus library. Supply a clear name, a description, and selection criteria.
3. Add a small source first. Choose Add pasted text or Upload files, or use source discovery.
4. Record the source and rights. Give the text a title and relevant metadata, enter a meaningful rights note, and confirm the rights review.
5. Open Review / edit. Inspect the working text against the original source. Correct extraction and transcription problems.
6. Mark the document ready. A draft is not eligible for analysis. Readiness is your explicit review decision, not an accuracy certificate.
7. Assign the study role. Use target or reference as appropriate, then check the study-selection controls.
8. Run the selected study. Choose ready targets and, optionally, references. You can select documents across collections.
9. Inspect the Stylistics report. Review evidence, internal checks, comparisons, warnings, and the recorded corpus snapshot.
10. Export what you need. Keep the corpus record and the analytical report; they serve different purposes.

3. Plan the collection before filling it

Start with a question and an inclusion rule. “All accessible texts” is rarely a sufficient sampling rationale. Decide what counts as one work, which editions are eligible, what dates and languages matter, and which genres or communicative contexts belong together.

A useful collection description might be: “Publicly reusable inaugural speeches from a defined period; complete speeches only; official transcripts where available; introductions and editorial apparatus excluded explicitly.” This is a planning example, not a claim that any particular speech is legally reusable.

• Selection criteria: explain why a text belongs and why another is excluded.
• Unit of analysis: distinguish an original work, a chapter, an excerpt, and a technical analysis part.
• Comparability: consider period, genre, language, topic, audience, medium, and editorial practice.
• Version control: record edition and transcription differences rather than treating them as interchangeable.
• Sampling bias: accessible or searchable texts may not represent the wider population you want to describe.

Use separate corpora when they help explain your design—for example, a target collection and a comparison collection. A study can draw ready documents from more than one collection, so organization need not dictate the final comparison.

4. Add pasted text or files

Pasted text

Select the intended collection and choose Add pasted text. Enter the title, text, author, year, genre, language information, source URL, tags, and rights details where available. Preserve literary line breaks and paragraph divisions relevant to the study.

For text you wrote specifically for practice, say so in the rights note. For a published or transcribed work, identify the relevant permission, licence, or basis for use. The confirmation is your acknowledgement; ADAR does not determine legal clearance.

Uploaded files

Upload files accepts DOCX, PDF, TXT, Markdown, CSV, and JSON. Convert legacy DOC to DOCX first. Choose the intended collection before uploading, provide the shared source/rights information, and check the resulting document list after the request completes.

Text extraction does not preserve all Word or PDF features. Tables, page columns, footnotes, text boxes, verse layout, running headers, and hyphenated line endings need checking. A file that opens in a PDF reader is not necessarily a file with reliable extractable text.

Structured batches

CSV needs a text column. Supported document fields can include a title, author, year, genre, language, role, source URL, tags, and rights information. Use the form's rights acknowledgement; do not assume a value inside an uploaded file substitutes for the review decision.

JSON accepts an array of document objects or {"documents": [...]}. An exported corpus record is valuable for audit, but uploading JSON is not a guaranteed restoration of every trusted import field, review state, version, or provenance record. Check the resulting documents explicitly.

5. Discover and import sources

Use the source-discovery panel to search a provider. Inspect the result, its supplied rights, and its scope before choosing Review rights & import where offered. Imported texts are not automatically ready for analysis.

Practical search advice

• Begin with an identifiable title, author, subject, or provider record.
• For Gutenberg, a numeric ebook ID is useful when general search is unavailable.
• For scholarly work, inspect the full-text licence rather than relying on an abstract or an open-access badge.
• Check imported word counts and text boundaries. An XML article body is not a PDF facsimile and may omit figures or other material.
• If a provider is unavailable, retain the error and try later or use another permitted source. The builder does not replace failed retrieval with invented content.

The builder does not scrape arbitrary websites, bypass paywalls, use restricted lending access, or download from arbitrary user-entered source URLs. Source URLs in metadata identify provenance; they are not a general-purpose web crawler.

6. Record rights and provenance

A useful record answers: What is this work? Which version did I use? Where did I obtain it? When did I retrieve it? What permission or licence applies? What did extraction include or leave out?

Where supplied by a trusted import, the record can retain provider and record IDs, source and download URLs, retrieval date, rights, extraction scope, and warnings. You should still verify the bibliographic description and applicable rights.

For your own uploads, record the source URL and an informative rights note. “Found online” is not a licence. Private access inside ADAR does not automatically authorize redistribution of the source or inclusion of full text in a shared export.

Attribution and modification disclosures travel into the selected study and available reports, but provenance is not legal clearance. Respect creator attribution, licence conditions, institutional policy, confidential material, and jurisdictional differences.

7. Read scanned PDFs with OCR

OCR converts page images into editable text. Corpus Builder performs recognition locally with Tesseract; the source is not sent to an external recognition provider. Recognition can make mistakes even when a page looks clear.

Choose a recognition mode

Set the OCR language before upload

Use installed Tesseract codes, such as eng for English, ara for Arabic, or ron for Romanian. For mixed-language pages, a supported combination such as eng+ron can be used. The OCR language is a recognition setting, not a promise of equivalent linguistic analysis for every language.

A missing language produces an explicit error; the builder does not silently substitute English. If recognition is poor, first check the language, page clarity, orientation, cropping, typography, and presence of a faulty hidden text layer.

Review recognition details

1. Upload the permitted scan with the intended mode and language.
2. Open the imported document using Review / edit.
3. Expand the recognition-details disclosure.
4. Read the per-page method, confidence estimate, and uncertain-word hints.
5. Compare every OCR page against your retained scan. Correct names, numbers, punctuation, diacritics, paragraph boundaries, and omitted or reordered text.
6. Only then mark the document ready.

OCR imports always begin as drafts, even if upload metadata asks for ready status. The original recognized text, source-file hash, selected language, and per-page extraction details remain part of the record. Corrections belong in the working text.

The current budget is 20 OCR pages and 90 seconds per upload batch, with up to 25 seconds for a page. Multiple PDFs share the batch budget. Blank or unreadable scanned pages, timeouts, unsafe image sizes, or unsupported files fail explicitly rather than silently dropping pages or saving a partially recognized batch. Remove genuinely blank pages from a retained copy or split the upload appropriately.

8. Review, correct, and mark ready

Open Review / edit for the document. Inspect title, language, source, rights, genre, tags, working text, extraction warnings, and any alternate-edition notice.

• Original text: the retained extracted or recognized text. It is not overwritten when you correct the working copy.
• Working text: the editable version you intend to use, subject to any recorded exclusions.
• Version and hashes: identifiers that help connect a source state to a later study. A matching hash identifies the same recorded content, not legal permission or scientific accuracy.
• Draft: review is unfinished and the document cannot be selected for analysis.
• Ready: you have explicitly reviewed the current state and made it eligible.

Preserve original wording rather than “improving” literary style. Correct extraction errors, but document substantive editorial choices. Do not standardize dialect, spelling, capitalization, or punctuation merely because it looks unusual; those may be the features you need to study.

Changing text requires renewed review. After editing, confirm the ready status you intend to save and inspect the resulting record. If you need to undo the working copy, Reset to original restores the retained original, clears exclusions, and returns the document to draft. This discards your current working corrections, so keep any necessary record before confirming.

9. Exclude material without losing it

Use exclusions for material that belongs in the source record but not in the analysis—for example, a publisher's note, a repeated editorial header, or a Gutenberg licence block. Explain your rule consistently across the collection.

Each exclusion needs a start, an end, and a reason. Positions are zero-based Unicode code-point offsets in the current working text. The start is included; the end is excluded. They are not page numbers, word numbers, or byte offsets. Browser UTF-16 positions can differ for some characters, so do not assume a position copied from another tool uses the same convention.

The record retains the source and the exclusion reason. Analysis text is prepared without the excluded ranges, inserting paragraph boundaries rather than accidentally joining words on either side. Inspect the analysis word count and review the intended scope before marking ready.

Editing the working text can change positions. Recheck all ranges after corrections. Do not use exclusions to hide counterevidence; state and justify editorial removal in your method.

10. Handle duplicates and alternate editions

The builder detects exact duplicates after whitespace and case normalization. It skips matching items rather than storing the same normalized text repeatedly, including duplicates within one upload batch. This is not fuzzy similarity detection.

Because normalization ignores case and whitespace differences, inspect a skip if those differences matter to your research. Keep your source files and explain any edition distinction rather than assuming the import created both versions.

Texts with the same title and author but different content can be retained as possible alternate editions. The warning asks you to compare them; it does not decide which is authoritative or establish that one is corrupt.

Before selecting a study, avoid unintentionally including overlapping editions or using the same underlying text as both target and reference. Copies in separate collections do not become independent works.

11. Organize and find documents

Use descriptive corpus names and consistent metadata. Tags can record project-relevant categories such as period, genre, subcorpus, transcription status, or edition. A tag is your label, not an automatically verified classification.

Use the workspace's search and metadata filters to narrow the visible list, including review state and language where available. Clear filters before concluding that a document is missing. Draft and ready counts help distinguish stored sources from analysis-eligible material.

The role shown beside a document helps organize the current study. Check roles and selections again before running an analysis, especially after changing collections or reloading the page. A document merely being visible in a collection does not mean it is selected.

12. Select targets and references

1. Choose ready documents for the Target subset: the material your question investigates.
2. Optionally choose ready documents for the Reference subset: the comparison baseline.
3. Use the role/selection controls to check the exact subsets. You can select across collections.
4. Confirm that no original source is selected twice or on both sides of the comparison.
5. Give the study a meaningful report title and run the selected evidence into ADAR's stylistics workflow.
6. Inspect the resulting report and corpus snapshot before interpreting counts.

Choose references for a reason. Comparable genre, period, language, medium, and topic often matter more than merely accumulating a large word count. A mismatch can produce a genuine numerical contrast with an unhelpful explanation.

Without references, you can investigate supported local patterns but cannot claim external deviation against a corpus that was not supplied. The report's internal checks do not create a missing baseline.

13. Understand long-work segmentation

The reusable library can retain works larger than a single direct Stylistics document. For analysis, a long work is divided into source-preserving parts of at most 20,000 words.

The selected study is bounded to 30 parts and 100,000 total target/reference words, with available Stylistics capacity also enforced. An oversized selection fails explicitly. It is not silently shortened to fit.

If you exceed the study limits, narrow the selected works or curate and record an explicit excerpt in the working text. If a scan exceeds its OCR budget, split that upload separately; an OCR batch limit is not the same thing as an analysis word limit.

Technical parts are not independent works. Dispersion groups them by the recorded original work. Splitting at word boundaries can affect local observations near a boundary; note this when discussing sentence patterns, repetition, or local contrast.

These are the current limits of Corpus Builder's selected stylistics study. They should not be confused with an assertion that all ADAR venues have the same limits or that any finite server can process an unlimited input.

14. Understand snapshots and later changes

A report created from the builder pins a corpus snapshot: the analyzed selection, source versions and hashes, collection criteria, attribution, rights, exclusions, provenance, and original-work identity available at that point.

Later library corrections or deletion do not silently rewrite that pinned selection. A new version requires a new study if you want the new text reflected in the results.

Internal validation checks the stored evidence, supported rules, provenance, and calculations. It is not independent empirical validation, a certification of OCR accuracy, or confirmation that your source licence permits sharing.

15. Export and share responsibly

Corpus-library exports

• JSON: the fuller corpus record with originals, working text, and provenance. Use it for audit and preservation; do not assume it is a complete automatic restore mechanism.
• CSV: a source/version manifest for inspection or a spreadsheet. It is not a full-text export of every document.

Analytical-report exports

After the selected study opens in Stylistics, export the analytical report as PDF or HTML, the coding sheet as CSV, and the fuller structured report as JSON. Full snapshot metadata and text are available in report JSON. Use the Stylistics export guide for the distinctions.

Source attribution, rights, retrieval, and modification disclosures travel into available analytical reports and owner manuscript exports. Inspect the actual export before sharing and keep original source files separately.

16. A small worked workflow

This exercise teaches the controls. Both passages below are invented for practice; they are not authentic literary quotations or statistically adequate research corpora.

1. Create a corpus named Practice: repeated openings. Record that it contains invented teaching material.
2. Add a target titled Practice target with the text below. Record English as its language and identify it as a teaching example in the rights note.
3. Open Review / edit, verify line breaks, and mark the target ready.
4. Add the reference passage below as Practice reference, review it, and mark it ready.
5. Select the first document as target and the second as reference. Check that neither is selected on both sides.
6. Run the study and inspect exact evidence for the repeated phrase. Review the report's internal checks and source snapshot.
7. Accept or reject a candidate with a reason, then export the report and corpus record.

Use the exercise to ask how the software connects a pattern to an exact passage. Do not use the tiny reference to infer an author's distinctive vocabulary or claim reliable significance.

17. Current limits at a glance

Limits protect bounded, explicit processing. An error is preferable to an apparently complete study with missing pages or undisclosed truncation. Check current on-screen limits if a later version changes them.

18. Troubleshooting

19. Before-analysis checklist

20. Plain-language glossary

Corpus: a defined collection of texts assembled for a purpose.

Draft / ready: awaiting source review / explicitly reviewed and eligible for selection.

Exclusion: a recorded range retained in the source record but omitted from the prepared analysis text.

Hash: a content fingerprint that helps identify a recorded file or text version; not a quality score.

OCR: recognition of editable text from an image of a page.

Original / working / analysis text: retained extraction / editable copy / the text prepared for the selected study after recorded exclusions.

Provenance: recorded origin, retrieval, rights information, and processing history.

Snapshot: the pinned source selection and version information attached to a report.

Target / reference: the material under investigation / the material selected as a comparison baseline.

Next: build your collection, then follow the Literary & Corpus Stylistics guide to review and interpret the analysis.