> ## Documentation Index
> Fetch the complete documentation index at: https://docs.helix-db.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Upload and organise documents

> Upload Markdown, text, PDF, CSV, JSON, Parquet and other files to a Helix Foundry workspace, keep them in document sets, see how ontologies read each format, and which ontologies read each one

Guide

Documents carry what your tables don't: who signed a contract, what was agreed, who
referred whom. They belong to the workspace and live in **Connectors**, in document
sets: folders of documents an ontology reads together. A load reads the documents of
the sets its version reads: text and PDFs for the names and facts they hold, linked to
the records your tables hold, and table files, such as CSV exports, as tables of their
own.

## Document sets

In the Connectors explorer, the **Documents** folder holds a folder per set, and each
set holds its documents. A workspace gets a set named **General** with its first upload.

![The Connectors overview beside the Connectors explorer, whose Documents folder holds the General and Board minutes sets and their documents](https://assets.helix-db.com/docs/foundry/data/overview.png)

Some connections write documents of their own, in sets of their own: Granola keeps each
meeting note's summary in **Notes**, and its transcripts in **Transcripts** when you turn
them on; an [API source](https://docs.helix-db.com/foundry/data/api-sources.md) read by entity extraction keeps its
newest 500 pieces in its set. Each sync updates them, so they can't be uploaded to,
moved, renamed or deleted, and they don't count toward the workspace's uploads. A
connection keeps the newest 1,000 documents of each of its sets.

Owners and admins manage the sets from their rows in the explorer:

* **New set**, on the Documents row, names a set and gives it an optional description.
  The agent that drafts the next version reads the description.
* **Describe** changes a set's name and description.
* The pencil beside a set's name, or <kbd>F2</kbd> on its row, renames it in place.
* **Delete** removes a set that holds no documents. Move or delete its documents first.

A workspace holds up to 20 sets. Set names are unique in the workspace, ignoring case
and accents.

## Upload

Upload in any of these ways; Foundry checks each file before it uploads it, then sends
it straight to storage and shows its progress.

* **Upload documents**, in the explorer's head or on the Connectors overview, uploads
  into General, or the oldest set when there is no General.
* **Upload into**, on a set's row, uploads into that set.
* Drop files onto a set in the explorer. Files dropped anywhere else on the explorer or
  the Connectors overview go where **Upload documents** puts them.
* On New ontology and in Generate, **Upload documents** uploads in place.

| Format | Extensions | Size | Ontologies read it |
| - | - | - | - |
| Markdown, MDX and text | `.md`, `.markdown`, `.mdx`, `.txt` | Up to 1 MiB | As text, for its names and facts |
| PDF | `.pdf` | Up to 3 MiB and 150 pages | As the text Foundry reads from it |
| CSV, TSV, JSON, JSONL and Parquet | `.csv`, `.tsv`, `.json`, `.jsonl`, `.ndjson`, `.parquet` | Up to 3 MiB | As a table of its own |

A workspace holds up to 200 uploaded documents. A name is at most 255 characters, with
no slash, backslash or control character. Compressed files aren't taken.

Each upload says which set it went into and which ontologies read that set.

![The Connectors explorer with contract.md uploaded into General, and a note that Renewals reads it with its first load](https://assets.helix-db.com/docs/foundry/data/documents-uploaded.png)

A workspace keeps one copy of each file. Uploading a file it already holds, under any
name, doesn't add another: the upload says **Already uploaded** and names the set the
file is in.

![An upload of contract.md that says Already uploaded: it's in General](https://assets.helix-db.com/docs/foundry/data/upload-duplicate.png)

## A document's page

Select a document in the explorer to open its page: what it holds, its set, its size and
when it was uploaded. Foundry shows each document by its type:

| Type | Shown as |
| - | - |
| Markdown and MDX | Rendered, with headings, lists, tables and links; images show as their alt text and are never loaded. MDX is never run: its components and expressions show as written |
| Text | As it is |
| JSON | A tree you open node by node |
| CSV, TSV, JSONL and Parquet | A table of the first 5,000 rows, with the rows it couldn't read |
| PDF | **Text**: the text Foundry read from it, page by page; **Pages**: its pages as they look |

Markdown, JSON, CSV, TSV and JSONL also have **Source**, to read the file's own text.

A table file's page also says how Foundry read it: its columns, as **Rows**. Until
Foundry has read it, which takes a moment after the upload, a generation leaves it out.

![A CSV document's page in Connectors showing its rows as a table, how ontologies load it as a table of its own, and its details](https://assets.helix-db.com/docs/foundry/data/document-csv.png)

A PDF's text is read once, after its upload. Until then, it says **Reading its text**,
and a generation leaves it out. A scanned PDF has no text to read, so ontologies can't
read it; it stays in its set to view. The same goes for a PDF that is encrypted,
damaged, or longer than 150 pages.

![A two-page master services agreement PDF in Connectors: the text Foundry read from each page, its Text and Pages tabs, the ontology that reads it and its details](https://assets.helix-db.com/docs/foundry/data/document-pdf.png)

**Details** lists its media type, size, SHA-256 and ID. **Ontologies** lists each
ontology that reads the document's set, or whose current load holds the document, with
what its current load made of it and **Open in** to read it there.

![contract.md's page in Connectors with its Markdown rendered, and Customer contracts and Renewals under Ontologies with their statuses](https://assets.helix-db.com/docs/foundry/data/document.png)

Each ontology's status says where the document stands in its current load: **Not read
yet**, **Read**, **Needs review**, **Duplicate**, **Couldn't be read** or **Not read
by its version**.

On a document's page, owners and admins can:

* **Move** it to another set (or use **Move to another set…** on its grip in the
  explorer). Loads that read it keep what they read until their next load, and every
  ontology that reads the new set reads it there.
* **Delete** it. It's removed from Foundry at once; what loads read from it stays in
  their ontologies' databases.
* Rename it in place in the explorer, with the pencil beside its name. A rename may
  change its extension.

## How ontologies read documents

An ontology doesn't read every document in the workspace. Each version reads the sets
its generation picked:

* **Generate** picks whole sets. The agent reads up to five of the newest documents in
  each and drafts routes for them, which say what each set's documents may fill, such
  as a contract's signing date and the customer it was signed with. See
  [Generate and version](https://docs.helix-db.com/foundry/ontologies/generate-and-version.md#generate-a-draft).
* **Loads** read every text document and PDF in the version's sets: its sections, the
  names it mentions, and the facts it states, each with the quote it came from. A
  document in a set no route reads is stored, not read, and shows **Not read by its
  version**. What a load can't settle, such as a name the database doesn't hold, an
  ambiguous name, a quote it couldn't verify or a fact that conflicts with your tables,
  goes to **Needs review** instead of becoming a record.
* **Table files** are tables of their own, which a version's mappings load as they load
  a synced table. Text in their cells isn't read for names and facts. A generation that
  picks a set drafts mappings for the table files in it.
* A document uploaded or moved into a set the current load reads is read into that load
  as it arrives.

An existing ontology reads a new set only after a new generation picks it, or a
[version edit](https://docs.helix-db.com/foundry/ontologies/generate-and-version.md#edit-a-version) routes it, and
that version is loaded.

Reading documents uses AI models: a load sends each document's text to the
workspace's [AI models](https://docs.helix-db.com/foundry/workspace/ai-models.md) to find its names and facts and
to check them. What it costs counts toward the month's
[loads cap](https://docs.helix-db.com/foundry/workspace/usage.md).

## In an ontology

An ontology's **Documents** lists what it reads of the workspace's documents: those in
the sets its version reads, and those its current load holds. Open it from the
ontology's pages in the Ontologies panel, from the document count in the ontology's
header, or from a load's **Documents read** and **Review documents**. It is read-only:
**Open in Connectors** goes back to the documents in Connectors.

There, the Ontologies panel becomes the ontology's documents explorer: a row back to
the ontology, a filter, and its sets as folders of their documents. Each document's dot
says what the current load made of it: orange needs review, red couldn't be read,
green read, grey waiting or not read. A set counts its documents that need review.

![Customer contracts' Documents page listing contract.md in General with one item needing review, beside the ontology's documents explorer with General open and contract.md's orange dot](https://assets.helix-db.com/docs/foundry/ontology/documents.png)

Open a document there to see its text as the load read it, shown as the document is
(Markdown rendered, text as it is), with names underlined, quoted facts tinted and
anything that needs review marked. Beside it, **Facts**, **Entities** and **Needs
review** list what the load found, each linked to its record, and **Details** describe
the document. Select a mark to follow it: a name opens its entity's page, a quote the
page of the entity its fact is about, and a review mark its card in **Needs review**.
Until a load holds it, the document shows as uploaded. Use the **Load** picker to read
the document as another load saw it.

![contract.md as the current load read it, rendered as Markdown with names underlined and facts tinted, beside its two signed\_with facts with their quotes, and the ontology's documents explorer](https://assets.helix-db.com/docs/foundry/ontology/document.png)

## Who can do what

Owners and admins upload, move, rename and delete documents, and create, describe and
delete sets. Members read them.

## Next steps

> **[Generate an ontology](https://docs.helix-db.com/foundry/ontologies/generate-and-version.md)**
>
> Pick the document sets a version reads.

> **[Load data](https://docs.helix-db.com/foundry/ontologies/load-data.md)**
>
> Write tables and documents into the database.

> **[Connect data](https://docs.helix-db.com/foundry/data/connect-data.md)**
>
> The Connectors explorer, and how to arrange it.

> **[Records and entities](https://docs.helix-db.com/foundry/explore/records-and-entities.md)**
>
> See every place a record comes from.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.