# Record Managers

***

Record Managers keep track of your indexed documents, preventing duplicated vector embeddings in [Vector Store](/integrations/langchain/vector-stores.md).

When document chunks are upserting, each chunk will be hashed using [SHA-1](https://github.com/emn178/js-sha1) algorithm. These hashes will get stored in Record Manager. If there is an existing hash, the embedding and upserting process will be skipped.

In some cases, you might want to delete existing documents that are derived from the same sources as the new documents being indexed. For that, there are 3 cleanup modes for Record Manager:

{% tabs %}
{% tab title="Incremental" %}
When you are upserting multiple documents, and you want to prevent deletion of the existing documents that are not part of the current upserting process, use **Incremental** Cleanup mode.

1. Let's have a Record Manager with `Incremental` Cleanup and `source` as SourceId Key

<div align="left"><figure><img src="/files/Uq02FhrjRsd6qQoaDGCZ" alt="" width="264"><figcaption></figcaption></figure> <figure><img src="/files/wi64G6IF1D1Hrq6u1BS9" alt="" width="410"><figcaption></figcaption></figure></div>

2. And have the following 2 documents:

| Text | Metadata         |
| ---- | ---------------- |
| Cat  | `{source:"cat"}` |
| Dog  | `{source:"dog"}` |

<div align="left"><figure><img src="/files/8WApYfqtJsPyV5xRasqL" alt="" width="202"><figcaption></figcaption></figure> <figure><img src="/files/Aj17hcUIGHo2N9SJ8C1f" alt="" width="563"><figcaption></figcaption></figure></div>

<div align="left"><figure><img src="/files/RlU9DD37qtU2XnzPrOlu" alt="" width="231"><figcaption></figcaption></figure> <figure><img src="/files/kCEj5fQV7bhjWpKd915V" alt="" width="563"><figcaption></figcaption></figure></div>

3. After an upsert, we will see 2 documents that are upserted:

<figure><img src="/files/o5DLYf97u3ChHF9y8ZMW" alt="" width="433"><figcaption></figcaption></figure>

4. Now, if we delete the **Dog** document, and update **Cat** to **Cats**, we will now see the following:

<figure><img src="/files/bsC9tLnyrKP33JarU3Cz" alt="" width="425"><figcaption></figcaption></figure>

* The original **Cat** document is deleted
* A new document with **Cats** is added
* **Dog** document is left untouched
* The remaining vector embeddings in Vector Store are **Cats** and **Dog**

<figure><img src="/files/jYHn2ySsrPtXdZIskzSy" alt="" width="448"><figcaption></figcaption></figure>
{% endtab %}

{% tab title="Full" %}
When you are upserting multiple documents, **Full** Cleanup mode will automatically delete any vector embeddings that are not part of the current upserting process.

1. Let's have a Record Manager with `Full` Cleanup. We don't need to have a SourceId Key for Full Cleanup mode.

<div align="left"><figure><img src="/files/Uq02FhrjRsd6qQoaDGCZ" alt="" width="264"><figcaption></figcaption></figure> <figure><img src="/files/EV6XsD3IWMBx3qQ8alKe" alt="" width="407"><figcaption></figcaption></figure></div>

2. And have the following 2 documents:

| Text | Metadata         |
| ---- | ---------------- |
| Cat  | `{source:"cat"}` |
| Dog  | `{source:"dog"}` |

<div align="left"><figure><img src="/files/8WApYfqtJsPyV5xRasqL" alt="" width="202"><figcaption></figcaption></figure> <figure><img src="/files/Aj17hcUIGHo2N9SJ8C1f" alt="" width="563"><figcaption></figcaption></figure></div>

<div align="left"><figure><img src="/files/RlU9DD37qtU2XnzPrOlu" alt="" width="231"><figcaption></figcaption></figure> <figure><img src="/files/kCEj5fQV7bhjWpKd915V" alt="" width="563"><figcaption></figcaption></figure></div>

3. After an upsert, we will see 2 documents that are upserted:

<figure><img src="/files/o5DLYf97u3ChHF9y8ZMW" alt="" width="433"><figcaption></figcaption></figure>

4. Now, if we delete the **Dog** document, and update **Cat** to **Cats**, we will now see the following:

<figure><img src="/files/nZQOOphyaYb3PJzowZC6" alt="" width="430"><figcaption></figcaption></figure>

* The original **Cat** document is deleted
* A new document with **Cats** is added
* **Dog** document is deleted
* The remaining vector embeddings in Vector Store is just **Cats**

<figure><img src="/files/9t38e4Aog09LGmA8aqMd" alt="" width="527"><figcaption></figcaption></figure>
{% endtab %}

{% tab title="None" %}
No cleanup will be performed
{% endtab %}
{% endtabs %}

Current available Record Manager nodes are:

* SQLite
* MySQL
* PostgresQL

## Resources

* [LangChain Indexing - How it works](https://js.langchain.com/docs/how_to/indexing/#how-it-works)


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.flowiseai.com/integrations/langchain/record-managers.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
