# NVIDIA NIM

## Local

### Important Note on Running NIM with Flowise

If an existing NIM instance is already running (e.g., via NVIDIA’s ChatRTX), starting another instance through Flowise **without checking for an existing endpoint** may cause conflicts. This issue occurs when multiple `podman run` commands are executed on the same NIM, leading to failures.

For support, refer to:

* [**NVIDIA Developer Forums**](https://forums.developer.nvidia.com/) – For technical issues and questions.
* [**NVIDIA Developer Discord**](https://discord.gg/nvidiadeveloper) – For community engagement and [announcements](https://discord.com/channels/1019361803752456192/1340013505834647572).

### Prerequisite

1. Setup [NVIDIA NIM locally with WSL2](https://docs.nvidia.com/nim/wsl2/1.0.0/getting-started.html).

### Flowise

1. **Chat Models** > Drag the **Chat NVIDIA NIM** node > Click **Setup NIM Locally**.

<figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-456a272b7e69873db7eb3a12265a625c6b9a6879%2Fnvidia-nim-local-1.png?alt=media" alt=""><figcaption></figcaption></figure>

2. If NIM is already installed, click **Next**. Otherwise, click **Download** to start the installer.

<figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-9b8b696f3064e309c08ef29b3c7a9cf21495c99d%2Fnvidia-nim-local-2.png?alt=media" alt=""><figcaption></figcaption></figure>

3. Select a model image to download.

<figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-941dc67d63af105288d65682b92b37e4e3ce0595%2Fnvidia-nim-local-3.png?alt=media" alt=""><figcaption></figcaption></figure>

4. Once selected, click **Next** to proceed with the download.

<figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-f1de04d9b571372003e491bde473bc573618d490%2Fnvidia-nim-local-4.png?alt=media" alt=""><figcaption></figcaption></figure>

5. **Downloading Image** – Duration depends on internet speed.

<figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-be5dd54872cb4cff882a3ac78621cc1486d9f814%2Fnvidia-nim-local-5.png?alt=media" alt=""><figcaption></figcaption></figure>

6. Learn more about [Relax Memory Constraints](https://docs.nvidia.com/nim/large-language-models/1.7.0/configuration.html#environment-variables).\
   The **Host Port** is the port for the container to map to the local machine.

<figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-1e736e31d32b549b68cb4d5c7356786c8ec5c092%2Fnvidia-nim-local-6.png?alt=media" alt=""><figcaption></figcaption></figure>

7. **Starting the container...**

<figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-b58fa30e2f882d69dd7acba5f53d47760bd96db7%2Fnvidia-nim-local-7.png?alt=media" alt=""><figcaption></figcaption></figure>

*Note: If you already have a container running with the selected model, Flowise will ask you if you want to reuse the running container. You can choose to reuse the running container or start a new one with a different port.*

<figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-ac7008cd4765cb6e9710a48f35cf14f324a5df3b%2Fnvidia-nim-container-exists.png?alt=media" alt=""><figcaption></figcaption></figure>

8. **Save the chatflow**
9. [🎉](https://emojipedia.org/party-popper/) **Voila!** Your **Chat NVIDIA NIM** node is now ready to use in Flowise!

<figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-0d835c5e2226a71afcf116a6a6c318119b78b120%2Fnvidia-nim-local-8.png?alt=media" alt=""><figcaption></figcaption></figure>

## Cloud

### Prerequisite

1. Log in or sign up to [NVIDIA](https://build.nvidia.com/).
2. From the top navigation bar, click NIM:

<figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-94f1ca243f7efb64dbb07cf192d14481a2cf93d0%2Fimage%20(247).png?alt=media" alt=""><figcaption></figcaption></figure>

3. Search for the model you would like to use. To download it locally, we will be using Docker:

<figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-5a2e9cf129559bf593ebf81b3ce15eb360215ec8%2Fimage%20(248).png?alt=media" alt=""><figcaption></figcaption></figure>

4. Follow the instructions from the Docker setup. You must first get an API Key to pull the Docker image:

<figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-fdfdba23899a9c98f4135e7f464befcb785f2ba1%2Fimage%20(249).png?alt=media" alt="" width="563"><figcaption></figcaption></figure>

### Flowise

1. **Chat Models** > drag **Chat NVIDIA NIM** node

<figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-7f62e6b4ac6a4386a2b49adfafb1e654b0d60619%2Fimage%20(250).png?alt=media" alt=""><figcaption></figcaption></figure>

2. If you are using NVIDIA hosted endpoint, you must have your API key. **Connect Credential** > click **Create New.** However if you are using local setup, this is optional.

<div align="left"><figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-4bb42530d1b6af8c2e954bed7452d2e630622933%2Fimage%20(251).png?alt=media" alt=""><figcaption></figcaption></figure> <figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-2ad90f06a0f09e65a7ed27add288bf678d7ff01b%2FScreenshot%202024-12-23%20180712.png?alt=media" alt=""><figcaption></figcaption></figure></div>

3. Put in the model name and voila [🎉](https://emojipedia.org/party-popper/), your **Chat NVIDIA NIM node** is now ready to be used in Flowise!

<figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-72e5341b449fc14cf36d696080260d2844eaf299%2Fimage%20(252).png?alt=media" alt=""><figcaption></figcaption></figure>

### Resources

* [NVIDIA LLM Getting Started](https://docs.nvidia.com/nim/large-language-models/latest/getting-started.html)
* [NVIDIA NIM](https://build.nvidia.com/microsoft/phi-3-mini-4k?snippet_tab=Docker)


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.flowiseai.com/integrations/langchain/chat-models/nvidia-nim.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.