# NVIDIA NIM

## Local

### Important Note on Running NIM with Flowise

If an existing NIM instance is already running (e.g., via NVIDIA’s ChatRTX), starting another instance through Flowise **without checking for an existing endpoint** may cause conflicts. This issue occurs when multiple `podman run` commands are executed on the same NIM, leading to failures.

For support, refer to:

* [**NVIDIA Developer Forums**](https://forums.developer.nvidia.com/) – For technical issues and questions.
* [**NVIDIA Developer Discord**](https://discord.gg/nvidiadeveloper) – For community engagement and [announcements](https://discord.com/channels/1019361803752456192/1340013505834647572).

### Prerequisite

1. Setup [NVIDIA NIM locally with WSL2](https://docs.nvidia.com/nim/wsl2/1.0.0/getting-started.html).

### Flowise

1. **Chat Models** > Drag the **Chat NVIDIA NIM** node > Click **Setup NIM Locally**.

<figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-456a272b7e69873db7eb3a12265a625c6b9a6879%2Fnvidia-nim-local-1.png?alt=media" alt=""><figcaption></figcaption></figure>

2. If NIM is already installed, click **Next**. Otherwise, click **Download** to start the installer.

<figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-9b8b696f3064e309c08ef29b3c7a9cf21495c99d%2Fnvidia-nim-local-2.png?alt=media" alt=""><figcaption></figcaption></figure>

3. Select a model image to download.

<figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-941dc67d63af105288d65682b92b37e4e3ce0595%2Fnvidia-nim-local-3.png?alt=media" alt=""><figcaption></figcaption></figure>

4. Once selected, click **Next** to proceed with the download.

<figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-f1de04d9b571372003e491bde473bc573618d490%2Fnvidia-nim-local-4.png?alt=media" alt=""><figcaption></figcaption></figure>

5. **Downloading Image** – Duration depends on internet speed.

<figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-be5dd54872cb4cff882a3ac78621cc1486d9f814%2Fnvidia-nim-local-5.png?alt=media" alt=""><figcaption></figcaption></figure>

6. Learn more about [Relax Memory Constraints](https://docs.nvidia.com/nim/large-language-models/1.7.0/configuration.html#environment-variables).\
   The **Host Port** is the port for the container to map to the local machine.

<figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-1e736e31d32b549b68cb4d5c7356786c8ec5c092%2Fnvidia-nim-local-6.png?alt=media" alt=""><figcaption></figcaption></figure>

7. **Starting the container...**

<figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-b58fa30e2f882d69dd7acba5f53d47760bd96db7%2Fnvidia-nim-local-7.png?alt=media" alt=""><figcaption></figcaption></figure>

*Note: If you already have a container running with the selected model, Flowise will ask you if you want to reuse the running container. You can choose to reuse the running container or start a new one with a different port.*

<figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-ac7008cd4765cb6e9710a48f35cf14f324a5df3b%2Fnvidia-nim-container-exists.png?alt=media" alt=""><figcaption></figcaption></figure>

8. **Save the chatflow**
9. [🎉](https://emojipedia.org/party-popper/) **Voila!** Your **Chat NVIDIA NIM** node is now ready to use in Flowise!

<figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-0d835c5e2226a71afcf116a6a6c318119b78b120%2Fnvidia-nim-local-8.png?alt=media" alt=""><figcaption></figcaption></figure>

## Cloud

### Prerequisite

1. Log in or sign up to [NVIDIA](https://build.nvidia.com/).
2. From the top navigation bar, click NIM:

<figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-94f1ca243f7efb64dbb07cf192d14481a2cf93d0%2Fimage%20(247).png?alt=media" alt=""><figcaption></figcaption></figure>

3. Search for the model you would like to use. To download it locally, we will be using Docker:

<figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-5a2e9cf129559bf593ebf81b3ce15eb360215ec8%2Fimage%20(248).png?alt=media" alt=""><figcaption></figcaption></figure>

4. Follow the instructions from the Docker setup. You must first get an API Key to pull the Docker image:

<figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-fdfdba23899a9c98f4135e7f464befcb785f2ba1%2Fimage%20(249).png?alt=media" alt="" width="563"><figcaption></figcaption></figure>

### Flowise

1. **Chat Models** > drag **Chat NVIDIA NIM** node

<figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-7f62e6b4ac6a4386a2b49adfafb1e654b0d60619%2Fimage%20(250).png?alt=media" alt=""><figcaption></figcaption></figure>

2. If you are using NVIDIA hosted endpoint, you must have your API key. **Connect Credential** > click **Create New.** However if you are using local setup, this is optional.

<div align="left"><figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-4bb42530d1b6af8c2e954bed7452d2e630622933%2Fimage%20(251).png?alt=media" alt=""><figcaption></figcaption></figure> <figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-2ad90f06a0f09e65a7ed27add288bf678d7ff01b%2FScreenshot%202024-12-23%20180712.png?alt=media" alt=""><figcaption></figcaption></figure></div>

3. Put in the model name and voila [🎉](https://emojipedia.org/party-popper/), your **Chat NVIDIA NIM node** is now ready to be used in Flowise!

<figure><img src="https://823733684-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F00tYLwhz5RyR7fJEhrWy%2Fuploads%2Fgit-blob-72e5341b449fc14cf36d696080260d2844eaf299%2Fimage%20(252).png?alt=media" alt=""><figcaption></figcaption></figure>

### Resources

* [NVIDIA LLM Getting Started](https://docs.nvidia.com/nim/large-language-models/latest/getting-started.html)
* [NVIDIA NIM](https://build.nvidia.com/microsoft/phi-3-mini-4k?snippet_tab=Docker)