Uploads

Flowise allow users to upload files/images/audios from the chat. In this section, we will go through each and see how to enable the feature and using it.

Image

Certain Chat Models allow user to input images:

• ChatOpenAI

• AzureChatOpenAI

• ChatAnthropic

• AWSChatBedrock

• ChatGoogleGenerativeAI

When Allow Image Upload is enabled, images can be uploaded from chat interface.

To perform the same using API:

import requests
API_URL = "http://localhost:3000/api/v1/prediction/<chatlfowid>"

def query(payload):
    response = requests.post(API_URL, json=payload)
    return response.json()
    
output = query({
    "question": "Can you describe the image?",
    "uploads": [
        {
            "data": 'data:image/png;base64,iVBORw0KGgdM2uN0', # base64 string or url
            "type": 'file', # file | url
            "name": 'Flowise.png',
            "mime": 'image/png'
        }
    ]
})

async function query(data) {
    const response = await fetch(
        "http://localhost:3000/api/v1/prediction/<chatlfowid>",
        {
            method: "POST",
            headers: {
                "Content-Type": "application/json"
            },
            body: JSON.stringify(data)
        }
    );
    const result = await response.json();
    return result;
}

query({
    "question": "Can you describe the image?",
    "uploads": [
        {
            "data": 'data:image/png;base64,iVBORw0KGgdM2uN0', //base64 string or url
            "type": 'file', //file | url
            "name": 'Flowise.png',
            "mime": 'image/png'
        }
    ]
}).then((response) => {
    console.log(response);
});

Audio

Under Chatflow Configuration, user can select a Speech to Text module. Supported integrations are:

• OpenAI

• AssemblyAI

• LocalAI

When enabled, users can speak directly into microphone and speech will be transcribed into text.

To perform the same using API:

import requests
API_URL = "http://localhost:3000/api/v1/prediction/<chatlfowid>"

def query(payload):
    response = requests.post(API_URL, json=payload)
    return response.json()
    
output = query({
    "uploads": [
        {
            "data": 'data:audio/webm;codecs=opus;base64,GkXf', #base64 string
            "type": 'audio',
            "name": 'audio.wav',
            "mime": 'audio/webm'
        }
    ]
})

async function query(data) {
    const response = await fetch(
        "http://localhost:3000/api/v1/prediction/<chatlfowid>",
        {
            method: "POST",
            headers: {
                "Content-Type": "application/json"
            },
            body: JSON.stringify(data)
        }
    );
    const result = await response.json();
    return result;
}

query({
    "uploads": [
        {
            "data": 'data:audio/webm;codecs=opus;base64,GkXf', //base64 string
            "type": 'audio',
            "name": 'audio.wav',
            "mime": 'audio/webm'
        }
    ]
}).then((response) => {
    console.log(response);
});

Files

Users can upload files from the chat as well. Uploaded files will be upserted on the fly to the vector store. However, to enable file uploads feature, there are a few pre-requisites.

• A file-upload supported vector store must be present in the chatflow.

  • Pinecone

  • Milvus

  • Postgres

  • Qdrant

  • Upstash

• If you have multiple vector stores on a chatflow, you can only turn on file upload for one vector store at a time.

• At least one Document Loader node should be connected to the vector store's Document input.

• Supported Document Loader:

  • CSV File

  • Docx File

  • Json File

  • Json Lines File

  • PDF File

  • Text File

  • Unstructured File

One or multiple files can be uploaded on the chat:

Here's how it works:

• Uploaded files will have the metadata updated with the chatId.

• This will allow the file to be associated with the chatId.

• When querying, an OR filter is being applied:

  • Metadata contains flowise_chatId and the value equals to the current chat session id

  • Metadata does not contains flowise_chatId

An example of a vector embeddings upserted on Pinecone: