Text File
Load data from text files.
Last updated
Load data from text files.
Last updated
The Text File loader enables you to load and process content from various text-based file formats. It supports multiple file types and provides flexible options for text splitting and metadata handling.
Support for multiple text-based file formats
Multiple file loading capability
Text splitting support
Customizable metadata handling
Storage integration support
Base64 file handling
Multiple output formats
The loader supports a wide range of text-based file formats:
Text files (.txt)
Web files (.html, .aspx, .asp, .css)
Programming languages:
C/C++ (.cpp, .c, .h)
C# (.cs)
Go (.go)
Java (.java)
JavaScript/TypeScript (.js, .ts)
PHP (.php)
Python (.py, .python)
Ruby (.rb, .ruby)
Rust (.rs)
Scala (.sc, .scala)
Solidity (.sol)
Swift (.swift)
Visual Basic (.vb)
Markup/Style:
CSS/LESS/SCSS (.css, .less, .scss)
Markdown (.md, .markdown)
XML (.xml)
LaTeX (.tex, .ltx)
Other:
Protocol Buffers (.proto)
SQL (.sql)
RST (.rst)
Txt File: One or more text files to process
Accepts files from local upload or storage
Supports multiple file selection
Text Splitter: A text splitter to process the extracted content
Additional Metadata: JSON object with additional metadata to add to documents
Omit Metadata Keys: Comma-separated list of metadata keys to exclude
Format: key1, key2, key3.nestedKey1
Use * to remove all default metadata
Document: Array of document objects containing:
metadata: File metadata and custom fields
pageContent: Extracted text content
Text: Concatenated string of all extracted content
Each document contains:
pageContent: The main content from the text file
metadata:
Default file metadata
Additional custom metadata (if specified)
Filtered metadata (based on omitted keys)
The loader supports two file source modes:
Direct Upload: Files uploaded directly through the interface
Storage Integration: Files accessed through the storage system
Format: FILE-STORAGE::filename.txt
Supports organization and chatflow-specific storage
Handles both single and multiple file processing
Supports base64 encoded file content
Automatically handles different file encodings
Memory-efficient processing of large files
Preserves file metadata when needed
Supports text splitting for large documents
Handles escape characters in output text
Integrates with organization-specific storage
This section is a work in progress. We appreciate any help you can provide in completing this section. Please check our to get started.