Skip to main content

Files Datasources

Beak AI Studio enables agents to use files as data sources, supporting a wide range of formats and integration methods—including direct uploads and document repository connections.

Supported File Types​

  • CSV (Comma-Separated Values)
  • Excel (XLS, XLSX)
  • PDF (for text extraction)
  • DOCX (Word documents)
  • Plain text (TXT)
  • JSON
  • XML
  • Other common document and data formats

Integration Methods​

  • Upload Files:
    • Upload files directly from your local machine via the Studio interface.
    • Supports single and bulk (multi-file) uploads.
  • SharePoint Documents:
    • Connect to SharePoint document libraries or folders for enterprise document access.
    • Requires SharePoint site URL and appropriate authentication (OAuth2 or API token).

How to Add a File Datasource​

  1. Navigate to Data Sources

    • In Studio, go to the "Data Sources" tab.

  2. Add New File Source

    • Click "Add Data Source" and select "File".

  3. Choose Integration Method

    • Upload Files: Select files from your computer (supports multiple files at once).
    • SharePoint Documents: Enter the SharePoint folder URL and authenticate.

  4. Vector Database Option

  5. Save

    • Save the datasource for use in agent workflows.

Vector Store Option When Uploading Files​

When uploading files, you can choose to enable or disable storage in a vector store. This choice affects how your data can be searched, retrieved, and used by agents.

What is a Vector Store?​

A vector store stores file contents as high-dimensional embeddings, enabling semantic search and contextual retrieval. This is especially useful for unstructured data (text, documents, PDFs) and AI-powered applications.

Choosing to Use a Vector Store​

  • Advantages:

    • Enables semantic search (find similar content, not just exact matches)
    • Supports retrieval-augmented generation (RAG) for LLMs
    • Improves document Q&A, summarization, and knowledge extraction
    • Scales well for large document collections

  • Example Scenarios:

    • Uploading a knowledge base of support articles for semantic Q&A
    • Storing research papers for similarity search and literature review
    • Enabling agents to answer questions based on uploaded contracts or policies

Opting Out of Vector Database Storage​

  • Advantages:

    • Reduces storage and indexing costs for simple or structured data
    • Ensures compliance when vectorization is not permitted (e.g., sensitive or regulated data)
    • Simpler for tabular data or when only basic file access is needed

  • Example Scenarios:

    • Uploading CSVs or Excel files for direct data extraction or transformation
    • Storing files where only metadata or raw content is required
    • Compliance-driven workflows where embeddings are not allowed

How to Decide​

  • Use a vector store if you need advanced search, semantic retrieval, or plan to use LLM-powered features on unstructured text.
  • Opt out if you only need basic file access, are working with structured/tabular data, or have compliance constraints.

Example Use Cases​

  • Bulk Data Import: Upload multiple CSV or Excel files to process large datasets (vector store optional; often not needed for tabular data).
  • Document Summarization: Extract and summarize content from PDF or DOCX reports (vector store recommended for semantic search and retrieval).
  • Enterprise Document Search: Connect to a SharePoint library and enable agents to search and retrieve documents (vector store enables semantic search).
  • Data Transformation: Parse and transform JSON or XML files for downstream workflows (vector store optional).
  • Contract Q&A: Upload contracts and enable agents to answer questions using retrieval-augmented generation (vector store highly recommended).

Bulk Upload Features​

  • Select and upload multiple files in a single operation.
  • Files are processed in parallel for efficiency.
  • Vector store option can be applied to all files in a bulk upload.

File Datastore & Indexing Options​

Beak AI Studio supports new modes for file indexing and retrieval to suit diverse use cases:

  • Semantic snippet retrieval: quickly retrieve the most relevant sentence or passage using AI embeddings.
  • Best for FAQ, Q&A, and knowledge applications where users need concise, meaningful answers.
  • Combine structured queries (filters, facets, column selection) with semantic retrieval for hybrid precision+relevance.
  • Supports advanced analytics, mixed-format files, and enterprise use cases.

Full Content Access​

  • Directly browse, search, and navigate every part of a file (for in-depth review, compliance, or full context needs).
  • Allows agents/users to reference entire file contents, not just extracted snippets.

Selecting a Mode:

  • Choose your preferred file retrieval/indexing mode when configuring a new file datasource or upload.
  • Modes can be combined or changed based on workflow requirements.

Best Practices​

  • Consider your use case and compliance requirements before enabling vector store storage.
  • For unstructured text and AI-powered search, vector stores provide significant advantages.
  • For structured data or compliance-sensitive files, opt out to minimize risk and cost.