Data Integrations
Data Integrations give your applications access to your organization's documents. On each request, Optiak searches the documents an application uses and adds the most relevant passages to the model's context, so answers draw on your own content without you sending whole documents.
Required role
Applies to
Why use it
Models only know what they were trained on and what you send them. Product documentation, support policies or customer-specific content usually lives in files and workspace tools, out of the model's reach. Data Integrations bring that content into Optiak once and let the Gateway find the relevant parts for each request, so your services don't have to build retrieval themselves.
Keeping each source in its own Data Integration also lets you control which application uses which content. For example, a support assistant can use the support policies without seeing internal engineering notes.
How it works
A Data Integration connects one source of documents to Optiak: files you upload, or content Optiak syncs from a tool your organization already uses. Optiak keeps its own copy of the documents and prepares them for retrieval.
Retrieval on each request
Every chat completion or response request becomes a question to your documents. Optiak reads what the user is asking, finds the passages across the application's Data Integrations that speak to it, and places them in the context the model sees. The model answers with your knowledge in front of it, and your application sends nothing extra.
Retrieval is selective: only passages that are actually relevant make it into the context. When nothing in your documents fits the request, the request reaches the model as it was sent, with no padding.
Retrieval happens after the input checks of Guardrails and before the Context Engine shapes the final context, so the retrieved knowledge counts toward the same context budget as the rest of the request. It applies to streaming and non-streaming requests.
When an application can use a Data Integration
An application can use a Data Integration's documents only when all of these hold:
- The Data Integration is active. Inactive ones are kept, but they are never synced or used by applications.
- Its scope includes the application. The scope says which applications or projects may use it; see Set it up.
- The application has it enabled.
A request can't add Data Integrations its application doesn't use.
Sources
Each Data Integration reads from one source. File Upload takes files you upload to Optiak; the other sources are connected sources that Optiak syncs from a tool your organization already uses, through an organization-wide connection.
Connected sources sync when an Organization Admin starts a sync; changes in the source don't reach Optiak until then. Each sync adds new documents, updates changed ones and removes documents that are no longer in the source. Optiak only reads from the source and never changes its content. A Data Integration must be active to sync.
File upload
Upload files directly to Optiak, with no connection or sync needed. When you create the Data Integration, the files you choose upload when you save it; after that, files you add or delete take effect right away.
| Setting | Value |
|---|---|
| File types | PDF, Markdown (.md), plain text (.txt), Word (.docx), PowerPoint (.pptx) |
| Files per upload | Up to 10 |
| File size | Up to 50 MB per file |
Google Drive
Sync the documents in one Google Drive folder. You give the folder ID or the folder URL when you create the Data Integration; Optiak reads the folder through an organization-wide Google Drive connection. To connect an account, see Connect Data Sources.
| Setting | Value |
|---|---|
| What is synced | Files directly inside the folder. Subfolders and files in the trash are not synced. |
| File types | Google Docs, Google Slides, PDF, Markdown, plain text, Word (.docx), PowerPoint (.pptx). Other files are skipped. |
| Folder size | Up to 100 supported files. A larger folder fails to sync. |
| File size | Files over 50 MB are skipped. |
Notion
Sync the pages of a Notion workspace. Optiak reads the pages that are shared with its Notion integration, through an organization-wide Notion connection. To connect a workspace and choose the pages to share, see Connect Data Sources.
| Setting | Value |
|---|---|
| What is synced | The pages shared with the Optiak integration |
| Number of pages | Up to 100. If more pages are shared, the sync fails. |
| Page size | Pages over 50 MB are skipped. |
Set it up
Setting up a Data Integration takes two steps. First, an Organization Admin creates it and chooses its Scope: which applications or projects may use it. The scope can be the whole organization, selected projects, selected applications, or a mix of projects and applications. Then each of those applications decides whether to turn it on. For who can do each step, see Members and Access.
In the organization
- In the console, open Company setup, then Data integrations, and click Add data integration to pick a source from the catalog.
- Give it a name and a Scope, and keep Setup active turned on. For a connected source, connect the account from the same form; see Connect Data Sources for what to prepare in the provider.
- For File Upload, add the files. For a connected source, click Sync on the Data Integration.
A Data Integration with no scope is kept, but no application can use it yet. When you delete one, no application can use its documents any more.
In the application
In the console, open the application, then Configuration, then Data integrations. Add data integration lists the Data Integrations whose scope includes the application; add the ones it needs and keep them enabled. Changes apply to the next request.
Limits and behavior
- A request with no text from the user skips retrieval.
- An application with no enabled Data Integrations skips retrieval for every request.
Related
- Connect Data Sources: connect the accounts that connected sources read from.
- Context Engine: how the retrieved passages fit into the context sent to the model.
- Gateway overview: where Data Integrations act in a request.