[ENH]: CIP-5: Large Batch Handling Improvements Proposal (#1077)

- Including only CIP for review.

Refs: #1049

## Description of changes

*Summarize the changes made by this PR.*
 - Improvements & Bug fixes
	 - New proposal to handle large batches of embeddings gracefully

## Test plan
*How are these changes tested?*

- [ ] Tests pass locally with `pytest` for python, `yarn test` for js

## Documentation Changes
TBD

---------

Signed-off-by: sunilkumardash9 <sunilkumardash9@gmail.com>
Co-authored-by: Sunil Kumar Dash <47926185+sunilkumardash9@users.noreply.github.com>

docs/CIP_5_Large_Batch_Handling_Improvements.md

@@ -0,0 +1,59 @@
+# CIP-5: Large Batch Handling Improvements Proposal
+
+## Status
+
+Current Status: `Under Discussion`
+
+## **Motivation**
+
+As users start putting Chroma in its paces and storing ever-increasing datasets, we must ensure that errors
+related to significant and potentially expensive batches are handled gracefully. This CIP proposes to add a new
+setting, `max_batch_size` API, on the local segment API and use it to split large batches into smaller ones.
+
+## **Public Interfaces**
+
+The following interfaces are impacted:
+
+- New Server API endpoint - `/pre-flight-checks`
+- New `max_batch_size` property on the `API` interface
+- Updated `_add`, `_update` and `_upsert` methods on `chromadb.api.segment.SegmentAPI`
+- Updated `_add`, `_update` and `_upsert` methods on `chromadb.api.fastapi.FastAPI`
+- New utility library `batch_utils.py`
+- New exception raised when batch size exceeds `max_batch_size`
+
+## **Proposed Changes**
+
+We propose the following changes:
+
+- The new `max_batch_size` property is now available in the `API` interface. The property relies on the
+  underlying `Producer` class
+  to fetch the actual value. The property will be implemented by both `chromadb.api.segment.SegmentAPI`
+  and `chromadb.api.fastapi.FastAPI`
+- `chromadb.api.segment.SegmentAPI` will implement the `max_batch_size` property by fetching the value from the
+  `Producer` class.
+- `chromadb.api.fastapi.FastAPI` will implement the `max_batch_size` by fetching it from a new `/pre-flight-checks`
+  endpoint on the Server.
+- New `/pre-flight-checks` endpoint on the Server will return a dictionary with pre-flight checks the client must
+  fulfil to integrate with the server side. For now, we propose using this only for `max_batch_size`, but we can
+  add more checks in the future. The pre-flight checks will be only fetched once per client and cached for the duration
+  of the client's lifetime.
+- Updated `_add`, `_update` and `_upsert` method on `chromadb.api.segment.SegmentAPI` to validate batch size.
+- Updated `_add`, `_update` and `_upsert` method on `chromadb.api.fastapi.FastAPI`  to validate batch size (client-side
+  validation)
+- New utility library `batch_utils.py` will contain the logic for splitting batches into smaller ones.
+
+## **Compatibility, Deprecation, and Migration Plan**
+
+The change will be fully compatible with existing implementations. The changes will be transparent to the user.
+
+## **Test Plan**
+
+New tests:
+
+- Batch splitting tests for `chromadb.api.segment.SegmentAPI`
+- Batch splitting tests for `chromadb.api.fastapi.FastAPI`
+- Tests for `/pre-flight-checks` endpoint
+
+## **Rejected Alternatives**
+
+N/A