This proposes an initial CLI
The CLI is installed when you installed `pip install chromadb`.
You then call the CLI with
`chroma run --path <persist_dir> --port <port>` where path and port are
optional.
This also adds `chroma help` and `chroma docs` as convenience links -
but I'm open to removing those.
To make this easy - I added `typer` (by the author of FastAPI). I'm not
sure this is the tool that we want to commit to for a fuller featured
CLI, but given the extremely minimal footprint of this - I don't think
it's a one way door.
<img width="1477" alt="Screenshot 2023-08-23 at 4 59 54 PM"
src="https://github.com/chroma-core/chroma/assets/891664/30374228-d303-41e1-8e9e-188b7f8532d4">
***
#### TODO
- [x] test in fresh env - i think i need to add `typer` as a req
- [ ] consider expanding the test to make sure the service is actually
running
- [x] hide the test option from the typer UI
- [x] linking to a getting started guide could be interesting at the top
of the logs
## Description of changes
*Summarize the changes made by this PR.*
- Improvements & Bug fixes
- Add explicit env for persist directory defaults to docker compose so
that it can hint at usage
- New functionality
- ...
## Test plan
Existing tests
- [x] Tests pass locally with `pytest` for python, `yarn test` for js
## Documentation Changes
None required
Fixes#1035
## Description of changes
*Summarize the changes made by this PR.*
- Improvements & Bug fixes
- Default persistent directory isn't `/index_data` it's located at
`/chroma/chroma/`
[config.py#L85](https://github.com/chroma-core/chroma/blob/main/chromadb/config.py#L85)
fixing it in serveur core docker-compose
- Removing misleading volume `/index_data` in main docker-compose.yaml
file, that clearly induce in error users
- Adding comment in main docker-compose.yaml file so that Chroma users
won't waste time seeking for this directory path
- New functionality
Does it introduce a breaking change ?
- With this change data will now be persistent for users using
`docker-compose.server.example.yml` as it's a exemple file this
shouldn't be considered as a breaking change.
## Test plan
*How are these changes tested?*
- [x] Tests pass locally with `pytest` for python, `yarn test` for js
- [x] Start Chroma server with `docker-compose up -d` create a
collection, stop Chroma server with `docker-compose down` and start it
again `docker-compose up -d`, check the created collection still exists.
## Documentation Changes
*Are all docstrings for user-facing APIs updated if required? Do we need
to make documentation changes in the [docs
repository](https://github.com/chroma-core/docs)?*
Refs: #1027
## Description of changes
*Summarize the changes made by this PR.*
- New functionality
- Baseline functionality and tests implemented
- Example notebook updated
- Minor refactor on the client creds provider to allow for user specific
credentials fetching.
## Test plan
*How are these changes tested?*
- [x] Tests pass locally with `pytest` for python (regression)
- [x] New fixtures added for token-based auth
## Documentation Changes
Docs should be updated to highlight the new supported auth method.
## Description of changes
*Summarize the changes made by this PR.*
- Improvements & Bug fixes
- The docker-compose will now pick up ENV vars from supplied
`--env-file`
## Test plan
Manual tests done for both with and without env file scenarios
## Documentation Changes
Docs updated accordingly
## Description of changes
*Summarize the changes made by this PR.*
- New functionality
- Auth Provide Client and Server Side Abstractions
- Basic Auth Provider
## Test plan
Unit tests for authorized endpoints
## Documentation Changes
Docs should change to describe how to use auth providers on the client
and server. CIP added in `docs/`
## Description of changes
Base PR to release sqlite refactor, which spans many stacked PRs.
Remaining
- [x] Merge this to main
- [x] Layered Persistent Index #761
- [x] Remove old impls (In #781 )
- [x] Remove persist() API (In #787)
- [x] Add telemetry to SegmentAPI, it was not included. (#788)
- [x] New clients #805
- [x] locking and soak tests for thread-safety
- [x] Migration tool
- [x] Fix#739
- [x] Fix metadata None vs empty
- [x] Fix persist directory (addressed in #761)
- [x] Leave files open in #761 (merge stacked PR)
Post Release
- [ ] Un xfail cross version tests once we cut the release
- [x] Documentation updates for new silent ADD failure.
- [x] Update all documentation for new API instantiation
- [x] Update all documentation for settings changes
- [ ] Update terraform deployment
- [ ] Update cloudformation deployment
---------
Co-authored-by: Luke VanderHart <luke@vanderhart.net>
Co-authored-by: Jeffrey Huber <jeff@trychroma.com>
Co-authored-by: Anton Troynikov <atroyn@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Sebastian Sosa <37946988+CakeCrusher@users.noreply.github.com>
Co-authored-by: Russell Pollari <russell@sharpestminds.com>
Co-authored-by: russell-pollari <pollarir@mgail.com>
## Description of changes
Clickhouse data was being saved in an anonymous volume. Anonymous
volumes are not removed by default. However, as they don’t have a stable
name, they will not be automatically mounted by a subsequent `docker
compose up`. For data that needs to persist between updates, use
explicit paths as bind mounts or named volumes.
https://docs.docker.com/engine/reference/commandline/compose_down/#usage
The yaml in the repo makes it look like
clickhouse_data:/bitnami/clickhouse is where the data is supposed to go
but that isn't what's happening.
Modified Dockerfile to use multistage builder pattern to shrink the
docker image. The new image is 800MB smaller than the official chroma
image.
## Test plan
Build image locally. Unit tests all passed. Data persists after bringing
compose stack down and up. No actual chroma code change in this PR.
## Documentation Changes
None required.
---------
Co-authored-by: perry <perryaws2021@protonmail.com>
Co-authored-by: Jeffrey Huber <jeff@trychroma.com>
