|
@@ -1,2 +1,147 @@
|
|
|
-# cog
|
|
|
+# Cog: Containers for machine learning
|
|
|
+
|
|
|
+Cog is an open-source tool that lets you package machine learning models in a standard, production-ready container.
|
|
|
+
|
|
|
+You can deploy your packaged model to your own infrastructure, or to [Replicate](https://replicate.com/).
|
|
|
+
|
|
|
+## Highlights
|
|
|
+
|
|
|
+- 📦 **Docker containers without the pain.** Writing your own `Dockerfile` can be a bewildering process. With Cog, you define your environment with a [simple configuration file](#how-it-works) and it generates a Docker image with all the best practices: Nvidia base images, efficient caching of dependencies, installing specific Python versions, sensible environment variable defaults, and so on.
|
|
|
+- 🤬️ **No more CUDA hell.** Cog knows which CUDA/cuDNN/PyTorch/Tensorflow/Python combos are compatible and will set it all up correctly for you.
|
|
|
+- ✅ **Define the inputs and outputs for your model with standard Python.** Then, Cog generates an OpenAPI schema and validates the inputs and outputs with Pydantic.
|
|
|
+- 🎁 **Automatic HTTP prediction server**: Your model's types are used to dynamically generate a RESTful HTTP API using [FastAPI](https://fastapi.tiangolo.com/).
|
|
|
+- 🥞 **Automatic queue worker.** Long-running deep learning models or batch processing is best architected with a queue. Cog models do this out of the box. Redis is currently supported, with more in the pipeline.
|
|
|
+- ☁️ **Cloud storage.** Files can be read and written directly to Amazon S3 and Google Cloud Storage. (Coming soon.)
|
|
|
+- 🚀 **Ready for production.** Deploy your model anywhere that Docker images run. Your own infrastructure, or [Replicate](https://replicate.com).
|
|
|
+
|
|
|
+## How it works
|
|
|
+
|
|
|
+Define the Docker environment your model runs in with `cog.yaml`:
|
|
|
+
|
|
|
+```yaml
|
|
|
+build:
|
|
|
+ gpu: true
|
|
|
+ system_packages:
|
|
|
+ - "libgl1-mesa-glx"
|
|
|
+ - "libglib2.0-0"
|
|
|
+ python_version: "3.8"
|
|
|
+ python_packages:
|
|
|
+ - "torch==1.8.1"
|
|
|
+predict: "predict.py:Predictor"
|
|
|
+```
|
|
|
+
|
|
|
+Define how predictions are run on your model with `predict.py`:
|
|
|
+
|
|
|
+```python
|
|
|
+from cog import BasePredictor, Input, Path
|
|
|
+import torch
|
|
|
+
|
|
|
+class Predictor(BasePredictor):
|
|
|
+ def setup(self):
|
|
|
+ """Load the model into memory to make running multiple predictions efficient"""
|
|
|
+ self.model = torch.load("./weights.pth")
|
|
|
+
|
|
|
+ # The arguments and types the model takes as input
|
|
|
+ def predict(self,
|
|
|
+ image: Path = Input(description="Grayscale input image")
|
|
|
+ ) -> Path:
|
|
|
+ """Run a single prediction on the model"""
|
|
|
+ processed_image = preprocess(image)
|
|
|
+ output = self.model(processed_image)
|
|
|
+ return postprocess(output)
|
|
|
+```
|
|
|
+
|
|
|
+Now, you can run predictions on this model:
|
|
|
+
|
|
|
+```console
|
|
|
+$ cog predict -i @input.jpg
|
|
|
+--> Building Docker image...
|
|
|
+--> Running Prediction...
|
|
|
+--> Output written to output.jpg
|
|
|
+```
|
|
|
+
|
|
|
+Or, build a Docker image for deployment:
|
|
|
+
|
|
|
+```console
|
|
|
+$ cog build -t my-colorization-model
|
|
|
+--> Building Docker image...
|
|
|
+--> Built my-colorization-model:latest
|
|
|
+
|
|
|
+$ docker run -d -p 5000:5000 --gpus all my-colorization-model
|
|
|
+
|
|
|
+$ curl http://localhost:5000/predictions -X POST \
|
|
|
+ -H 'Content-Type: application/json' \
|
|
|
+ -d '{"input": {"image": "https://.../input.jpg"}}'
|
|
|
+```
|
|
|
+
|
|
|
+<!-- NOTE (bfirsh): Development environment instructions intentionally left out of readme for now, so as not to confuse the "ship a model to production" message.
|
|
|
+
|
|
|
+In development, you can also run arbitrary commands inside the Docker environment:
|
|
|
+
|
|
|
+```console
|
|
|
+$ cog run python train.py
|
|
|
+...
|
|
|
+```
|
|
|
+
|
|
|
+Or, [spin up a Jupyter notebook](docs/notebooks.md):
|
|
|
+
|
|
|
+```console
|
|
|
+$ cog run -p 8888 jupyter notebook --allow-root --ip=0.0.0.0
|
|
|
+```
|
|
|
+
|
|
|
+-->
|
|
|
+
|
|
|
+## Why are we building this?
|
|
|
+
|
|
|
+It's really hard for researchers to ship machine learning models to production.
|
|
|
+
|
|
|
+Part of the solution is Docker, but it is so complex to get it to work: Dockerfiles, pre-/post-processing, Flask servers, CUDA versions. More often than not the researcher has to sit down with an engineer to get the damn thing deployed.
|
|
|
+
|
|
|
+[Andreas](https://github.com/andreasjansson) and [Ben](https://github.com/bfirsh) created Cog. Andreas used to work at Spotify, where he built tools for building and deploying ML models with Docker. Ben worked at Docker, where he created [Docker Compose](https://github.com/docker/compose).
|
|
|
+
|
|
|
+We realized that, in addition to Spotify, other companies were also using Docker to build and deploy machine learning models. [Uber](https://eng.uber.com/michelangelo-pyml/) and others have built similar systems. So, we're making an open source version so other people can do this too.
|
|
|
+
|
|
|
+Hit us up if you're interested in using it or want to collaborate with us. [We're on Discord](https://discord.gg/replicate) or email us at [team@replicate.com](mailto:team@replicate.com).
|
|
|
+
|
|
|
+## Prerequisites
|
|
|
+
|
|
|
+- **macOS, Linux or Windows 11**. Cog works on macOS, Linux and Windows 11 with [WSL 2](docs/wsl2/wsl2.md)
|
|
|
+- **Docker**. Cog uses Docker to create a container for your model. You'll need to [install Docker](https://docs.docker.com/get-docker/) before you can run Cog.
|
|
|
+
|
|
|
+## Install
|
|
|
+
|
|
|
+<a id="upgrade"></a>
|
|
|
+
|
|
|
+If you're using macOS, you can install Cog using Homebrew:
|
|
|
+
|
|
|
+```console
|
|
|
+brew install cog
|
|
|
+```
|
|
|
+
|
|
|
+You can also download and install the latest release of Cog
|
|
|
+directly from GitHub by running the following commands in a terminal:
|
|
|
+
|
|
|
+```console
|
|
|
+sudo curl -o /usr/local/bin/cog -L "https://github.com/replicate/cog/releases/latest/download/cog_$(uname -s)_$(uname -m)"
|
|
|
+sudo chmod +x /usr/local/bin/cog
|
|
|
+```
|
|
|
+
|
|
|
+Alternatively, you can build Cog from source and install it with these commands:
|
|
|
+
|
|
|
+```console
|
|
|
+make
|
|
|
+sudo make install
|
|
|
+```
|
|
|
+
|
|
|
+## Next steps
|
|
|
+
|
|
|
+- [Get started with an example model](docs/getting-started.md)
|
|
|
+- [Get started with your own model](docs/getting-started-own-model.md)
|
|
|
+- [Using Cog with notebooks](docs/notebooks.md)
|
|
|
+- [Using Cog with Windows 11](docs/wsl2/wsl2.md)
|
|
|
+- [Take a look at some examples of using Cog](https://github.com/replicate/cog-examples)
|
|
|
+- [Deploy models with Cog](docs/deploy.md)
|
|
|
+- [`cog.yaml` reference](docs/yaml.md) to learn how to define your model's environment
|
|
|
+- [Prediction interface reference](docs/python.md) to learn how the `Predictor` interface works
|
|
|
+- [HTTP API reference](docs/http.md) to learn how to use the HTTP API that models serve
|
|
|
|