|
|
@@ -1,147 +1,21 @@
|
|
|
# Cog: Containers for machine learning
|
|
|
|
|
|
-Cog is an open-source tool that lets you package machine learning models in a standard, production-ready container.
|
|
|
-
|
|
|
-You can deploy your packaged model to your own infrastructure, or to [Replicate](https://replicate.com/).
|
|
|
-
|
|
|
-## Highlights
|
|
|
-
|
|
|
-- 📦 **Docker containers without the pain.** Writing your own `Dockerfile` can be a bewildering process. With Cog, you define your environment with a [simple configuration file](#how-it-works) and it generates a Docker image with all the best practices: Nvidia base images, efficient caching of dependencies, installing specific Python versions, sensible environment variable defaults, and so on.
|
|
|
-- 🤬️ **No more CUDA hell.** Cog knows which CUDA/cuDNN/PyTorch/Tensorflow/Python combos are compatible and will set it all up correctly for you.
|
|
|
-- ✅ **Define the inputs and outputs for your model with standard Python.** Then, Cog generates an OpenAPI schema and validates the inputs and outputs with Pydantic.
|
|
|
-- 🎁 **Automatic HTTP prediction server**: Your model's types are used to dynamically generate a RESTful HTTP API using [FastAPI](https://fastapi.tiangolo.com/).
|
|
|
-- 🥞 **Automatic queue worker.** Long-running deep learning models or batch processing is best architected with a queue. Cog models do this out of the box. Redis is currently supported, with more in the pipeline.
|
|
|
-- ☁️ **Cloud storage.** Files can be read and written directly to Amazon S3 and Google Cloud Storage. (Coming soon.)
|
|
|
-- 🚀 **Ready for production.** Deploy your model anywhere that Docker images run. Your own infrastructure, or [Replicate](https://replicate.com).
|
|
|
-
|
|
|
-## How it works
|
|
|
-
|
|
|
-Define the Docker environment your model runs in with `cog.yaml`:
|
|
|
-
|
|
|
-```yaml
|
|
|
-build:
|
|
|
- gpu: true
|
|
|
- system_packages:
|
|
|
- - "libgl1-mesa-glx"
|
|
|
- - "libglib2.0-0"
|
|
|
- python_version: "3.8"
|
|
|
- python_packages:
|
|
|
- - "torch==1.8.1"
|
|
|
-predict: "predict.py:Predictor"
|
|
|
-```
|
|
|
-
|
|
|
-Define how predictions are run on your model with `predict.py`:
|
|
|
-
|
|
|
-```python
|
|
|
-from cog import BasePredictor, Input, Path
|
|
|
-import torch
|
|
|
-
|
|
|
-class Predictor(BasePredictor):
|
|
|
- def setup(self):
|
|
|
- """Load the model into memory to make running multiple predictions efficient"""
|
|
|
- self.model = torch.load("./weights.pth")
|
|
|
-
|
|
|
- # The arguments and types the model takes as input
|
|
|
- def predict(self,
|
|
|
- image: Path = Input(description="Grayscale input image")
|
|
|
- ) -> Path:
|
|
|
- """Run a single prediction on the model"""
|
|
|
- processed_image = preprocess(image)
|
|
|
- output = self.model(processed_image)
|
|
|
- return postprocess(output)
|
|
|
-```
|
|
|
-
|
|
|
-Now, you can run predictions on this model:
|
|
|
-
|
|
|
-```console
|
|
|
-$ cog predict -i @input.jpg
|
|
|
---> Building Docker image...
|
|
|
---> Running Prediction...
|
|
|
---> Output written to output.jpg
|
|
|
-```
|
|
|
-
|
|
|
-Or, build a Docker image for deployment:
|
|
|
-
|
|
|
-```console
|
|
|
-$ cog build -t my-colorization-model
|
|
|
---> Building Docker image...
|
|
|
---> Built my-colorization-model:latest
|
|
|
-
|
|
|
-$ docker run -d -p 5000:5000 --gpus all my-colorization-model
|
|
|
-
|
|
|
-$ curl http://localhost:5000/predictions -X POST \
|
|
|
- -H 'Content-Type: application/json' \
|
|
|
- -d '{"input": {"image": "https://.../input.jpg"}}'
|
|
|
-```
|
|
|
-
|
|
|
-<!-- NOTE (bfirsh): Development environment instructions intentionally left out of readme for now, so as not to confuse the "ship a model to production" message.
|
|
|
-
|
|
|
-In development, you can also run arbitrary commands inside the Docker environment:
|
|
|
-
|
|
|
-```console
|
|
|
-$ cog run python train.py
|
|
|
-...
|
|
|
-```
|
|
|
-
|
|
|
-Or, [spin up a Jupyter notebook](docs/notebooks.md):
|
|
|
-
|
|
|
-```console
|
|
|
-$ cog run -p 8888 jupyter notebook --allow-root --ip=0.0.0.0
|
|
|
-```
|
|
|
-
|
|
|
--->
|
|
|
-
|
|
|
-## Why are we building this?
|
|
|
-
|
|
|
-It's really hard for researchers to ship machine learning models to production.
|
|
|
-
|
|
|
-Part of the solution is Docker, but it is so complex to get it to work: Dockerfiles, pre-/post-processing, Flask servers, CUDA versions. More often than not the researcher has to sit down with an engineer to get the damn thing deployed.
|
|
|
-
|
|
|
-[Andreas](https://github.com/andreasjansson) and [Ben](https://github.com/bfirsh) created Cog. Andreas used to work at Spotify, where he built tools for building and deploying ML models with Docker. Ben worked at Docker, where he created [Docker Compose](https://github.com/docker/compose).
|
|
|
-
|
|
|
-We realized that, in addition to Spotify, other companies were also using Docker to build and deploy machine learning models. [Uber](https://eng.uber.com/michelangelo-pyml/) and others have built similar systems. So, we're making an open source version so other people can do this too.
|
|
|
-
|
|
|
-Hit us up if you're interested in using it or want to collaborate with us. [We're on Discord](https://discord.gg/replicate) or email us at [team@replicate.com](mailto:team@replicate.com).
|
|
|
-
|
|
|
-## Prerequisites
|
|
|
-
|
|
|
-- **macOS, Linux or Windows 11**. Cog works on macOS, Linux and Windows 11 with [WSL 2](docs/wsl2/wsl2.md)
|
|
|
-- **Docker**. Cog uses Docker to create a container for your model. You'll need to [install Docker](https://docs.docker.com/get-docker/) before you can run Cog.
|
|
|
-
|
|
|
-## Install
|
|
|
-
|
|
|
-<a id="upgrade"></a>
|
|
|
-
|
|
|
-If you're using macOS, you can install Cog using Homebrew:
|
|
|
-
|
|
|
-```console
|
|
|
-brew install cog
|
|
|
```
|
|
|
|
|
|
-You can also download and install the latest release of Cog
|
|
|
-directly from GitHub by running the following commands in a terminal:
|
|
|
-
|
|
|
-```console
|
|
|
-sudo curl -o /usr/local/bin/cog -L "https://github.com/replicate/cog/releases/latest/download/cog_$(uname -s)_$(uname -m)"
|
|
|
+# 安装 Cog
|
|
|
+sudo curl -o /usr/local/bin/cog -L https://github.com/replicate/cog/releases/latest/download/cog_`uname -s`_`uname -m`
|
|
|
sudo chmod +x /usr/local/bin/cog
|
|
|
-```
|
|
|
|
|
|
-Alternatively, you can build Cog from source and install it with these commands:
|
|
|
-
|
|
|
-```console
|
|
|
-make
|
|
|
-sudo make install
|
|
|
-```
|
|
|
+# 克隆模型仓库
|
|
|
+git clone https://github.com/replicate/cog-sdxl
|
|
|
+cd cog-sdxl
|
|
|
|
|
|
-## Next steps
|
|
|
+# 构建镜像
|
|
|
+cog build
|
|
|
|
|
|
-- [Get started with an example model](docs/getting-started.md)
|
|
|
-- [Get started with your own model](docs/getting-started-own-model.md)
|
|
|
-- [Using Cog with notebooks](docs/notebooks.md)
|
|
|
-- [Using Cog with Windows 11](docs/wsl2/wsl2.md)
|
|
|
-- [Take a look at some examples of using Cog](https://github.com/replicate/cog-examples)
|
|
|
-- [Deploy models with Cog](docs/deploy.md)
|
|
|
-- [`cog.yaml` reference](docs/yaml.md) to learn how to define your model's environment
|
|
|
-- [Prediction interface reference](docs/python.md) to learn how the `Predictor` interface works
|
|
|
-- [HTTP API reference](docs/http.md) to learn how to use the HTTP API that models serve
|
|
|
+# 本地运行
|
|
|
+cog predict -i prompt="a cat"
|
|
|
|
|
|
+# 或者启动 API 服务
|
|
|
+cog run -p 5000 python -m cog.server.http
|
|
|
+```
|