2022-06-27 19:46:04 +00:00
# min(DALL·E)
2022-06-27 17:38:35 +00:00
2022-06-29 19:24:09 +00:00
[![Open In Colab ](https://colab.research.google.com/assets/colab-badge.svg )](https://colab.research.google.com/github/kuprel/min-dalle/blob/main/min_dalle.ipynb)
[![Replicate ](https://replicate.com/kuprel/min-dalle/badge )](https://replicate.com/kuprel/min-dalle)
2022-06-27 21:26:05 +00:00
2022-07-01 18:06:50 +00:00
This is a minimal implementation of Boris Dayma's [DALL·E Mini ](https://github.com/borisdayma/dalle-mini ) in PyTorch. It has been stripped to the bare essentials necessary for doing inference. The only third party dependencies are numpy and torch.
2022-06-29 17:56:29 +00:00
2022-07-01 01:05:02 +00:00
It currently takes **7.4 seconds** to generate an image with DALL·E Mega with PyTorch on a standard GPU runtime in Colab
2022-06-27 17:51:48 +00:00
2022-07-01 18:35:13 +00:00
The flax model, and the code for coverting it to torch, have been moved [here ](https://github.com/kuprel/min-dalle-flax ).
2022-07-01 18:06:50 +00:00
2022-07-01 22:16:55 +00:00
### Install
2022-06-27 17:51:48 +00:00
2022-07-01 22:52:40 +00:00
```bash
2022-07-01 22:30:22 +00:00
$ pip install min-dalle
```
2022-06-27 21:26:05 +00:00
2022-06-28 00:59:39 +00:00
### Usage
2022-06-27 21:28:40 +00:00
2022-07-01 23:02:49 +00:00
#### Command Line
2022-07-01 22:30:22 +00:00
Use the python script `image_from_text.py` to generate images from the command line.
2022-07-01 22:52:40 +00:00
```bash
2022-07-01 22:50:11 +00:00
$ python image_from_text.py --text='artificial intelligence' --seed=7
```
![Artificial Intelligence ](examples/artificial_intelligence.png )
2022-07-01 22:52:40 +00:00
```bash
2022-07-01 22:50:11 +00:00
$ python image_from_text.py --text='court sketch of godzilla on trial' --mega
```
![Godzilla Trial ](examples/godzilla_on_trial.png )
2022-07-01 23:02:49 +00:00
#### Python
2022-07-01 22:54:05 +00:00
To load a model once and generate multiple times, first initialize `MinDalleTorch`
2022-07-01 22:30:22 +00:00
2022-07-01 22:32:48 +00:00
```python
2022-07-01 22:30:22 +00:00
from min_dalle import MinDalleTorch
model = MinDalleTorch(
is_mega=True,
is_reusable=True,
models_root='./pretrained'
)
2022-06-27 19:46:04 +00:00
```
2022-07-01 23:01:07 +00:00
The required models will be downloaded to `models_root` if they are not already there. After the model has loaded, call `generate_image` with some text and a seed as many times as you want.
2022-07-01 22:54:05 +00:00
2022-07-01 22:50:11 +00:00
```python
image = model.generate_image("a comfy chair that looks like an avocado")
display(image)
2022-06-27 17:38:35 +00:00
```
2022-07-01 22:51:06 +00:00
![Avocado Armchair ](examples/avocado_armchair.png )
2022-06-27 17:38:35 +00:00
2022-07-01 22:50:11 +00:00
```python
2022-07-01 22:52:40 +00:00
image = model.generate_image(
"trail cam footage of gollum eating watermelon",
seed=1
)
2022-07-01 22:50:11 +00:00
display(image)
2022-06-27 17:38:35 +00:00
```
2022-07-01 22:52:40 +00:00
![Gollum Trailcam ](examples/gollum_trailcam.png )