min-dalle-test/README.md

# min(DALL·E)

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/kuprel/min-dalle/blob/main/min_dalle.ipynb) &nbsp;
[![Replicate](https://replicate.com/kuprel/min-dalle/badge)](https://replicate.com/kuprel/min-dalle)

This is a minimal implementation of Boris Dayma's [DALL·E Mini](https://github.com/borisdayma/dalle-mini) in PyTorch.  It has been stripped to the bare essentials necessary for doing inference.  The only third party dependencies are numpy and torch.

It currently takes **7.4 seconds** to generate an image with DALL·E Mega with PyTorch on a standard GPU runtime in Colab

The flax model, and the code for coverting it to torch, have been moved [here](https://github.com/kuprel/min-dalle-flax).

### Install

```
$ pip install min-dalle
```  

### Usage

Use the python script `image_from_text.py` to generate images from the command line.

To load a model once and generate multiple times, initialize `MinDalleTorch`, then call `generate_image` with some text and a seed.

```
from min_dalle import MinDalleTorch

model = MinDalleTorch(
    is_mega=True, 
    is_reusable=True,
    models_root='./pretrained'
)

image = model.generate_image("court sketch of godzilla on trial", seed=40)
```

Model parameters will be downloaded as needed to the directory specified.  The models can also be manually downloaded [here](https://huggingface.co/kuprel/min-dalle/tree/main).

### Examples

```
python image_from_text.py --text='artificial intelligence' --seed=7
```
![Alien](examples/artificial_intelligence.png)


```
python image_from_text.py --text='a comfy chair that looks like an avocado' --mega --seed=10
```
![Avocado Armchair](examples/avocado_armchair.png)


```
python image_from_text.py --text='court sketch of godzilla on trial' --mega --seed=40
```
![Godzilla Trial](examples/godzilla_on_trial.png)


```
python image_from_text.py --text='trail cam footage of gollum eating watermelon' --mega --seed=1
```
![Gollum Trailcam](examples/gollum_trailcam.png)
simplified 2022-06-27 19:46:04 +00:00			`# min(DALL·E)`
examples 2022-06-27 17:38:35 +00:00
Update README.md 2022-06-29 19:24:09 +00:00			`[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/kuprel/min-dalle/blob/main/min_dalle.ipynb)  `
			`[![Replicate](https://replicate.com/kuprel/min-dalle/badge)](https://replicate.com/kuprel/min-dalle)`
updated readme 2022-06-27 21:26:05 +00:00
moved flax model and conversion code to separate repository 2022-07-01 18:06:50 +00:00			`This is a minimal implementation of Boris Dayma's [DALL·E Mini](https://github.com/borisdayma/dalle-mini) in PyTorch. It has been stripped to the bare essentials necessary for doing inference. The only third party dependencies are numpy and torch.`
simplified attention and keys_values state resulted in decrease in inference time to 7.3 seconds (from ~10 seconds) 2022-06-29 17:56:29 +00:00
update readme 2022-07-01 01:05:02 +00:00			`It currently takes 7.4 seconds to generate an image with DALL·E Mega with PyTorch on a standard GPU runtime in Colab`
readme 2022-06-27 17:51:48 +00:00
cleanup 2022-07-01 18:35:13 +00:00			`The flax model, and the code for coverting it to torch, have been moved [here](https://github.com/kuprel/min-dalle-flax).`
moved flax model and conversion code to separate repository 2022-07-01 18:06:50 +00:00
added to pypi 2022-07-01 22:16:55 +00:00			`### Install`
readme 2022-06-27 17:51:48 +00:00
update readme 2022-07-01 22:30:22 +00:00			```
			`$ pip install min-dalle`
			```
updated readme 2022-06-27 21:26:05 +00:00
updated colab 2022-06-28 00:59:39 +00:00			`### Usage`
readme formatting 2022-06-27 21:28:40 +00:00
update readme 2022-07-01 22:30:22 +00:00			Use the python script `image_from_text.py` to generate images from the command line.

			To load a model once and generate multiple times, initialize `MinDalleTorch`, then call `generate_image` with some text and a seed.

			```
			`from min_dalle import MinDalleTorch`

			`model = MinDalleTorch(`
			`is_mega=True,`
			`is_reusable=True,`
			`models_root='./pretrained'`
			`)`

			`image = model.generate_image("court sketch of godzilla on trial", seed=40)`
			```
readme wording 2022-06-29 14:47:08 +00:00
added to pypi 2022-07-01 22:16:55 +00:00			`Model parameters will be downloaded as needed to the directory specified. The models can also be manually downloaded [here](https://huggingface.co/kuprel/min-dalle/tree/main).`

readme wording 2022-06-29 14:54:01 +00:00			`### Examples`
examples 2022-06-27 17:38:35 +00:00
			```
remove config.json dependency, default to torch in image_from_text.py 2022-07-01 16:03:37 +00:00			`python image_from_text.py --text='artificial intelligence' --seed=7`
simplified 2022-06-27 19:46:04 +00:00			```
updated readme with torch examples 2022-06-29 14:43:46 +00:00			`![Alien](examples/artificial_intelligence.png)`
simplified 2022-06-27 19:46:04 +00:00
fixed an issue with argument parser 2022-06-27 20:49:42 +00:00
simplified 2022-06-27 19:46:04 +00:00			```
remove config.json dependency, default to torch in image_from_text.py 2022-07-01 16:03:37 +00:00			`python image_from_text.py --text='a comfy chair that looks like an avocado' --mega --seed=10`
examples 2022-06-27 17:38:35 +00:00			```
			`![Avocado Armchair](examples/avocado_armchair.png)`


			```
remove config.json dependency, default to torch in image_from_text.py 2022-07-01 16:03:37 +00:00			`python image_from_text.py --text='court sketch of godzilla on trial' --mega --seed=40`
examples 2022-06-27 17:38:35 +00:00			```
update examples 2022-07-01 18:28:51 +00:00			`![Godzilla Trial](examples/godzilla_on_trial.png)`
update readme 2022-07-01 18:19:35 +00:00

			```
			`python image_from_text.py --text='trail cam footage of gollum eating watermelon' --mega --seed=1`
			```
			`![Gollum Trailcam](examples/gollum_trailcam.png)`