32 lines
1.2 KiB
Markdown
32 lines
1.2 KiB
Markdown
```toml
|
|
content_type = "project"
|
|
title = "Cartman is public!"
|
|
date = "2022 10 20"
|
|
```
|
|
|
|
[Cartman](https://old.doordesk.net/cartman) is trained by combining Microsoft's
|
|
[DialoGPT-medium](https://huggingface.co/microsoft/DialoGPT-medium)
|
|
NLP model (GPT2 model trained on 147M samples of multi-turn dialogue from Reddit) with 17 seasons of
|
|
[South Park](https://southparkstudios.com)
|
|
transcripts.
|
|
|
|
Requests are routed from
|
|
[Nginx](https://nginx.com)
|
|
through
|
|
[WireGuard](https://www.wireguard.com)
|
|
to a
|
|
[Raspberry Pi 4B 8GB](https://www.tomshardware.com/news/raspberry-pi-4-8gb-tested) running
|
|
[FastAPI](https://fastapi.tiangolo.com),
|
|
and the Cartman model using [PyTorch](https://pytorch.org).
|
|
It has enough RAM for more, but the CPU is pretty much at its limit. Expect it to take a few
|
|
seconds, I'm cheap. Sorry(kinda).
|
|
|
|
You can download a Docker image if you'd like to run it on your own hardware for either
|
|
[x86_64](https://old.doordesk.net/files/chatbots_api_x86_64.tar.gz)
|
|
or
|
|
[aarch64](https://old.doordesk.net/files/chatbots_api_aarch64.tar.gz).
|
|
|
|
More info [here](https://github.com/adoyle0/cartman) as well as
|
|
[example scripts](https://github.com/adoyle0/cartman/tree/master/api/test)
|
|
to talk to the docker container.
|
|
|