ML library
Replicate + Hugging Face
Templates for every major Replicate category and many Hugging Face pipeline types. Use in chat, call the ML API, or chain with the data lake.
Env
REPLICATE_API_TOKEN= HUGGINGFACE_API_KEY= # or HF_TOKEN ALLOW_DEMO_PIPELINE_ANON=1 # local dry-run without login
API
# List categories + tasks
GET /api/openchat/ml
# Replicate text-to-image
POST /api/openchat/ml
{ "provider": "replicate", "category": "text-to-image",
"input": { "prompt": "OpenChat logo, minimal" } }
# Hugging Face summarization
POST /api/openchat/ml
{ "provider": "huggingface", "task": "summarization",
"inputs": "Long text…" }
# Override model (any Hub / Replicate id)
{ "provider": "huggingface", "task": "text-generation",
"model": "meta-llama/Meta-Llama-3-8B-Instruct",
"inputs": "Explain inflation" }Replicate categories (17)
| Category | Default model | Agent slug |
|---|---|---|
| Text → Image | black-forest-labs/flux-schnell | replicate-text-to-image |
| Image → Image | black-forest-labs/flux-kontext-pro | replicate-image-to-image |
| Upscale / Enhance | nightmareai/real-esrgan | replicate-image-upscale |
| Background removal | cjwbw/rembg | replicate-background-removal |
| Face restore | tencentarc/gfpgan | replicate-face-restore |
| Text → Video | minimax/video-01 | replicate-text-to-video |
| Image → Video | stability-ai/stable-video-diffusion | replicate-image-to-video |
| Video → Video | lucataco/animate-diff | replicate-video-to-video |
| Lip-sync / Talking head | devxpy/cog-wav2lip | replicate-lip-sync |
| Text → Speech (Replicate) | jaaari/kokoro-82m | replicate-text-to-speech |
| Music generation | meta/musicgen | replicate-music |
| Speech → Text | openai/whisper | replicate-speech-to-text |
| Language / LLM | meta/meta-llama-3-8b-instruct | replicate-language |
| Vision caption | salesforce/blip | replicate-vision-caption |
| Object detection | yorickvp/llava-13b | replicate-vision-detect |
| 3D generation | firtoz/trellis | replicate-3d |
| Embeddings | andreasjansson/clip-features | replicate-embeddings |
Hugging Face tasks (22)
| Task | Default model | Agent slug |
|---|---|---|
| Text generation (LLM) | mistralai/Mistral-7B-Instruct-v0.3 | huggingface-text-generation |
| Summarization | facebook/bart-large-cnn | huggingface-summarization |
| Translation | Helsinki-NLP/opus-mt-en-es | huggingface-translation |
| Text classification / sentiment | distilbert-base-uncased-finetuned-sst-2-english | huggingface-text-classification |
| Zero-shot classification | facebook/bart-large-mnli | huggingface-zero-shot-classification |
| NER / token classification | dslim/bert-base-NER | huggingface-token-classification |
| Extractive QA | deepset/roberta-base-squad2 | huggingface-question-answering |
| Fill-mask | bert-base-uncased | huggingface-fill-mask |
| Sentence similarity | sentence-transformers/all-MiniLM-L6-v2 | huggingface-sentence-similarity |
| Embeddings / features | sentence-transformers/all-mpnet-base-v2 | huggingface-feature-extraction |
| Text → Image | black-forest-labs/FLUX.1-schnell | huggingface-text-to-image |
| Image classification | google/vit-base-patch16-224 | huggingface-image-classification |
| Object detection | facebook/detr-resnet-50 | huggingface-object-detection |
| Image segmentation | facebook/detr-resnet-50-panoptic | huggingface-image-segmentation |
| Image captioning | Salesforce/blip-image-captioning-base | huggingface-image-to-text |
| Visual QA | dandelin/vilt-b32-finetuned-vqa | huggingface-visual-question-answering |
| Document QA | impira/layoutlm-document-qa | huggingface-document-question-answering |
| Speech recognition (ASR) | openai/whisper-large-v3 | huggingface-automatic-speech-recognition |
| Audio classification | MIT/ast-finetuned-audioset-10-10-0.4593 | huggingface-audio-classification |
| Text → Speech | espnet/kan-bayashi_ljspeech_vits | huggingface-text-to-speech |
| Table QA | google/tapas-base-finetuned-wtq | huggingface-table-question-answering |
| Conversational | facebook/blenderbot-400M-distill | huggingface-conversational |
Any Inference-compatible Hub model can override the default via the model field — that is how we connect “many many” models without one file per weight.
Grow further
- Add a Replicate preset in
connectors/replicate.ts→ auto agent in registry - Add an HF task/model in
connectors/huggingface.ts - Bundle multi-step agents in
registry/ml-models.ts