The composable
operations of AI.

Convert text into dense vector representations for semantic search, clustering, and retrieval.

• OpenAI text-embedding-3-largeapi
• Cohere embed-v3api
• Voyage AI voyage-3api
• BGE-M3open-source
• + 2 more on the block page

Transform, generate, or reason about text. The core building block for chatbots, summarization, translation, and more.

• GPT-4oapi
• Claude 3.5 Sonnetapi
• Gemini 1.5 Proapi
• Llama 3.1 405Bopen-source
• + 2 more on the block page

Generate images from text descriptions. Powers creative tools, marketing, and synthetic data.

• DALL-E 3api
• Midjourneyapi
• Stable Diffusion 3open-source
• FLUX.1open-source
• + 1 more on the block page

Convert text to natural-sounding speech. Powers voice assistants, audiobooks, and accessibility features.

• ElevenLabsapi
• OpenAI TTSapi
• Coqui XTTSopen-source
• Barkopen-source

Generate 3D models from text descriptions. Enables rapid prototyping and creative 3D content generation.

• Shap-Eopen-source
• Point-Eopen-source
• MVDreamopen-source
• Meshyapi
• + 1 more on the block page

Generate videos from text descriptions. The frontier of generative AI for content creation.

• Soraapi
• Runway Gen-3api
• Klingapi
• CogVideoXopen-source
• + 1 more on the block page

Classify text into predefined categories. Powers spam detection, sentiment analysis, topic categorization, and content moderation.

• SetFitopen-source
• DistilBERTopen-source
• DeBERTa-v3open-source
• BART-large-mnliopen-source
• + 1 more on the block page

Translate text between languages. Essential for global communication, localization, and cross-lingual applications.

• NLLB-200open-source
• Google Cloud Translationapi
• DeepLapi
• MADLAD-400open-source
• + 1 more on the block page

Condense long documents into concise summaries. Essential for news aggregation, research, and document processing.

• BART-large-cnnopen-source
• Pegasusopen-source
• LongT5open-source
• Claudeapi
• + 1 more on the block page

Answer questions based on context or knowledge. Foundation for chatbots, search, and knowledge systems.

• RoBERTa-SQuADopen-source
• DPR (Dense Passage Retrieval)open-source
• FiD (Fusion-in-Decoder)open-source
• Perplexity APIapi
• + 1 more on the block page

Extract named entities (people, organizations, locations, dates) from text. Key for information extraction and knowledge graphs.

• spaCyopen-source
• GLiNERopen-source
• BERT-NERopen-source
• Flairopen-source
• + 1 more on the block page

Re-score retrieved passages with a cross-encoder to boost search precision.

• Cohere Rerankapi
• BGE-Rerankeropen-source
• monoT5open-source

Detect and redact personally identifiable information to stay compliant.

• Microsoft Presidioopen-source
• spaCy PII Pipelinesopen-source
• AWS Comprehend PIIapi

Score or flag generated text for factuality and grounding.

• RAGASopen-source
• SelfCheckGPTopen-source
• G-Evalopen-source

Generate text with constraints on style, length, structure, or safety guardrails.

• Guidance/Outlinesopen-source
• Guardrails AIopen-source
• NeMo Guardrailsopen-source

Summarize 100K+ token inputs like transcripts, hearings, or books with structured outputs.

• Gemini 1.5 Proapi
• Claude 3.5 Sonnet 200Kapi
• LLama 3.1 70B 128Kopen-source

Generate, refactor, or fix code with language models specialized for programming.

• GPT-4o (Code)api
• DeepSeek-Coder-V2open-source
• CodeLlama 70B Instructopen-source

Combine lexical (BM25) and dense retrieval with weighted fusion or cascades to improve recall and precision for search and RAG.

• Elasticsearch + ELSER/BM25open-source
• Pyserini + Faissopen-source
• Weaviate Hybridopen-source

Convert images directly to dense vector representations for semantic search, clustering, and similarity matching.

• OpenAI CLIPopen-source
• SigLIPopen-source
• OpenCLIPopen-source
• DINOv2open-source

Generate natural language descriptions of image content. Enables text-based search over visual content.

• GPT-4 Visionapi
• Claude 3.5 Sonnetapi
• LLaVAopen-source
• BLIP-2open-source
• + 1 more on the block page

Locate and classify objects in images with bounding boxes. Foundational for autonomous vehicles, surveillance, and robotics.

• YOLOv8/YOLOv11open-source
• RT-DETRopen-source
• Grounding DINOopen-source
• Florence-2open-source
• + 1 more on the block page

Classify each pixel in an image. Enables precise object boundaries for medical imaging, autonomous vehicles, and image editing.

• Segment Anything (SAM)open-source
• SAM 2open-source
• Mask2Formeropen-source
• YOLOv8-segopen-source
• + 1 more on the block page

Predict depth from a single image. Critical for 3D reconstruction, AR/VR, and robotics.

• Depth Anything V2open-source
• MiDaSopen-source
• ZoeDepthopen-source
• Marigoldopen-source

Generate 3D models from single or multiple images. Powers 3D asset creation, VR/AR, and e-commerce.

• TripoSRopen-source
• LGM (Large Gaussian Model)open-source
• InstantMeshopen-source
• CSM (Common Sense Machines)api
• + 1 more on the block page

Animate still images into videos. Bring photos to life with natural motion.

• Stable Video Diffusionopen-source
• Runway Gen-3 Alphaapi
• Klingapi
• Pikaapi

Answer natural language questions about images. Combines vision and language understanding.

• GPT-4Vapi
• Claude 3.5 Sonnetapi
• LLaVAopen-source
• Qwen-VLopen-source
• + 1 more on the block page

Transform images: style transfer, inpainting, super-resolution, editing, or generation from image prompts.

• Stable Diffusion XLopen-source
• ControlNetopen-source
• InstructPix2Pixopen-source
• Real-ESRGANopen-source
• + 1 more on the block page

Detect and read text in images and documents. Core for document intake, receipts, and scene text search.

• PaddleOCRopen-source
• TrOCRopen-source
• Tesseractopen-source

Detect human or object keypoints. Enables AR overlays, sports analytics, and motion capture.

• RTMPose (MMPose)open-source
• MoveNetopen-source
• OpenPoseopen-source

Estimate pixel-wise motion between frames. Useful for video editing, stabilization, and robotics.

• RAFTopen-source
• GMFlowopen-source
• FlowFormeropen-source

Segment foreground and remove or replace backgrounds for product photos and portraits.

• MODNetopen-source
• U^2-Netopen-source
• Segment Anythingopen-source

Blur, mask, or re-synthesize faces to protect privacy in images and video frames.

• DeepPrivacy2open-source
• YOLOv8-face + OpenCVopen-source
• BriaRMBG + Bluropen-source

Parse charts, diagrams, and tables into structured data for analysis and QA.

• Table Transformer (TATR)open-source
• DocTRopen-source
• ChartQA Modelsopen-source

Transcribe spoken audio into text. The foundation for voice interfaces, meeting transcription, and audio search.

• OpenAI Whisper APIapi
• Whisper (local)open-source
• Deepgramapi
• AssemblyAIapi
• + 2 more on the block page

Classify audio into categories like music genres, environmental sounds, speaker emotions, or speech commands.

• Audio Spectrogram Transformer (AST)open-source
• Wav2Vec2open-source
• CLAPopen-source
• YAMNetopen-source
• + 1 more on the block page

Detect when speech is present in audio. Essential preprocessing for ASR, diarization, and voice interfaces.

• Silero VADopen-source
• WebRTC VADopen-source
• pyannote VADopen-source
• Speechbrain VADopen-source

Transform audio signals: enhance, denoise, separate sources, change voice, or convert music styles.

• Demucsopen-source
• RVC (Retrieval Voice Conversion)open-source
• so-vits-svcopen-source
• DeepFilterNetopen-source
• + 1 more on the block page

Separate 'who spoke when' in audio. Vital for meetings, call centers, and transcription QA.

• pyannote.audioopen-source
• NVIDIA NeMo Diarizationopen-source
• Resemblyzeropen-source

Detect wake words and short commands with low latency and tiny footprints.

• Google Speech Commands KWSopen-source
• openWakeWordopen-source
• Picovoice Porcupineapi

Classify speaker emotion or affective state from voice.

• SpeechBrain SERopen-source
• Wav2Vec2-Emotionopen-source
• Emo-CLAPopen-source

Replicate a speaker’s voice or convert one voice to another (TTS-to-TTS).

• RVCopen-source
• so-vits-svcopen-source
• OpenVoiceopen-source

Detect or verify watermarks in synthetic or distributed audio.

• Audiowmarkopen-source
• AudioSeal Detectoropen-source
• Stable Signature (beta)api

Understand and describe video content. Powers video search, summarization, and analysis.

• Gemini 1.5 Proapi
• GPT-4V (with frames)api
• VideoLLaMAopen-source
• InternVideo2open-source
• + 1 more on the block page

Classify actions or activities in video clips for safety, sports, and analytics.

• TimeSformeropen-source
• VideoMAEopen-source
• SlowFastopen-source

Track multiple objects across video frames with consistent identities.

• ByteTrackopen-source
• StrongSORT/BoT-SORTopen-source
• OC-SORTopen-source

Extract on-screen text from video frames for subtitles, broadcast monitoring, and compliance.

• PaddleOCR + FFmpegopen-source
• EasyOCRopen-source
• TrOCR + SAM trackingopen-source

Separate or enhance speech in videos using both audio and lip cues. Improves meeting transcription, TV/movie captioning, and noisy recordings.

• AV-Separation (MS3)open-source
• SpeechSplit + Visual Conditioningopen-source
• NeMo AV-Diarizationopen-source

Extract structured information from documents like PDFs, invoices, forms, and contracts.

• Docling (IBM)open-source
• Unstructured.ioopen-source
• Azure Document Intelligenceapi
• Google Document AIapi
• + 1 more on the block page

Answer questions about document content including text, tables, and layouts. Essential for document AI.

• GPT-4Vapi
• LayoutLMv3open-source
• Donutopen-source
• DocVQA-BERTopen-source
• + 1 more on the block page

Build a complete Retrieval-Augmented Generation system for documents. Parse PDFs and documents, chunk intelligently, embed for semantic search, retrieve relevant context, and generate grounded answers with LLMs.

• LlamaIndexopen-source
• LangChainopen-source
• Haystackopen-source
• RAGFlowopen-source
• + 4 more on the block page

Direct Visual Search

Embed images directly with CLIP/SigLIP, search by text or image query.

Caption + RAG Visual Search

Generate captions for images, embed captions, search via text RAG.

Document RAG Pipeline

Extract text from documents, chunk, embed, retrieve, generate with LLM.

Voice Assistant Pipeline

Speech-to-text, process with LLM, text-to-speech response.