Trigger crawler tasks manually
Execute the orchestrator to automatically crawl sites
Process the queue of pending pages to crawl
Discover new domains and insert/update them in Sites
Run brain analysis on sites
Classify site categories
Detect duplicate pages
Update provider usage metrics
Generate multimodal embeddings (text + images) for posts and crawler items, with optional image captions for enriched search
Upload a SEMrush categories CSV for a site to extract and classify landing page categories