Start a new crawling session

Initiates a crawling session with the specified URL and optional filters for paths.

Authorizations

x-api-key

string

header

required

Headers

x-api-key

string

required

API key for authentication

Body

application/json

url

string

required

URL to start crawling

maxDepth

integer

Maximum depth to traverse during the crawl

maxPages

integer

Maximum number of pages to crawl

pathsFilterIn

string[]

List of path patterns to include in the crawl

pathsFilterOut

string[]

List of path patterns to exclude from the crawl

proxyCountry

string

country flag code to assign to proxy configuration to navigate behind the proxy

strictDomain

boolean

Set this flag to true when you want the crawler to stay on the same exact domain, not going to subdomains

timeout

integer

Maximum time for crawling in milliseconds

Response

200 - application/json

error

string | null

Null if no error occurred

message

string

sessionId

string

Unique ID for the started crawl session