API Documentation

Endpoints

Process Endpoint

POST /api/v1/start/{endpoint_id}/

Headers


{ "x-api-key": "YOUR_API_KEY", "Content-Type": "application/json" }

Request Body


{
  "query": "search query",
  "instructions": "extraction instructions",
  "url_pattern_match": "regex pattern",
  "pattern_exclude": false,
  "results": 5,
  "schema": {
    "field_name": {
      "type": "string|integer|boolean",
      "description": "field description"
    }
  }
}

query Required - The search query to use
instructions Required - Instructions for data extraction
schema Optional - Definition of fields to extract. If not provided, a schema will be automatically generated based on the instructions
url_pattern_match Optional - Regex pattern to filter URLs
pattern_exclude Optional - Whether to exclude or include pattern matches
results Optional - Number of results to process (default: 5)
merge_result Optional - When true, merges keys from multiple results into a single result

Note: These parameters can be set as defaults when creating the endpoint, and overridden in the request body.

Response


{ 
  "status": "success", 
  "message": "Request accepted and being processed", 
  "task_id": "550e8400-e29b-41d4-a716-446655440000",
  "merged_result": {  // Only present if merge_result is true
    "price": "$35,000",
    "model": "Model 3 Standard Range",
    "year": 2024
  }
}

Results will be sent to your webhook endpoints as they are processed. Each URL's results will be sent as a separate webhook request.

Error Responses

401 Missing or invalid API key


{"status": "error", "message": "Invalid API key"}

403 Not authorized for this endpoint


{"status": "error", "message": "Not authorized to access this endpoint"}

404 Endpoint not found


{"status": "error", "message": "Endpoint not found"}

400 Invalid request data


{"status": "error", "message": "Invalid JSON data"}

429 Rate limit exceeded


{"status": "error", "message": "Rate limit exceeded. Please wait 10 seconds."}

500 Server error


{"status": "error", "message": "Server error: {error details}"}

Webhook Integration

Configure a webhook to receive results automatically. Each search result will be processed and sent individually to your webhook endpoint. This is useful for requests that may take longer to process.

Webhook Configuration

Configure your webhook URL in the endpoint settings
Configure custom headers for your webhook (optional)
Choose HTTP method for the webhook (GET, POST, etc.)
Use the task_id from the initial request to track related results

Webhook Payload Format


{ 
  "task_id": "unique-task-id", 
  "success": true, 
  "url": "https://example.com/page", 
  "items": [{ 
    "price": "$35,000",
    "model": "Model 3 Standard Range",
    "year": 2024
  }] 
}

Each webhook response represents one processed URL. Multiple webhook requests may be sent for a single API request, all sharing the same task_id.

Webhook Response

Webhook expects a response within 10 seconds
Failed webhook deliveries will be logged but not retried
Each webhook can have its own custom headers

Rate Limits

Plan	Requests per Day	Results per Request
Free Account	10	10
Business Account	Custom	Custom

Fetch Task Results

After submitting a task, you can check its status and results using this endpoint.

Request

          GET /api/endpoint/task/{task_id}/ X-API-Key: your-api-key

Response

The response includes the overall task status and results from each processed URL. Here's a complete example:


{
  "action": null,
  "error_message": null,
  "created_at": "2025-01-11",
  "updated_at": "2025-01-11",
  "task_status": "pending",
  "status": "done",
  "success": true,
  "task_usages": [
    {
      "results_json": {
        "url": "https://www.reddit.com/r/productivity/comments/e9gf6i/avoid_productivity_tool_obsession/",
        "items": [
          {
            "name": "Any.do",
            "purpose": "Task management"
          },
          {
            "name": "Joplin", 
            "purpose": "Note-taking"
          }
        ],
        "cached": true,
        "blocked": false,
        "success": true
      },
      "error_message": null,
      "status": "done",
      "response_code": 200,
      "success": true,
      "action": null,
      "url": "https://www.reddit.com/r/productivity/comments/e9gf6i/avoid_productivity_tool_obsession/",
      "created_at": "2025-01-11",
      "updated_at": "2025-01-11",
      "title": "Avoid Productivity Tool Obsession",
      "domain": "reddit.com",
      "size_mb": 0.42
    },
    {
      "results_json": {
        "url": "https://www.reddit.com/r/VisionPro/comments/1bv2bh2/reviewing_the_vision_pro_with_productivity_and/",
        "items": [
          {
            "name": "Vision Pro",
            "purpose": "Spatial computing as a productivity tool"
          }
        ],
        "cached": false,
        "blocked": false,
        "success": true
      },
      "error_message": null,
      "status": "done",
      "response_code": 200,
      "success": true,
      "action": null,
      "url": "https://www.reddit.com/r/VisionPro/comments/1bv2bh2/reviewing_the_vision_pro_with_productivity_and/",
      "created_at": "2025-01-11",
      "updated_at": "2025-01-11",
      "title": "Reviewing the Vision Pro with productivity and development in mind",
      "domain": "reddit.com",
      "size_mb": 0.89
    },
    {
      "results_json": null,
      "error_message": "Access blocked by robots.txt",
      "status": "error",
      "response_code": 403,
      "success": false,
      "action": null,
      "url": "https://www.example.com/blocked-page",
      "created_at": "2025-01-11",
      "updated_at": "2025-01-11",
      "title": null,
      "domain": "example.com",
      "size_mb": null
    },
    {
      "results_json": null,
      "error_message": null,
      "status": "pending",
      "response_code": null,
      "success": true,
      "action": "visiting",
      "url": "https://www.reddit.com/r/productivity/comments/6qwqko/how_are_you_utilizing_your_phone_as_a/",
      "created_at": "2025-01-11",
      "updated_at": "2025-01-11",
      "title": null,
      "domain": "reddit.com",
      "size_mb": null
    },
    {
      "results_json": null,
      "error_message": null,
      "status": "pending",
      "response_code": 200,
      "success": true,
      "action": "AI Processing",
      "url": "https://www.reddit.com/r/Surface/comments/meleeb/great_problems_using_surface_book_3_135_32gb_as/",
      "created_at": "2025-01-11",
      "updated_at": "2025-01-11",
      "title": "Great problems using Surface Book 3 (13.5\" 32GB) as productivity tool",
      "domain": "reddit.com",
      "size_mb": 0.51
    }
  ]
}

Response Fields:

action - The current action being performed
created_at - When the task was created
updated_at - When the task was last updated
task_status - Overall status of the task (pending, done)
status - Processing status
success - Whether the task completed successfully
task_usages - Array of individual URL processing results:
- results_json - Extracted data based on your schema
- status - Status of this specific URL (pending, done)
- response_code - HTTP response code from the URL
- action - Current action for this URL
- url - The processed URL
- domain - Domain of the processed URL
- size_mb - Size of the processed page in megabytes

Error Responses

401 Missing or invalid API key


{"status": "error", "message": "Invalid API key"}

404 Task not found


{"status": "error", "message": "Task not found"}


import requests

API_KEY = 'your_api_key'
ENDPOINT_ID = 'your-endpoint-id'

# Start processing
response = requests.post(
    f'https://api.autoserp.com/api/v1/start/{ENDPOINT_ID}/',
    json={
        'query': 'tesla model 3 price',
        'instructions': 'Extract the price for each variant',
        'results': 5
    },
    headers={
        'x-api-key': API_KEY,
        'Content-Type': 'application/json'
    }
)

# Get task_id from response
task_id = response.json()['task_id']

# Fetch results
results = requests.get(
    f'https://api.autoserp.com/api/v1/fetch/{task_id}/',
    headers={'x-api-key': API_KEY}
)

print(results.json())


const API_KEY = 'your_api_key';
const ENDPOINT_ID = 'your-endpoint-id';

// Start processing
const response = await fetch(
    `https://api.autoserp.com/api/v1/start/${ENDPOINT_ID}/`,
    {
        method: 'POST',
        headers: {
            'x-api-key': API_KEY,
            'Content-Type': 'application/json'
        },
        body: JSON.stringify({
            query: 'tesla model 3 price',
            instructions: 'Extract the price for each variant',
            results: 5
        })
    }
);

const data = await response.json();
const taskId = data.task_id;

// Fetch results
const results = await fetch(
    `https://api.autoserp.com/api/v1/fetch/${taskId}/`,
    {
        headers: {
            'x-api-key': API_KEY
        }
    }
);

const resultData = await results.json();
console.log(resultData);


# Start processing
curl -X POST https://api.autoserp.com/api/v1/start/your-endpoint-id/ \
  -H "x-api-key: your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "tesla model 3 price",
    "instructions": "Extract the price for each variant",
    "results": 5
  }'

# Fetch results (replace {task_id} with the ID from the previous response)
curl https://api.autoserp.com/api/v1/fetch/{task_id}/ \
  -H "x-api-key: your_api_key"