Scraping Starters

Slot provides powerful CLI commands for scraping starter templates from various sources. These tools help you build and maintain your own collection of templates.

GitHub Scraping

The GitHub scraper finds repositories that match specific criteria and downloads their contents for analysis.

Basic Usage

slot scrape github <token> [options]

Options

-s, --stars <number>

Minimum stars for GitHub repositories

-t, --topics <items>

GitHub topics to search for (comma-separated)

-L, --languages <items>

GitHub languages to search for (comma-separated)

-k, --keywords <items>

NPM keywords to search for (comma-separated)

-b, --batch <size>

Batch size for parallel processing

-d, --delay <ms>

Delay between requests in milliseconds

-r, --rph <limit>

Requests per hour limit

-l, --limit <number>

Maximum number of results to fetch

Examples

# Scrape React starters from GitHub
slot scrape github $GITHUB_TOKEN -s 100 -t react-starter,react-boilerplate -L typescript

# Scrape Next.js templates with rate limiting
slot scrape github $GITHUB_TOKEN -t nextjs,starter -r 100 -d 1000

# Scrape NPM packages with keywords
slot scrape npm -k react-starter,react-boilerplate -l 50

# Scrape NPM packages with rate limiting
slot scrape npm -k nextjs-starter -b 10 -r 100

NPM Scraping

The NPM scraper finds starter template packages based on keywords and download counts.

Basic Usage

slot scrape npm [options]

Options

-k, --keywords <items>

NPM keywords to search for (comma-separated)

-b, --batch <size>

Batch size for parallel processing

-d, --delay <ms>

Delay between requests

-r, --rph <limit>

Requests per hour limit

-l, --limit <number>

Maximum results to fetch

Examples

# Search for React starter kits
slot scrape npm -k react-starter,react-boilerplate

# Find Next.js templates with limits
slot scrape npm -k nextjs-starter -l 50 -r 100

# Search for Vue.js starters in batches
slot scrape npm -k vue-starter,vue-template -b 10

Rate Limiting

Be mindful of API rate limits when scraping. Use the --rph and --delay options to avoid hitting rate limits. For GitHub, authenticate with a token that has appropriate permissions.

Next Steps

After scraping templates, you can: